To make sure that research included inthe Value of Sport
Monitor is of high quality it is essential to have selection
criteria. Below sets out the criteria used to judge potential
entries.
General criteria
Aims of study
These should be stated clearly and their relationship to wider
theoretical and/or policy issues explained. Key words should be
provided.
Contribution to understanding
The paper should contribute to our understanding of the impact
of sport and sports participation on the areas of interest included
in the monitor. It should be located within an appropriate
literature and contribute to the development of theory and/or
policy. In this regard, where studies simply re-affirm already well
established and widely reviewed impacts, they may not be
included.
Outline of methods
Appropriateness of methods
An explanation should be provided for the choice of methods and
their relevance to the nature of the issue being investigated.
-
Qualitative research
The focus of the study should lend itself to qualitative approaches
and the choice of method (eg participant observation, in-depth
interviews) should be explained. This may, for example, relate to
the development of an understanding of the experience of sports
participation that leads to a better understanding of the processes
leading/not leading to certain subjective/objective outcomes.
-
Quantitative research
In terms of surveys, the choice of data collection methods (eg
self-completion questionnaires, face-to-face interviews, telephone
polling) should be explained. An explanation for the choice of
experimental designs (eg pre and/or post testing; use/non-use of
control groups) should be provided. In both cases the implications
for the data collected should be discussed.
Sampling
-
Qualitative research
Although samples do not need to be representative, nor random,
there is a need for a rationale for selection, the provision of
clear details of the nature and context of the sample and how it
was recruited. Sample size can be justified on the basis of the
aims of the study and the claims the authors wish to make about
their findings and their generalisability.
-
Quantitative research
Where sampling has been used, details should be provided about the
nature of the population from which the sample is drawn, the type
of sampling (eg quota, purposive, simple random, stratified) and
its strengths and weaknesses (this information should also be
provided if secondary analysis of existing data is being
undertaken).
Data analysis
Validity
-
Qualitative research
Comment should be provided as to the steps taken to address issues
of validity – increasing the plausibility of the data and analysis;
triangulation; inter-connectedness of various aspects of the data.
If the researchers checked their interpretations with respondents
details of how this was done and comments dealt with should be
provided.
-
Quantitative research
An explanation should be provided as to how core categories were
chosen and operationalised (eg intensity of participation in sport;
self-esteem, fitness), the extent of conceptual and operational
validity (eg content, criterion and construct) and the implications
for findings.
Reliability
-
Qualitative research
If reliability checks were possible some discussion should be
provided.
-
Quantitative research
Some discussion of the reliability of the
data, attempts to maximise this and the implications for the
generalisability of findings. Sampling errors should be specified
for both full sample analysis and any sub-sample analysis (eg
minority ethnic groups, age categories, individual
sports).
Analysis procedures and processes
The processes and procedures for analysis should be detailed.
The reader should be able to understand the processes or procedures
or steps through which the data analysis evolved.
Statistics
Where measures of association or inferential statistics are used
(for both experimental and survey work), an explanation for their
choice should be provided. Where possible, the use and explanation
of statistics should be as accessible as possible for a
non-academic audience.
Description of analytic framework
An analytic framework should be outlined. In some cases there
may be a pre-existing framework for data analysis and the
derivation of this framework should be explained. If there is no
pre-existing framework then an explanation for the rationale for
the analysis strategy should be provided.
Alternative explanations
Where relevant, the researcher should consider alternative
explanations for the results and possible different interpretations
of the direction of causality
Use of transcript excerpts
When using quotes to highlight findings, the source of the
excerpt should be referenced, and the basis for its selection
explained (eg: are they representative, illustrative etc.)
Findings and discussion
Researcher impact
For qualitative studies the researcher should clearly outline
their role in the research process. They should provide some
comment on their approach and how this might have influenced the
results.
Consistency
The results should seem credible and appropriate, relating back
to the research questions and located within the relevant
literature. Where the paper has been of an exploratory nature, with
the aim of using data analysis to inductively produce theory, then
the pertinent literature can be outlined in the discussion
section.
Theoretical considerations
The claims made for theory development need to be consistent
with the data presented.
Generalisability
Authors should comment on the extent to which their findings can
be generalised, the limitations of the data and the extent to which
it indicates a need for further research.
Utilisation
Ideally the findings should be useful and relevant to the
development of policy. The author will provide added value by
commenting on the policy and/or management implications.