Checking raw data
Once the final data had been collated and all illegitimate responses removed (for example, insufficiently completed surveys), further checks were undertaken.
All variables were checked to ensure that the number and patterns of response tallied with what was expected based on the survey routing.
Once this was confirmed, further sense-checks were conducted to ensure that the broad pattern of responses made sense against what might be expected.
Checking final survey weights
After the survey weights had been created, checks were conducted to ensure that the weighted profile of respondents matched the weighting targets at school year, gender, and school year within gender, as closely as possible.
All weighted counts were found to match the population distribution when rounded to the nearest whole number.
For measures not used in weighting (e.g. ethnicity, disability), the weighted counts do not match the population. However, checks were conducted to ensure that the weighted profile on these measures was broadly consistent with national data from the DfE pupil population statistics (January 2024).
These checks found the following:
- The weighting corrected for region well.
- The effect on ethnicity is good when looking at the distribution of White British and all other ethnic groups combined. However, our data show that Asian and Black respondents are under-represented compared with White and Mixed ethnicity.
- The weighted and unweighted survey samples for those with long-term limiting disabilities match the population well, with a closer match than the previous disability questions had (disability questions were updated in academic year 2019-20 (Year 3). See the academic year 2019-20 (Year 3) technical note for more details about the changes). Analysis by disability cannot be compared between academic years since 2019-20 (Year 3) and the first two academic years 2017-18 (Year 1) and 2018-19 (Year 2) because of the change to the questions.