Beyond the mean: Modelling relationships in PISA data for different performance groups

International Large Scale Assessment (ISLA) data, for example the Programme for International Student Assessment (PISA) data, have influenced education policy across the world. When comparing country achievement, analysis has typically focused on mean scores. Our recent work has shown that comparing means alone can mask differences in performance (we found gender differences in achievement scores varied across percentiles – countries with seemingly equal means, masked relatively large differences in performance in their top and bottom deciles). This project seeks to replicate analyses that have been carried out on PISA data, modelling maths, reading and science performance by socio-economic status, gender, immigration status, but splitting the analysis into quartiles, deciles and percentiles by achievement. Whilst the links between socio-economic status, gender, immigration status and academic performance have been much analysed, little is known as to how effects differ by achievement group. Current analyses of achievement may mask differences in performance (as we have found for gender) which represent opportunities for more equitable policy making (we have found boys are over-represented in the highest and lowest achieving percentiles).

We aim to replicate results that have been found in the last three PISA cycles (2022, 2018, 2015).
We aim to examine the following relationships across different achievement groups (i.e. do corelations differ across high and low achieving students) for the three PISA test subjects, mathematics, reading and science:

• The relationship between achievement and wealth / social class
• The relationship between achievement and gender
• The relationship between achievement and immigration status

With support, the research assistant will carry out the following tasks, working on data sets for PISA 2022, 2018, 2015, which we have already cleaned for analysis:

• Group students into a) quartiles, b) deciles, and c) percentiles by achievement
• Run linear regression models for the relationships above, split by achievement: a) quartile; b) decile; c) percentile
• Run structural equation models for the relationships between achievement and the variables above (wealth/ social class, gender, country) for achievement groups
• If time, compare patterns of change over the three cycles
• Blogging their ongoing work on our quantitative methods blog (https://richardabrock.github.io/mastemquantblog/)
• Write a paper summarising the findings of the models
• Present their work to a meeting of the Centre for Research in Education in STEM academic group

• The student would have individual tutorial time with the co-proposers to develop their R skills
• The student would gain experience of the statistical techniques used in the project (which the co-proposers will teach), namely linear regression, logistic regression , and structural equation modelling
• The student will gain familiarity with an important data set – the PISA data
• The student will work towards a quantitative publication

We require some basic knowledge of statistics or statistical analysis tools. We don’t require any experience with R, Structural Equation Modelling or the PISA data set, as we aim to upskill the student in this, but some prior experience would be beneficial The willingness and ability to learn new programming skills.

Through the writing of a journal article, a blog post, and a presentation to our academic group

1. Please send your CV and a brief cover letter outlining your interest and suitability to the project supervisor(s). Please contact the project supervisor(s) in advance of submitting the application with any questions. Please focus on describing your previous experience with quantitative methods – describe data sets you have analysed, the tests you have previously performed, and the packages you are familiar with.

2. If selected by the project supervisor, the student must then complete the Placement /Internship Application form. This ensures that there is approval of PhD supervisor, and the necessary information is obtained to extend funding (for DTP1 students) or confirm placement requirement fulfilled (for DTP2 students), and to fulfil ESRC reporting obligations.  

Please note:

• Research Assistant Internships must not be undertaken with the student’s current supervisor and/or home department.
• DTP1 students (those whose funding commenced before Oct24): a maximum of 4 Research Assistant internships will be funded. These will be filled on a first-come, first-served basis. Once the 4 DTP1 places are filled, we will inform PIs that only DTP2 students are eligible for the Research Assistant internships. PLEASE NOTE THAT ALL DTP1 PLACES HAVE NOW BEEN FILLED.
• DTP2 students (those whose funding commenced from Oct24): are required to complete a 3-month placement, which is funded through their studentship. No limits to number that can be funded.
• Reports: at the conclusion of the internship, the student will be required to complete an internship report, which will include a question for the internship host to feedback on the internship.

Contact liss-dtp@kcl.ac.uk with any questions.