Analysing the relationship between Sparx Maths and maths outcomes
Time spent using and actively working in Sparx Maths is positively and significantly associated with higher outcomes in maths for all students, but access alone — ignoring usage — did not have any effects.
What is the issue?
The use of digital tools and practices in education – also known as Educational Technology (EdTech) – is undergoing a period of unprecedented growth in the UK, with schools increasingly looking to EdTech to support teaching and learning in and outside the classroom. This growth has been intensified due to the unprecedented need to move to online modes of teaching, as a means of containing the spread of COVID-19.
Evidence from over 40 years of research has shown the potential for EdTech to support learning, when implemented effectively. Researchers at RAND Europe and the University of Cambridge were commissioned by Sparx, a socially focused learning technology company, to explore how usage of their flagship Sparx Maths learning platform is associated with student outcomes. Sparx Maths is an electronic, personalised learning platform developed to aid the delivery of lessons and homework for Key Stage 3 and GCSE students (aged 11 to 16) in the UK.
How did we help?
Researchers undertook an analysis of data collected in 2019 by Sparx from 3,956 Year 7 and Year 8 students across 14 schools in the UK, following a study plan published ahead of the analysis.
RAND Europe supported the development of a Sparx Maths Theory of Change to provide a shared understanding of how Sparx impacts on learning outcomes of interest. Researchers asked the Sparx team to identify key elements of Sparx Maths delivery and contextual factors, as well as isolate key impacts and outcomes. This report provides an overview of the research findings, starting with an overview of the Sparx Maths programme, followed by the key findings from the analyses.
What did we find?
Time spent using Sparx Maths is positively and significantly associated with higher outcomes in maths
- On average, students used Sparx Maths for 36.9 hours, with a range between zero hours (no use at all) and just over 256 hours.
- The use of Sparx Maths for the recommended one hour of homework per week was associated with an increase of almost 20 per cent of a (predicted) GCSE grade.
Time spent actively working in Sparx Maths is positively and significantly associated with higher outcomes in maths
- Sparx Maths records time spent using Sparx Maths (i.e. overall time spent on the platform, including working time and time spent watching tutorials), as well as time spent working on Sparx Maths (i.e. active working on the platform, solving set problems).
- Actively working on Sparx for the recommended one hour of homework per week for one whole school year was associated with an increase of almost 30 per cent of a (predicted) GCSE grade.
There is no evidence that access to Sparx Maths alone (regardless of usage) is associated with maths outcomes
- The analysis compared the 61% of students in the sample who had access to Sparx Maths (with different levels of time spent on the platform) to students without access to Sparx Maths.
- There was no evidence of differences in maths outcomes by access to Sparx Maths when accounting for students’ backgrounds.
Outcomes for specific sub-groups of students follow the same trends as their peers
- Data was also available on students’ memberships of specific sub-groups, in particular English as an Additional Language (EAL), eligibility for free school meals (FSM) and KS2 attainment.
- The impact of Sparx Maths is similar across different subgroups and does not exacerbate any existing gaps for disadvantaged or lower pre-attainment students, although it is also not able, on its own, to close these gaps.
What are the key learnings?
- Researchers studying digital learning should avoid confounding access to digital learning with usage, and instead focus on collecting and analysing active usage as part of their research.
- Further research could be done to strengthen understanding of how Sparx Maths works in practice, including how teachers and schools select students and why schools choose to start Sparx at different times in the year. A study using a pre-established comparison group (such as a randomised controlled trial) would further enhance the robustness of future conclusions.