Evaluating Digital Feedback in Primary Maths

Background

Of all possible teacher influences on pupil achievement, teacher feedback is among the most powerful. At the same time, there is great variability in the effect sizes reported in studies examining the impact of feedback, indicating that some forms of feedback are more effective than others.

These studies suggest that computer-assisted feedback or feedback provided in the form of video or audio are among the most effective. Interventions such as the one developed by the Kyra Teaching School Alliance – a partnership of schools in Lincolnshire – which seek to maximise the impact of teachers’ feedback on pupils’ outcome with the use of video thus hold particularly high potential for impact.

The intervention aims to improve teachers’ diagnostic and feedback skills when teaching maths in primary schools. It involves a diagnostic tool which teachers can use to assess their pupils’ understanding of a topic, and an app that allows teachers to provide pupils with digitally-recorded feedback on a tablet, rather than written feedback. Pupils have the opportunity to review their feedback and develop their work further, the rationale behind this being that the video feedback would be more relevant and of higher quality than written feedback and would thereby improve pupil outcomes.

Goals

RAND Europe was commissioned by the Education Endowment Foundation (EEF) to conduct an efficacy trial of Kyra’s digital teacher feedback intervention.

This trial was accompanied by a theory-driven process evaluation to map and understand the causal pathway from intervention to pupil outcomes. The process evaluation assessed whether the steps underlying the intervention were implemented to a sufficient standard and whether the assumptions on which the intervention is based hold.

Methodology

The project team undertook a two-armed randomised controlled trial (treatment and control), focusing on mathematics in 34 primary schools (Years 4 and 5). While the intervention took place in each school, classrooms were randomly assigned to the treatment or control group.

Recruitment of schools began in early 2017, and the intervention began in September 2017. Outcomes were then assessed at the end of the school year in June 2018. The trial protocol and final report can be found on the Education Endowment Foundation project page.

Findings

• There is no evidence that Digital Feedback in Primary Maths has an impact on the maths outcomes of pupils eligible for free school meals.
• The programme, as designed, was not easily implemented by teachers and schools. There was evidence that teachers and research leads sometimes did not attend training, have coaching sessions, or provide regular digital feedback in the classroom.
• Both the diagnostic assessment and training in effective feedback were reported as more useful by teachers than the use of the app to provide digital feedback. Evidence from surveys, Professional Learning Community observations, and an interview indicated that the diagnostic assessments were eventually being used as a global classroom assessment tool.
• There were significant challenges with identifying which pupils were in the treatment group and which were in the control. This meant that 431 pupils of the 2,564 total could not be included in the main analysis. This presented a small threat to the security of the trial.

---

Read the full study