Developing the OECD TALIS Starting Strong Survey

Fotolia

A survey for the OECD aimed to fill key international and national data gaps on Early Childhood Education and Care (ECEC) staff, their professional development, pedagogical practices, the ECEC environment and the impact that staff can have on children in their care.

RAND Europe researchers developed the survey’s conceptual framework, which helped guide the design of the questionnaires and ensure that they were grounded in theory and past evidence.

Background

As part of an international consortium led by the International Association for the Evaluation of Educational Achievement (IEA), RAND Europe and Statistics Canada were commissioned by the Organisation for Economic Cooperation and Development (OECD) to develop, manage and implement the first cycle of the Starting Strong Survey, a survey of staff in early childhood education and care centres that was implemented across 9 participating countries.

The aim of the survey is to help draw international comparisons of strengths and opportunities in Early Childhood Education and Care (ECEC) across different countries, in order to generate policy discussions to re-think the work organisation of ECEC staff and enhance the overall quality of its workforce. RAND Europe was involved in developing the first round of the survey (OECD TALIS Starting Strong 2018) – with future rounds envisioned to increase the body of knowledge in the subject and enable longitudinal analyses.

Goals

RAND Europe was responsible for leading the development of the conceptual framework and questionnaires for the survey, with the aim of filling key international and national data gaps on ECEC staff, their professional development, pedagogical practices, the ECEC environment and the impact that staff can have on children in their care.

Methodology

The survey had three main data collection components: the pilot, field trial and main survey. The main purposes of the pilot phase, which took place in the last quarter of 2016, were to:

  1. trial questionnaire items;
  2. collect information on the international applicability of the items;
  3. gather information on how well the instruments perform in the field for all target populations (core survey and international option); and
  4. fine-tune the questionnaires for the field trial based on analysis of the pilot outcomes.

The objective of the field trial, which took place in mid-2017, was to test the survey instruments and the operational procedures in all participating countries and economies in preparation for the main survey.

The main survey, which took place in 2018, represented the core data collection period for the project. It used the instruments developed and revised in the previous pilot and field trial phases and was implemented according to the guidelines and technical standards agreed by the OECD and participating countries.

Key Outputs

The full questionnaire templates for the OECD TALIS Starting Strong Survey 2018 can be accessed here. In addition, RAND Europe researchers led the development of the survey’s conceptual framework to help guide the design of the questionnaires and ensure that they were grounded in theory and past evidence. The framework helped to ensure that the questions utilised constructs that existing research had identified as having an impact. For example, it explores aspects influencing child development, learning and wellbeing, process and structural quality, and staff well-being, among others.

Researchers also produced a document outlining key analysis principles and high-level steers for content- and context-related issues that need to be taken into consideration when analysing the survey.

The research team also contributed to a technical report detailing the steps, procedures, methodologies, standards and rules that the survey used to collect high-quality data.

Outcomes

A report detailing the outcomes of the surveys can be found on the OECD website.