Monitoring and Evaluation Support for SPARK’s Jobs Programme

What is the issue?

The Syrian refugee crisis is in its thirteenth year as of 2023 and is currently the world’s second largest displacement of people. Nearly 7 million Syrians have fled to neighboring countries, primarily, Turkey, Jordan, Lebanon, Iraq, and Egypt. In 2018, RAND undertook a detailed study of the labor markets for refugees in Turkey, Jordan, and Lebanon. This study found that despite enormous challenges, the host countries were accommodating the vast influx of refugees. Surveys of firms in these countries found that to varying degrees they were employing refugees, often informally and without needed permits, which were difficult to obtain. One of the main conclusions of the study was that these refugees need not be viewed as a problem as much as an opportunity to bolster host economies through their productive employment and entrepreneurship.

Spark, a Dutch NGO, was influenced by this study, and has been engaged in their Jobs Programme to create jobs for refugees and host communities during the Syrian crisis. They are pursuing a four-prong strategy of: ‘Scale Up’ (an expansion of promising SMEs), ‘Match Up’ (connecting supply of and demand for opportunities), ‘Start Up’ (creating businesses and thereby decent and productive opportunities), and ‘Skill Up’ (increasing employability skills), all aimed at creating jobs, primarily for at-risk youth, especially women.

Spark has experience in and commitment to the monitoring and evaluation (M&E) of the Jobs Programme in order to assess ongoing progress toward the achievement of the four target outcomes. Spark’s M&E intends to use the OECD/DAC Evaluation Criteria to examine the relevance, coherence, effectiveness, efficiency, impact, and sustainability of the intervention. RAND will support M&E efforts of the Spark Jobs Programme by providing assistance to support the development and execution of an M&E system for the programme’s efforts in Turkey (which currently has Scale Up, Match Up, and Start Up activities) and Jordan (which currently has Scale Up, Skill Up, and Start Up activities).

How are we helping?

RAND will work with Spark to assess the M&E system relative to best practices and contribute to the M&E system’s development. In particular, RAND will focus on working with Spark to finalise a comprehensive data collection plan with well-defined metrics and indicators, plan for the collection and storage of data, and establish processes for verifying data veracity and maintaining quality control. The key performance indicators (KPIs) are expected to be different for each of the four pillars of the Jobs Programme. Work in this area will be guided by the programme theory of action, anticipated intermediate results, and programme objectives. RAND will assess the degree to which Spark’s M&E plan adheres to the OECD/DAC evaluation criteria (i.e., relevance, coherence, effectiveness, efficiency, impact, and sustainability), and suggest modifications, as necessary. While cross-country comparisons are aided by keeping the metrics and indicators the same, attention will be paid to adapting them to cultural and other on-the-ground considerations. Similarly, indicators may need to be adapted to or added for different population sub-groups.

Throughout this project, RAND will, as needed, also support Spark’s development and piloting of surveys and other data collection forms that will enable Spark to gather the data from implementation partners to populate its Management Information Systems. RAND will make contributions to the data collection instruments developed by Spark, such as suggesting items based on prior experience with Syrian refugees and businesses in the region and identifying the best subjective scales to include. In addition, RAND will provide input on various aspects of the data collection. This could include input on best practices for data management and analysis, including how to follow up with selected partners and address non-response.

When Spark has begun to collect, clean, and store M&E data, RAND will test a sample of collected data to see if the M&E system is functioning properly. Testing data during this stage will allow for early-phase corrective action if adjustment is needed in order to improve the M&E system. Data to be tested will be drawn from Spark’s Management Information Systems. RAND will assess the degree to which the data meet Spark’s quality data standards (i.e., validity, reliability, precision, integrity and timeliness).

---