Competency-Based Education in Three Pilot Programs

Photo by michaeljung/Shutterstock

Competency-based education is an approach to schooling that (1) meets students where they are academically through flexible pacing, (2) provides students with opportunities for choice, and (3) awards credit for evidence of proficiency, not for the amount of time students spend studying a subject.

Based in part on Benjamin Bloom's theory of mastery learning — the notion that all students can meet high academic standards if given personalized instruction tailored to their needs — competency-based learning has had a recent resurgence as technological tools have made individualized instruction more feasible. In the past decade, competency-based approaches have shown promise in some rural and alternative school settings, but results have been anecdotal, and fewer attempts have been made to apply this approach in larger, urban settings.

In 2011, the Bill & Melinda Gates Foundation extended grants to three educational organizations working to develop or enhance competency-based approaches in large, urbanized school systems. The grant initiative, called Project Mastery, funded the development of technology-enhanced tools, including curriculum materials and online learning management systems, as well as the implementation of these tools in secondary school classrooms. The Project Mastery grants were awarded to three organizations: Adams County District 50, the Asia Society, and the School District of Philadelphia. Each organization emphasized different features of competency-based education (see sidebar). Still, all focused their grant resources on curriculum development and on creating or upgrading online learning management systems to suit competency-based education environments.

In late 2011, RAND was invited to evaluate the organizations' competency-based education experiences, including implementation, student experiences, and student outcomes. The evaluation team collected site-visit data during the two grant years — academic years 2011–2012 and 2012–2013 — and analyzed data during the 2013–2014 year.

Key Findings

Implementation Tensions Emerged Across Sites

Credit for out-of-school learning emerged as a tension, with educators varying in their agreement that certain after-school and travel-based learning experiences were equivalent to class assignments or courses.

No common definition of proficiency. Another tension focused on holding all students to a common definition of proficiency, with some teachers disagreeing that students should be evaluated strictly on what they knew and not on the effort and practice put toward learning.

Technology challenges. The sites also faced technological challenges in implementing learning management systems that were in beta version.

Financial and logistical sustainability questions arose as well — in some cases, sites lacked the computer hardware to ensure student access to online instructional tools; in other cases, developers partnering with sites placed cost-prohibitive limits on use of the new curricula.

Equity challenges in implementing competency-based education were also noted as a key tension. Especially in sites emphasizing flexible pacing and proficiency-based evaluation, some teachers reported slow progress among students who struggled academically.

Photo by ryflip/Fotolia

Students' Experiences of Competency-Based Education Appeared More Similar Across Sites Than Expected

Student choice. In Project Mastery sites that emphasized student choice, students were only slightly more likely than their peers in sites without this emphasis to report they that had choices of topics and materials.

Pacing. The proportion of students who reported working on different material and at different paces than their classmates was nearly as high in the sites that favored synchronous learning as in those that emphasized flexible pacing.

Enjoyment of learning. In most surveyed sites, fewer than half of students agreed that they liked how learning occurred in their surveyed class and that learning was enjoyable. The exceptions were two Asia Society sites, Denver Center for International Studies (DCIS) in Denver, Colo., and Sharpstown International School (Sharpstown) in Houston, Tex., where more than two-thirds of students reported enjoyment of learning in the surveyed class.

Engagement. By far the highest reports of engagement, flexible pacing, and choice came from DCIS, an Asia Society site in which surveyed students were focused on yearlong, self-directed projects that applied academic content to real-world contexts.

Student Performance Data Favored Sites That Emphasized Student Choice, but Estimates Are Correlational, Not Causal

Adams County District 50. The conversion of this district in Westminster, Colo., to a competency-based model was associated with lower mathematics performance on the state accountability test than the performance of similar schools in the state would have predicted. The difference was equivalent to about 9 percentage points in mathematics for the 2012–2013 school year, a statistically significant difference.

Asia Society. Among the Asia Society sites, the conversion to a competency-based model at Newfound Regional High School in Bristol, N.H., was associated with an increase in reading performance of about 4 percentage points, but the difference was not statistically significant. In two urban schools of choice (DCIS and Sharpstown), students had higher proficiency rates than demographically similar schools in their respective states by 18 and 17 percentage points. The other Asia Society school, Denver Center for International Studies–Montbello, which was also a school of choice, performed about the same as demographically similar schools.

Philadelphia. In the School District of Philadelphia, adjusting for student demographics and prior performance, Project Mastery students performed below their peers in language arts after three months of exposure and about the same as their peers after five months of exposure; they also had similar rates of promotion to tenth grade. Yet their ninth-grade attendance rates were 1.4 percentage points higher than those of their peers — a statistically significant difference. For Philadelphia Project Mastery students who also took part in an afterschool badge program, both attendance rates and promotion rates were higher than those of non–Project Mastery students by statistically significant margins of 3.0 and 6.6 percentage points, respectively.

The analysis of student outcomes focused on students' exposure to competency-based education models rather than strictly on exposure to Project Mastery–funded tools. Based on these findings, the current evidence favors competency-based models that emphasize student choice and project-based learning more than models that emphasize flexible pacing or proficiency-based grading. However, the research design did not permit researchers to conclude that competency-based education was the cause of any of the estimated effects.

Photo by Kirill Kedrinski/Fotolia

Lessons and Recommendations

Policy

Assess competency-based education programs on a variety of nearer-term and longer-term outcomes. Nearer-term measures include the kinds of outcomes examined in this report, such as accountability test scores, high school attendance rates, and high school persistence rates. Longer-term measures would include high school completion rates as well as postsecondary enrollment, remediation, and persistence rates.

Implement flexibly timed accountability tests to provide better measures of progress than fixed, annual tests. To accommodate flexible pacing of student progress, students in a competency-based evaluation system would be tested for proficiency only after they had been exposed to the tested content and had a chance to master it. This differs from the current practice of testing all same-grade students on the same content at a fixed point during the year.

Partnerships

Negotiate favorable terms and anticipate technical challenges in collaboration with technology developers. Schools and technology developers who agree to help refine or pilot-test a tool, such as a curriculum tool or learning management system, should negotiate licensing terms up front that make scale-up financially feasible for the school if the tool shows promise.

Consider local infrastructure and capacity in collaborations between sites and funders. Grants for software tools should ensure that adequate hardware is in place, grants for hardware should ensure that electrical capacity is in place, and so forth. If infrastructure is a key barrier to competency-based reform, then improvements to infrastructure should be a key funding priority for both the schools and the funders.

Practice

Promote skillful teaching in support of student autonomy. Increased student autonomy calls for skillful teaching that generates student engagement. Rather than reducing the importance of quality classroom instruction, the student-autonomy demands of competency-based education make creative and supportive instruction especially important.

Be alert for widening achievement gaps. Competency-based education systems must be vigilant about equity. Because competency-based education emphasizes academic skill over effort and increases the importance of student self-direction, systems adopting these approaches must be alert for widening achievement gaps and provide extra support to students who appear to be falling behind.