Download Free Electronic Document

FormatFile SizeNotes
PDF file 2.8 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief
Close-up of aboriginal students and their tutor sitting outdoors in Australia.

Photo by SolStock/iStock

Project background and overview

Professional accreditation provides an assessment of the quality of training delivered by education providers and a mechanism for accountability against professional standards, including both national and international benchmarking requirements. It ensures that education providers produce graduates who are sufficiently competent to join the workforce and, within the healthcare sector, ensures graduates can perform their roles safely and effectively. Although approaches to this vary, the ultimate goal is always to ensure the public receive safe and high-quality care from practitioners who graduate from accredited programs. However, despite the widespread implementation of professional accreditation, there is a limited evidence base regarding its impacts and how to enhance these.

The Australian Dental Council (ADC) is an independent accreditation authority for the dental professions assigned by the Dental Board of Australia under the National Registration and Accreditation Scheme (NRAS); for the purposes of this report ADC is referred to as a regulator as the accreditation function is a policy lever in health practitioner regulation. ADC commissioned RAND Europe to develop a framework for understanding and evaluating the impact of their dental education program accreditation work as the first step in their larger evaluation plans. The framework will help ADC to understand and measure whether its accreditation-related activities are effective in producing competent and safe dental practitioners. One aspect of this focused on understanding the mechanisms by which ADC's activities and outputs may deliver these desired impacts. This Research Brief outlines the findings from this aspect of the research; for further details please see the final report.

Mechanisms of impact

This research built on the mechanisms of impact described by Smithson et al. (2018) to explore the ways in which ADC ensures public safety through accrediting individual programs of study to produce safe and competent newly qualified practitioners and the broader strategic work that is undertaken to support and improve accreditation standards and processes.

We considered nine mechanisms of impact that operate at different levels:

  • Strategic, directive, relational and stakeholder mechanisms relate to how the regulatory body (e.g. ADC) interacts with the provider (e.g. education providers) and associated stakeholders.
  • Anticipatory, organisational and lateral mechanisms describe the actions of the provider, including working with other providers.
  • Informational and systemic mechanisms relate primarily to the actions of the regulatory body in terms of disseminating information about providers, including identification of any issues.

These mechanisms may lead to both positive and negative impacts, and the extent of impact via these different mechanisms is not expected to be uniform.

Australian Senior Citizen Couple Enjoying Life and Living Independently At Own Home

Photo by Thurtell/iStock

ADC's current mechanisms of impact

This research provides insight into the different mechanisms by which ADC's activities and outputs may drive outcomes and impact, discussed below with reference to examples from the empirical data collection.

Mechanisms relating to how ADC interacts with providers and other stakeholders

Accreditation reports from ADC provide an essential, directive mechanisms of impact. However, as shown in Table 1, the ways ADC can achieve its desired outcomes and impacts are much broader than this. ADC's work appears to generate impact via strong strategic and stakeholder mechanisms, which is anticipated given that continuous improvement and stakeholder engagement are at the core of ADC's approach. Perhaps less anticipated were findings regarding relational impact mechanisms, which highlighted that both ADC staff and staff from education providers drew benefit from the less 'formal' discussions that occur during self-assessment support meetings and when the ADC Accreditation Team (AT) visit providers in person.

Table 1. Mechanisms of impact relating to how ADC interacts with providers and other stakeholders with selected examples

Impact mechanism Description of logic/causal chain/process Selected examples identified in this research
Strategic The regulator reinforces their credibility and that of the accreditation process through engagement with the wider regulatory landscape and with providers and other key stakeholders to ensure accreditation meets their needs and has their support and investment. Education providers recognised and reflected positively on ADC's's efforts to improve and streamline program accreditation and introduce new relevant accreditation standards such as cultural safety.
Directive Providers take actions that they have been directed or guided to take by the regulator. This includes enforcement actions and, at the extreme, may involve formal legal repercussions such as prosecution or cancellation of accreditation. Recommendations and conditions in ADC's accreditation report are acted on by education providers in order to gain or retain program approval.
Relational Results from the nature of relationships between accreditation staff (i.e. inspectors) and accredited providers. Informal, soft, influencing actions have an impact on providers. Discussions between the ADC AT and provider staff during a physical site visit to the provider were reported to stimulate ideas about new approaches to or strategies for improving a program.
Stakeholder The regulator engages with stakeholders and incorporates their perspectives into the accreditation approach and standards. Accreditation actions encourage, mandate or influence other stakeholders to take action or to interact with the accredited provider. ADC's proactive approach to eliciting feedback from stakeholders (e.g. their survey of providers) results in stakeholders recognising that their views and suggestions have been addressed in revisions to ADC's's accreditation approach.

Source: Adapted from Smithson et al. (2018)

Mechanisms relating to the actions of the provider

Anticipatory impact mechanisms will always be a function of accreditation processes, but our findings suggest that ADC's approach and the resources they provide strengthen this in a positive way. We found less evidence for impact via organisational and lateral mechanisms (see Table 2). Organisational mechanisms were difficult to assess because provision and accreditation of a single program usually occurs within the context of a larger education provider with programs in a range of areas. ADC's ability to stimulate organisational change may be limited by the competing demands faced by the education provider.

Table 2. Mechanisms of impact relating to actions of the provider with selected examples

Impact mechanism Description of logic/causal chain/process Examples identified in this research
Anticipatory The regulator sets quality expectations; providers understand those expectations and seek compliance in advance of any accreditation interaction. Education providers reported using the resources ADC provides to support self-assessment, including informal discussion sessions to identify and address potential shortcomings prior to commencing the formal accreditation process.
Organisational Regulator interaction leads to internal organisational developments, reflection and analysis by providers that are not related to specific regulator directions. This leads to changes in areas such as internal team dynamics, leadership, culture, motivation and whistle-blowing. Some stakeholders reported that the external assessment provided by ADC can prompt reflections on how to simulate and support improvement among senior leadership of an education provider organisation.
Lateral Accreditation interactions stimulate inter-organisational interactions, such as providers working with their peers to share learning and undertake improvement work. There was some evidence of providers working together to improve their programs to meet the new ADC Accreditation Standard relating to cultural safety.

Source: Adapted from Smithson et al. (2018)

Mechanisms relating to ADC's actions

We identified limited examples of informational and systemic impact mechanisms (see Table 3), although many stakeholders who contributed to this project were aware of ADC's research efforts.

Table 3. Mechanisms of impact relating to ADC's actions with selected examples

Impact mechanism Description of logic/causal chain/process Examples identified in this research
Informational The regulator collates intelligence and puts information about provider performance into the public domain or shares it with other actors who then use it for decision making. ADC disseminations information on program accreditation outcomes via its website, but it is not currently clear how widely this information is used.
Systemic Aggregated findings/information from accreditation are used to identify systemic or inter-organisational issues and to influence stakeholders and wider systems other than the accredited providers themselves. Although we found evidence for awareness of ADC's research, its reach currently appears to be limited as not all stakeholders who participated were aware of it.

Source: Adapted from Smithson et al. (2018)

Sixteen-year-old boy smiling at park.

Photo by GaryRadler/iStock

Recommendations for strengthening the impact of dental program accreditation

This research has identified some potential ways ADC could strengthen the impact of dental program accreditation based on direct suggestions by study participants and identification of common themes in the survey and interview data. These potential avenues relate to six of the nine mechanisms of impact considered in this framework:

Strategic
Expanding the geographic remit of ADC (beyond Australia and New Zealand) or adoption of its principles and processes by other professional accreditation schemes would strengthen ADC's strategic impact.
Relational
Increasing collaboration with education providers beyond the accreditation process, such as engaging with education providers through joint research and quality improvement work, could further strengthen relationships between ADC staff and provider staff.
Informational
A greater analysis of the data collected by ADC as part of the accreditation process could be used to strengthen its impact. This could include sharing examples of best practice or innovative approaches that providers could learn from.
Stakeholder
Explore what constitutes a stakeholder for ADC and what their perspectives might be on ADC's work to support targeted engagement.
Lateral
Incentivise education providers to work collaboratively on curriculum development that addresses the accreditation standards, encouraging the development of provider networks.
Systemic
Use accreditation data to: identify trends and potential issues that could be disseminated to providers prior to accreditation; expand the key issues addressed by ADC's research to include a broader definition of groups that experience inequalities; license overseas-trained workforce (in terms of mobility and fairness to low-/middle-income countries); manage the uneven urban/rural distribution of dental practitioners; and achieve a broader understanding of funding models relevant to quality improvement and interactions between dental and other health professions.

Next steps

In line with ADC's commitment to continuous improvement and stakeholder engagement, the research presented represents the first step in ADC's strategy for enhancing dental program accreditation. The next step is to test and refine the framework through broader stakeholder consultation. This will lead to further refinements of the framework and indicators and will generate additional examples of impact mechanisms and more suggestions for how ADC can enhance its impact. This will ultimately strengthen ADC's understanding of how its work achieves desired goals and support robust evaluation of its impact.

Research conducted by

This report is part of the RAND research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.