- What processes relating to evaluation of both overall implementation of the 3AP program and site-level implementation work best?
- What are the lessons to be learned for the evaluation of Aboriginal and Torres Strait Islander family violence programs for the future?
Violence against women and children continues to challenge well-being in Australia, especially for Aboriginal and Torres Strait Islanders. In this report, the authors discuss their evaluation of the Department of the Prime Minister and Cabinet's (PM&C's) Third Action Plan (3AP) of the National Plan to Reduce Violence Against Women and Their Children, and specifically 3AP's Priority Area 2: Aboriginal and Torres Strait Islander Women and Their Children.
PM&C provided funding to 14 different service providers across Australia that work specifically on family issues in Aboriginal communities. Aboriginal communities are mostly located in remote and sparsely populated areas, creating challenges for service delivery. The 3AP funding was intended to help service providers deliver initiatives to address one or more of four categories: (1) trauma-informed therapeutic services, (2) men's behaviour change programs, (3) intensive family case management and (4) victim services through legal service providers.
This report is intended to guide the Australian government's future work in providing family violence services in Aboriginal and Torres Strait Islander communities, and to provide lessons to other organisations outside the federal government that may want to undertake similar work. Researchers conducted a thematic analysis of interviews with site staff and analysed site documentation to make recommendations stemming from the evaluation findings. The authors' recommendations can serve as a strong foundation for increasing funders' return on investment and creating long-term change in these communities across Australia.
- Interviewees from sites frequently described how a holistic approach to treatment provided positive results.
- Programs that were able to engage informal and formal institutions in communities, from law enforcement to local clubs, created extended systems of support for program clients that had positive impacts on program goals.
- Clinicians noted that storytelling and narrative therapeutic approaches were often quite effective, in part because these approaches fit well with Aboriginal and Torres Strait Islander cultural practices.
- Making sure to staff programs with Aboriginal and Torres Strait Islander staff was critical.
- Family violence among Aboriginal and Torres Strait Islanders has been created by generations of trauma, social marginalisation and structural inequality.
- Many communities have seen programs come and go over the years (often due to inconsistent funding) and have experienced broken promises from government entities or other efforts to address local problems.
- Opinions on communication with PM&C — at both the national and regional levels — were inconsistent across sites.
- Interviewees criticised PM&C for its lack of knowledge about local communities and the contexts of implementation.
- At the outset, the national PM&C office could have made things smoother by clearly identifying the roles and responsibilities for the regional and national offices, and for the sites.
- Sites did, however, appreciate and praise the feedback they received on program design and development.
- PM&C may have had unrealistic expectations for site implementation.
- Data collection was extremely challenging for sites.
- Site staff expressed a desire for more meaningful contact with other sites.
- Sites should select a technical assistance provider and evaluator before or at the same time as they are selected for funding.
- Sites should engage local institutions during program design to provide an extended network of services and support.
- PM&C should assess a site's readiness for implementation before a program commences.
- PM&C should create a plan for coordinated and transparent communication with funded sites throughout the implementation period.
- PM&C should allow for decentralised management coupled with centralised record-keeping.
- PM&C should provide more technical assistance and support for data collection.
- PM&C should support efforts to develop assessment tools that are appropriate for Aboriginal and Torres Strait Islander individuals.
- PM&C should understand that the timeline for change is long and requires sustained investment.
- Future work should integrate more quantitative measurement for evaluative rigor while continuing to collect rich qualitative data.
- The use of locally developed, culturally grounded measurement approaches should be emphasised.
- Sites should focus on evaluating implementation processes first, then evaluate outcomes if and when appropriate.
- Evaluators should set up a Cultural Advisory Board before beginning any research to ground the evaluation in the local context.
Table of Contents
Third Action Plan Program Logic Models
Initial Interview Guides
Process Evaluation Interview Guide
Site Information Requirements