Design and development of an evaluation framework for the NHMRC

Review of international practice, overview of NHMRC's data and grant management systems, and an evaluation framework and metrics

by Susan Guthrie, Daniela Rodriguez-Rincon, Gordon R. McInroy, Becky Ioppolo, Salil Gunashekar

Download eBook for Free

FormatFile SizeNotes
PDF file 2.2 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Question

  1. How can fairness, burden and innovation and creativity in research funding processes be measured?

The National Health and Medical Research Council (NHMRC), Australia, have established a new grant program which aims to: (i) encourage greater creativity and innovation in research; (ii) minimise the burden on researchers of application and peer review so that researchers can spend more time producing high quality research; and (iii) provide opportunities for talented researchers at all career stages to contribute to the improvement of human health. The aim of this study is to help NHMRC think about ways to assess and measure the impact of the changes to their restructured grant program against those three aims. To do this, we have conducted a review of international practice in the program evaluation landscape for grant funding processes, focusing on the three core aims of NHMRC's new grant program; a review of NHMRC's data and grant management systems and capabilities; and developed an evaluation framework and metrics that NHMRC can use to monitor the performance of the grant program against these three aims over time.

Key Finding

  • We have developed a framework and metrics for the evaluation of burden, fairness and innovation and creativity in NHMRC's funding.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Results from review of international practice

  • Chapter Three

    Overview of NHMRC's data and grant management systems and capabilities

  • Chapter Four

    A framework and metrics for NHMRC's new grant program

  • Chapter Five

    Considerations and caveats

  • Annex A

    Methodological approach

  • Annex B

    Interview protocol for stakeholders external to NHMRC

  • Annex C

    Protocol for focus groups with NHMRC representatives

  • Annex D

    Grouping of RGMS label descriptors

  • Annex E

    Longlist of metrics

Research conducted by

The research described in this report was commissioned by the National Health and Medical Research Council (NHMRC) and conducted by RAND Australia.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.