Is it possible to accurately assess the impact of scholarly research and let funding bodies use these assessments to determine which universities should receive funding? The answer appears to be yes, at least in the U.K. There, research outputs from universities are assessed every five years to determine future funding allocations from government. In 2014, for the first time, the Research Excellence Framework (REF) included an assessment of research impact. This component was worth 20 percent of the score awarded to each institution.
RAND Europe, an independent research institute that focuses on policy and decision making, worked with higher education funding councils in the U.K. to evaluate how well the system worked in its first outing.
RAND Europe evaluated the way universities prepared submissions as well as the ways submissions were assessed by panels of academics and research users. In particular, the organization attempted to identify the benefits and burdens schools realized under the new system, outline the intended and unintended consequences, recommend ways to improve the process, and highlight best practices.
Among the organization’s findings: The impact component of the REF 2014 encouraged scholars to think more broadly and strategically about their research and helped universities better understand the impact of their research. Nonetheless, complying with the impact component consumed time and resources for people and institutions; academics considered the process burdensome, but research users did not.
On the positive side, REF 2014 is perceived as contributing to a cultural shift in which institutions and individuals are focusing more on the current and potential impacts of their research. And a majority of academics and research users felt that the REF 2014 process enabled them to assess impact fairly and reliably.
RAND Europe identified areas for discussion and improvement, including managing the variations in the way the process is conducted, avoiding the risk of false claims, and clarifying the processes for assessing different kinds of impact.
Catriona Manville, senior analyst and lead author on the evaluation reports, says, “Given that the process for assessing impact was new, our evaluation shows that it worked, and it worked well. REF 2014 now provides a valuable working model for measuring impact in other countries and research systems.”
For project information and links to reports, visit www.randeurope.org/REF2014impact