Research Assessment

QMU is committed to fair and transparent mechanisms in research assessment, which is reflected in our Institutional Statement on the Use of Metrics. We apply these principles across Research and Knowledge Exchange.

This position supports the use of both qualitative and quantitative indicators to assess individual and institutional performance, and we acknowledged the limitations of using either approach in isolation.

QMU recognises the need for the fair and transparent use of quantitative indicators in the external measurements of our reputation, as measured by league tables and funding agencies. We do not use single, non-normalised metrics in research assessment. We only use indicators that are transparent and contextualised with citation practices within the relevant discipline.

We are support the recommendations and principles set out by the global and national initiatives calling for improvements in the ways in which the outputs of scholarly research are evaluated and seeking to develop and promote best practice. These include San Francisco Declaration on Research Assessment (2013), Research England (formerly Higher Education Funding Council for England) The Metric Tide Report (2015), the Leiden Manifesto for Research Metrics (2015), and the Agreement on Reforming Research Assessment (2022).

DORA was developed in 2012 during the Annual Meeting of the American Society for Cell Biology in San Francisco, in recognition of the need to improve the ways in which the outputs of scholarly research are evaluated. It has become a worldwide initiative, with recommendations covering all scholarly disciplines collectively seeking to develop and promote best practice.

The key objectives of DORA are:

Raise awareness - to call attention to new tools and processes in research assessment and the responsible use of metrics that align with core academic values and promote consistency and transparency in decision-making
Facilitate implementation - to aid development of new policies and practices for hiring, promotion, and funding decisions
Catalyse change - to spread research assessment reform broadly by working across scholarly disciplines and globally
Improve equity - to call for broader representation of researchers in the design of research assessment practices that directly address the structural inequalities in academia.

In 2024, DORA launched a new initiative, Tools to Advance Research Assessment (TARA), a project which aims to facilitate the development of new policies and practices for academic career assessment.

In 2014 the UK Government commissioned an independent expert group to critically examine the prospects for using metrics in the evaluation of research, notably in the Research Excellence Framework (REF).

The resulting report, The Metric Tide, published in July 2015, provided an extensive review of the literature on peer review, the use of metrics and altmetrics, and a statistical analysis of the predictive power of various numerical indicators. The report concluded that, while some metrics may sometimes be a useful adjunct to peer review, they should always be used carefully and with due consideration of context. A follow up report, Harnessing the Metric Tide was published in 2022 as part of the joint UK Funding Bodies, Future Research Assessment Programme (FRAP). Two new bodies were also created, the UK Forum for Responsible Research Metrics (2016) and the Research on Research Institute (2019), that have strengthened the evidence base and enriched international debates on assessment cultures and practices.

The Leiden Manifesto provides ten principles for the appropriate use of metrics in research evaluation. These principles can be used to maintain accountability of both evaluators and the indicators they use in metrics-based research assessment.

The Principles:

Quantitative evaluation should support qualitative, expert assessment.
Measure performance against the research missions of the institution, group or researcher.
Protect excellence in locally relevant research.
Keep data collection and analytical processes open, transparent and simple.
Allow those evaluated to verify data and analysis.
Account for variation by field in publication and citation practices.
Base assessment of individual researchers on a qualitative judgement of their portfolio.
Avoid misplaced concreteness and false precision.
Recognize the systemic effects of assessment and indicators.
Scrutinize indicators regularly and update them.

The Coalition for Advancing Research Assessment (CoARA) is a collective of organisations committed to reforming the methods and processes by which research, researchers, and research organisations are evaluated. The Coalition recognised that current research assessment methods rely heavily on publication-based metrics such as citation counts and often fail to recognise the wide array of contributions made by researchers.

Consequently, the members have agreed on a common direction and guiding principles to implement reform in the assessment of research, researchers, and research organisations, outlined in the Agreement on Reforming Research Assessment. Published in 2022, the Agreement sets a shared direction for changes in assessment practices for research, researchers and research performing organisations, with the overarching goal to maximise the quality and impact of research.