Universities that sign the San Francisco Declaration on Research Assessment (DORA) or join the Coalition for Advancing Research Assessment (CoARA) commit to judging research on its quality and its full range of contributions, rather than on journal-based metrics alone. Recent studies suggest the commitment is real but uneven. Action plans speak more about values than about concrete practices. Repositories are ready to hold datasets, software and other non-traditional outputs, but few libraries actively promote them. Practical procedures for hiring and promotion now exist, and critics warn that new criteria must fit different kinds of research.
What responsible research assessment means
DORA was developed by a group of editors and publishers who met during the Annual Meeting of the American Society for Cell Biology in San Francisco on 16 December 2012. Its general recommendation asks institutions and others: "Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure" when assessing individual scientists or making hiring, promotion or funding decisions. The declaration contains 18 numbered recommendations addressed to funding agencies, academic institutions, journals, organizations that supply metrics, and individual researchers.
CoARA is a coalition of organizations that have signed the Agreement on Reforming Research Assessment. According to CoARA, the Agreement sets out principles, commitments and a timeframe for reform, and signatories decide their own pace. Researchers studying the coalition describe signatories as institutions publicly committed to reforming research assessment, and they examine the action plans those institutions publish.
Across both initiatives, the shared idea is that indicators may inform judgment, but should not replace it, and that a wider range of research contributions should count.
What the recent evidence shows
Action plans: values ahead of practice
A 2026 study in Scientometrics by Rushforth, Gogadze, Skhirtladze and Pölönen analysed 248 publicly available CoARA action plans and coded how often open science terms appear. Universities made up 66% of the organizations behind these plans (164 of 248). The authors report that most terms appear in fewer than 20% of the documents, and that values-based terms outnumber operational ones among the most frequent mentions.
| Term | Action plans mentioning it | Share of 248 plans |
|---|---|---|
| Open Science (umbrella term) | 167 | 167 of 248 |
| Open Access | 102 | about 41% (our calculation) |
| Research Data Management | 47 | 19% |
| FAIR Data | 32 | 13% |
| Open Peer Review | 7 | 3% |
Open Access was by far the most widely mentioned operational practice. Activities emphasized in policy frameworks, such as open evaluation, open methods and open software, received minimal attention. The authors conclude that integrating open science into assessment reform is at an early stage, with a gap between policy aspirations and organizational commitments.
Repositories: ready, but rarely promoted
A 2023 study in Evidence Based Library and Information Practice by Christie Hurrell examined 77 UK universities that are DORA signatories and run institutional repositories. Almost all of these repositories (96%) held some non-traditional research outputs, although such outputs were a small share of their content. 82% showed usage metrics of some kind, but 67% did not mint persistent identifiers for their items. 53% of the universities also ran a separate data repository; of those, 90% assigned persistent identifiers and all showed metrics.
Only 47% of the library websites reviewed mentioned non-traditional research outputs when promoting their repositories. Repository managers who answered the survey said their units were involved in implementing DORA and considered it influential. The study concludes that the repositories are well positioned to support broader assessment, but that most libraries in the sample were not actively collecting non-traditional outputs.
Practical models for hiring and promotion
Assessment reform becomes concrete in hiring and promotion. A report by a task force of the German Psychological Society, published in Meta-Psychology in December 2025 (Schönbrodt and colleagues), proposes four principles of responsible research assessment and a two-phase procedure. The procedure combines indicators, for their objectivity and efficiency, with a qualitative, discursive assessment of shortlisted candidates. The proposal broadens the contributions that count to include published datasets and research software alongside papers, and puts greater weight on quality and methodological rigor.
The critique: one standard does not fit every field
Reform proposals have their own critics. In a 2024 commentary in Meta-Psychology on two 2022 articles that proposed DORA-based assessment for psychology, Sven Ulpts argues that their approach rests on a narrow statistical and quantitative understanding of knowledge production. He warns that criteria suited to some research approaches could systematically disadvantage others. He calls for clearer statements of when a proposal applies and when it does not, and for more intellectual humility in reform efforts.
Our interpretation
Taken together, these studies suggest that signing a declaration is the easy step. The harder steps are operational. They include:
- defining which contributions count;
- making those contributions findable and citable;
- training the people who sit on hiring and promotion panels;
- checking that new criteria do not disadvantage particular fields.
The evidence above comes mostly from Europe and from library and psychology settings, so it shows the direction of change rather than a global picture.
Practical steps for universities
- Turn principles into procedures. Publish the specific criteria and steps used in hiring and promotion, not only a statement of support for DORA or CoARA.
- Name the outputs that count. List datasets, software, protocols, policy reports and other non-traditional outputs that may be submitted, and explain how they will be judged.
- Make those outputs citable. Make sure the institutional repository, or a data repository, assigns persistent identifiers such as DOIs to non-traditional outputs.
- Tell researchers where to deposit. Library and research office web pages should say clearly that the repository accepts non-traditional outputs.
- Use indicators carefully. If indicators are used, document their purpose and limits, never rely on journal-based metrics for individuals, and pair indicators with qualitative review.
- Check fit by discipline. Review criteria with each faculty to make sure they do not penalize fields with different research traditions.
- Monitor and report progress. Track whether action-plan commitments lead to changed practice, and publish updates.
Limitations
This review covers four recent studies selected for relevance and verifiability; it is not a systematic review.
- The CoARA analysis measures which terms appear in action plans, not what institutions actually do.
- The repository study covers UK DORA signatories only, and its publication date (December 2023) means practices may have changed since.
- The hiring and promotion proposal and the critique both come from psychology and may not transfer directly to other fields.
- One study (the CoARA analysis) was read in full on the publisher site; the other three were assessed from the abstracts on their publisher pages.
- Several relevant studies could not be included because their publisher pages could not be opened for verification on the day of writing.
Related reading on UNIRANKS: How to Measure and Increase Your Research Impact: Introduction and Definitions and Digital Tools and Metrics for Research Impact. More on this dimension: Academic & Research Performance.
Editorial disclosure
UNIRANKS publishes university rankings, and its methodology includes an Academic & Research Performance dimension. This article reviews independent research on how universities assess their own researchers. It does not describe how UNIRANKS scores institutions.
Research References
- Rushforth A, Gogadze N, Skhirtladze T, et al. (2026). Advancing open science through research assessment reform? Content analysis of CoARA action plans. Scientometrics, 131(6), 3897–3918. https://doi.org/10.1007/s11192-026-05650-w (verified 19 November 2026)
- Schönbrodt FD, Gärtner A, Frank M, et al. (2025). Responsible Research Assessment I: Implementing DORA and CoARA for hiring and promotion in psychology. Meta-Psychology, 9. https://doi.org/10.15626/mp.2024.4601 (verified 19 November 2026)
- Ulpts S (2024). Responsible assessment of what research? Beware of epistemic diversity! Meta-Psychology, 8. https://doi.org/10.15626/mp.2023.3797 (verified 19 November 2026)
- Hurrell C (2023). Research Assessment Reform, Non-Traditional Research Outputs, and Digital Repositories: An Analysis of the Declaration on Research Assessment (DORA) Signatories in the United Kingdom. Evidence Based Library and Information Practice, 18(4), 2–20. https://doi.org/10.18438/eblip30407 (verified 19 November 2026)
Sources
- San Francisco Declaration on Research Assessment. DORA. https://sfdora.org/read/ (accessed 19 November 2026)
- The Agreement. Coalition for Advancing Research Assessment (CoARA). https://coara.org/agreement/the-agreement-full-text/ (accessed 19 November 2026)
- UNIRANKS Global University Ranking Methodology. UNIRANKS. https://www.uniranks.com/methodology (accessed 19 November 2026)
