Open research information is crucial for responsible research assessment, which needs data, tools, infrastructures that secure transparency, reproducibility, geographic-discipline-output coverage in data and indicators. Their sustainability, interoperability, openness, and community-based accountability are key to the reform.

This working group’s mission is to enable institutions to move from proprietary infrastructure and research information, to open (interoperable) alternatives–in support of the transition to responsible research assessment practices. This effort will take into consideration the wide range of research outputs and open science practices, and address the diversity of the global research
community.

Objectives:

• Enroll and engage a Community of Practice (CoP) of both open infrastructure providers and Responsible Research Assessment stakeholders. The role of this CoP will include working together to develop practical solutions for optimal use of existing open infrastructures.
• Carry out systematic analysis of longer-term scenarios for moving toward open, interoperable, sustainable, community-governed OI4RRA.
• Conduct a scenario analysis, as the basis for an OI4RRA implementation plan, focusing on agreed principles , governance models that ensure inclusive and equitable participation, and sustainability models informed by a detailed analysis of the
  expected costs.
• Submit implementation recommendations to CoARA members and the CoARA board.

Research assessment generally relies on proprietary databases often using proprietary analytic tools. The existing OI community has created and operates open solutions (e.g., OpenAIRE, OpenCitations, COKI, OpenAlex, Semantic Scholar), which, with very few exceptions, have yet to find their way in official research assessment operations. As research becomes more digital and research assessment practices are moving towards inclusion of increased diversity of research outputs (data, software, protocols, etc.) and practices (open science, innovation), the need for an interconnected and decentralised open infrastructure ecosystem is increasingly evident.

The OI4RRA Working Group (WG) has focused on two key areas to assist the CoARA Community in effectively navigating Open Infrastructures (OI) for Responsible Research Assessment: 

1. 1. Development of comprehensive principles, characteristics, and terminology for OI to support open, sustainable, community-governed, fair, and equitable practices in Reforming Research Assessment (RRA). Key outcomes currently include:
      • Community Governance: Emphasising stakeholder-driven models that reflect the diversity and needs of the scholarly community, ensuring inclusive decision-making.
      • Transparency and Openness: Promoting clear communication of policies, operations, and data management, alongside the adoption of open standards and technologies to ensure interoperability.
      • Sustainability and Equity: Developing long-term, viable financial models and ensuring equitable access to data and resources, with a strong focus on inclusivity and diversity.
      • Responsibility, Integrity and Ethical Standards: Upholding the highest standards of ethical practices, including rigorous quality control, respect for privacy, and a commitment to reproducibility, as well as ensuring infrastructures to guarantee the authenticity, reliability, and accessibility of scholarly content.
      • Innovation and Adaptability: Encouraging continuous improvement and alignment with evolving community values and technological advancements. 
        
   2. Architectural reference model for effective OI in RRA that outlines the technical, legal, and financial capacities necessary to develop, understand, and adopt OI in RRA. The schema is comprised of 4 tiers with the following key highlights to date:
      • Tiers 0 – Foundation Tier which is the backbone of the essential metadata, ensuring consistent tracking with Persistent Identifiers (PIDs), and adopting standardised protocols
      • Tiers 1 – Research Publishing Venues, including repositories, open access journals and publishers where the public is accessing the research products like articles, data, software, and many others
      • Tiers 2 – Scholarly Metadata Aggregators are those database aggregators or Scientific Knowledge Graph that contains the metadata in tiers 0 and 1
      • Tier 3 – Research Assessment Services: Services and platforms that are leveraging the previous tiers in order to generate metrics, analytics and indicators tailored for RRA.

To efficiently pursue its goals and outlined work plan, the OI4RRA WG has established two focused subgroups.

Subgroup 1 concentrates on the social dimensions, exploring the “whys” and “hows” of adoption, promoting the shift to OI for Responsible Research Assessment (RRA) with arguments grounded in integrity, inclusivity, and fairness. The subgroup will collect and share case studies of using OIs for RRA. It will also conduct focus group/targeted interviews to identify the current landscape of using OIs vs closed ones.

Subgroup 2 delves into the technical and operational facets, advocating for collaborative development and funding of OI for RRA.

The two subgroups have become operational and have initiated their activities in line with the work plan. A preliminary document outlining the definition and principles of OI fit for RRA has been drafted. Additionally, relevant case studies have been identified by the community.

The Working Group (WG) will focus on mobilising existing Open Infrastructures for RRA, while also establishing a path toward a global OI4RRA ecosystem.

Updated on 13 January 2025:

• Barcelona Declaration Conference: On September 23-24, 2024, the WG gave a presentation titled ‘Harmonising the Barcelona Declaration and CoARA Efforts: Advancing Open Infrastructures for Responsible Research Assessment’.
• CoARA Co-Chairs Working Group meeting: On September 10-11, 2024 OI4RRA presented our latest advancements in Helsinki and engaged in exchanges to learn from other CoARA working groups.
• WG Meeting took place on the 6th September
• The OI4RRA WG participated at the CoARA EOSC Workshop, Wednesday 10 April 2024, organized by the Open Science and Research Infrastructures Unit of the European Commission.
• The Working Group is currently interviewing organisations on their primary motivations moving from closed, large-scale commercial infrastructures towards OI. These interviews aim to identify key resources and challenges for the transition towards OI, as well as to determine opportunities available to enable this transition. If you are interested in providing an interview on your organisation’s motivation for closed infrastructure in research assessment, or can recommend other organisations that would be a good fit, please reach out to Angeliki Tzouganatou or Giulia Malaguarnera. 
• We are currently conducting interviews with institutions that have already transitioned to OI or are in the process of doing so. These interviews aim to gather best practices and insights to guide others on the same path. We would love to hear from you! If you would like to share your experiences with us, please reach out to Angeliki Tzouganatou at angeliki.tzouganatou@openaire.eu and Giulia Malaguarnera at giulia.malaguarnera@openaire.eu.
• The WG has made significant progress in developing user personas that demonstrate how various tiers can be adopted by different individuals and groups. These personas serve as practical examples, illustrating how these tools can be effectively used to assess research impact, openness, and reproductibility. See below for more details on the architectual reference model updates.
• Recent publication: Di Donato, F. (2024). What we talk about when we talk about research quality. A discussion on responsible research assessment and Open Science. Bollettino Telematico Di Filosofia Politica. https://doi.org/10.5281/zenodo.10650140
• The Irish National Open Access Monitor was launched in March, featuring dashboards tailored for researchers, RPOs, RFOs, and national overviews. Explore the tools and consider their applications for your projects. Your feedback would be invaluable. Watch the Video | Visit the Website | Read the News