Objectives:
- Enroll and engage a Community of Practice (CoP) of both open infrastructure providers and Responsible Research Assessment stakeholders. The role of this CoP will include working together to develop practical solutions for optimal use of existing open infrastructures.
- Carry out systematic analysis of longer-term scenarios for moving toward open, interoperable, sustainable, community-governed OI4RRA.
- Conduct a scenario analysis, as the basis for an OI4RRA implementation plan, focusing on agreed principles , governance models that ensure inclusive and equitable participation, and sustainability models informed by a detailed analysis of the
expected costs.
- Submit implementation recommendations to CoARA members and the CoARA board.
Research assessment generally relies on proprietary databases often using proprietary analytic tools. The existing OI community has created and operates open solutions (e.g., OpenAIRE, OpenCitations, COKI, OpenAlex, Semantic Scholar), which, with very few exceptions, have yet to find their way in official research assessment operations. As research becomes more digital and research assessment practices are moving towards inclusion of increased diversity of research outputs (data, software, protocols, etc.) and practices (open science, innovation), the need for an interconnected and decentralised open infrastructure ecosystem is increasingly evident.
The OI4RRA Working Group (WG) has focused on two key areas to assist the CoARA Community in effectively navigating Open Infrastructures (OI) for Responsible Research Assessment:
-
- Development of comprehensive principles, characteristics, and terminology for OI to support open, sustainable, community-governed, fair, and equitable practices in Reforming Research Assessment (RRA). Key outcomes currently include:
- Community Governance: Emphasising stakeholder-driven models that reflect the diversity and needs of the scholarly community, ensuring inclusive decision-making.
- Transparency and Openness: Promoting clear communication of policies, operations, and data management, alongside the adoption of open standards and technologies to ensure interoperability.
- Sustainability and Equity: Developing long-term, viable financial models and ensuring equitable access to data and resources, with a strong focus on inclusivity and diversity.
- Responsibility, Integrity and Ethical Standards: Upholding the highest standards of ethical practices, including rigorous quality control, respect for privacy, and a commitment to reproducibility, as well as ensuring infrastructures to guarantee the authenticity, reliability, and accessibility of scholarly content.
- Innovation and Adaptability: Encouraging continuous improvement and alignment with evolving community values and technological advancements.
- Architectural reference model for effective OI in RRA that outlines the technical, legal, and financial capacities necessary to develop, understand, and adopt OI in RRA. The schema is comprised of 4 tiers with the following key highlights to date:
- Tiers 0 – Foundation Tier which is the backbone of the essential metadata, ensuring consistent tracking with Persistent Identifiers (PIDs), and adopting standardised protocols
- Tiers 1 – Research Publishing Venues, including repositories, open access journals and publishers where the public is accessing the research products like articles, data, software, and many others
- Tiers 2 – Scholarly Metadata Aggregators are those database aggregators or Scientific Knowledge Graph that contains the metadata in tiers 0 and 1
- Tier 3 – Research Assessment Services: Services and platforms that are leveraging the previous tiers in order to generate metrics, analytics and indicators tailored for RRA.
To efficiently pursue its goals and outlined work plan, the OI4RRA WG has established two focused subgroups.
Subgroup 1 concentrates on the social dimensions, exploring the “whys” and “hows” of adoption, promoting the shift to OI for Responsible Research Assessment (RRA) with arguments grounded in integrity, inclusivity, and fairness. The subgroup will collect and share case studies of using OIs for RRA. It will also conduct focus group/targeted interviews to identify the current landscape of using OIs vs closed ones.
Subgroup 2 delves into the technical and operational facets, advocating for collaborative development and funding of OI for RRA.
The two subgroups have become operational and have initiated their activities in line with the work plan. A preliminary document outlining the definition and principles of OI fit for RRA has been drafted. Additionally, relevant case studies have been identified by the community.
The Working Group (WG) will focus on mobilising existing Open Infrastructures for RRA, while also establishing a path toward a global OI4RRA ecosystem.
