Working Group: Responsible Metrics and Indicators (RMI)

The WG has three objectives, tackled in three stages (Oct. 23 – Sept. 25):

• Stage 1: Assessing the status quo: Which indicators are currently employed?
  We investigate this across varied disciplines and cultural contexts, specifically in the evaluation of researchers, awards, institutional assessments, and the progression of scientific advancement (like tenure, PhD, habilitation, professor titles, etc.). Starting with CoARA members, institutions are asked to provide information on their current usage of indicators via a structured survey.
• Stage 2: Critical evaluation of the indicators and recommendations. Based on the survey results, we develop guiding questions and recommendations (2a) when to use indicators (and when not),and (2b) how to integrate their usage with qualitative modes of assessment. This could be a decision path diagram, and will not be a “one size fits all” recommendation.
• Stage 3: Strategic plan for dissemination of recommendations. Share findings, strategies, and recommendations with a wider audience, emphasizing their applicability across disciplines and cultures.

Current status of the WG

• Indicator survey in preparation  
• Landing page: https://github.com/rachelHey/metricsWG 

Next steps 

• April 2024 – distribution of the Indicator Survey among all members of the RMI WG 
• Autumn 2024 – distribution of the Indicator Survey among all members of CoARA 

Planned meetings, deliverables 

• General meeting on Monday, 2023-12-18; then quarterly meetings 

It is planned to present preliminary survey results and discuss the usage of metrics and indicators in advancing research assessment during the 9th Annual Conference of the Society for Economic Measurement.

Want to become a member of the RMI WG? 

• Members of the WG are CoARA members (i.e., institutions), represented by a concrete person. 
• Experts of the WG are individual persons who are experts in the field, but are not officially representing a CoARA member. The WG can (proactively) contact them for consultation. 
• Friends of the WG are generally interested persons who want to simply stay informed. 

Get in touch: Github. There you can find all contacts and instructions on how to join or contribute to the working group.

Contacts:

• Felix Schönbrodt, Ludwig-Maximilians-Universität München, felix.schoenbrodt[at]psy[dot]lmu[dot]de
• Davide Crepaldi, SISSA,  dcrepaldi[at]sissa[dot]it
• Rachel Heyard, University of Zurich (UZH), rachel.heyard[at]uzh[dot]ch
• Katarzyna Nawrot, University of Warsaw, nawrotini[at]gmail.com