About
Towards Open Bibliometric Indicators (TOBI) is a project co-founded by the ETH Library and swissuniversities within the Open Science Program. The project has started in early 2023. The final results are planned to be published in late 2024.
Why TOBI?
DORA Declaration requires open data sources
DORA Declaration has been introduced in 2013 as a countermeasure to the overreliance on journal-driven metrics to estimate quality of researchers and performance of individuals and institutions. Among its recommendations, points #11-#14 call for openness and transparency in data driven methods of performing research assessment.
For organizations that supply metrics:
11. Be open and transparent by providing data and methods used to calculate all metrics.
12. Provide the data under a licence that allows unrestricted reuse, and provide computational access to data, where possible.
13. Be clear that inappropriate manipulation of metrics will not be tolerated; be explicit about what constitutes inappropriate manipulation and what measures will be taken to combat this.
14. Account for the variation in article types (e.g., reviews versus research articles), and in different subject areas when metrics are used, aggregated, or compared.
At the core of data-driven research assessment is a statistical evaluation of scientific publications and their citations, basing on which we can estimate the popularity or impact of particular research studies. Until now, the “gold standard” for such an evaluation was to use bibliometric data gathered in one of the commercial databases, such as Web of Science or Scopus. Unfortunately, those databases come with very prohibitive licenses and proprietary methods of data curation. This means that using them for research assessment will never be fully conform with DORA recommendations mentioned above, and open alternatives are needed.
In recent years, many new sources of open bibliometric data have been published. However, some quality characteristics of these datasets remain unclear, such as affiliation disambiguation, missing values, metadata availability, and under-/overrepresentation of particular research fields and geographic regions. So despite the data source being open, the bibliometric indicators calculated based on such data cannot be fully trusted, which limits their reliability and application scope.
Situation in Switzerland
In Switzerland, there is a growing effort for establishing fair, DORA-compliant research assessment indicators adapted for the Swiss community. However, for the reasons described above, this effort cannot fully succeed without an open, high-quality dataset, which covers metadata of research outputs (such as publications, datasets, conference proceedings…) produced by researchers affiliated at Swiss HEIs.
TOBI aims to contribute to this effort by systematically evaluating existing open bibliometric and altmetric data sources in a subset related to Swiss HEIs. The analysis will help to identify critical gaps in data quality, while addressing the issue of deduplicating and merging of items, as well as improving the interpretability of scientometric indicators.
Aims of TOBI
The results of the project will be made publicly available and will show to which extent open bibliometric data could be a viable alternative to commercial bibliometric datasets for the Swiss research community.
Additionally, associated open-source code will be made available in this project to empower institutions to try out and scrutinize classical and novel research-assessment indicators on their own.
The project outcome will be relevant not only for the research output assessment of individual Swiss HEIs (e.g., for evaluating their strength and weaknesses), but also for accessible and transparent research-monitoring initiatives – e.g., of Open Access publishing, research funding, or technology transfer in Switzerland.
Open Peer Review
swissuniversities follows the open peer review principle. This means that anyone can access the abstract and the reviews of the TOBI proposal here.