Publication Metrics
A prerequisite for bibliometric analyses of research output is the responsible use of metrics and key performance indicators. KIT Library supports you in this endeavor and offers a wide range of training courses and tools, as well as personalized advice when needed.
sprungmarken_marker_6888
My Research Statistics
The KITopen Metrics tool can be used to calculate key performance indicators—for example, for annual reports, rankings, and program and center progress reports within the framework of program-oriented funding (PoF).
KITopen Kennzahlen
What is Bibliometrics?
Bibliometrics refers to the quantitative analysis of bibliographic information from (scholarly) published texts, such as journal articles or monographs. Bibliographic data includes all metadata associated with a publication and provides information about the authors, editors, and the journal. In addition, databases also record the references or article citations that cite a paper. This information can be statistically analyzed and provides an indication of the visibility and impact of a scholarly article. Bibliometric indicators—often referred to as metrics or indicators—are based on mathematical and statistical models. The analysis examines characteristics related to individual publications, individuals, or institutions, as well as publication vehicles such as journals.
The analysis of scholarly output in the form of publications (e.g., articles, research data, or presentations)—for example, through publication and citation analyses—is a key component of research assessment. Research assessment takes place in all divisions of everyday research:
- Analysis and monitoring of publications and their impact within the scientific community and beyond
- Evaluation of researchers and organizations in the context of awards, grant allocations, or hiring processes, such as for faculty appointments
- Identification and evaluation of scientific trends
- Selection of appropriate publication venues and development of a publication strategy
- Identifying potential new collaboration partners or interdisciplinary research areas
Bibliometrics and other quantitative surveys should never be used on their own to evaluate research performance and its impact. An evaluation should always be based on a qualitative analysis. Key metrics serve as a useful supplement and an evidence-based data foundation in this context.
KIT Library Services
KIT Library conducts bibliometric analyses for KIT researchers and institutions.
Conducting Bibliometric Analyses
KIT Library conducts the following bibliometric analyses for KIT:
- Compilation of bibliometric metrics based on KITopen, “Web of Science,” and “Scopus” for individuals, institutes, and divisions at KIT
- Identification of outstanding researchers at KIT for the awarding of research prizes and grants, as well as support in decision-making for tenure-track appointments
- Publication-based collaboration and citation network analyses
Bibliometrics and Reporting with KITopen
As a university bibliography, KITopen consolidates all of KIT’s scientific publications in one central location. In addition to bibliometric analyses, KITopen serves as the foundation for scientific reporting within the framework of the Helmholtz Reports.
The following tools are available for bibliometric evaluations:
- Calculation of traditional and alternative bibliometric metrics for KIT publications using the KITopen Metrics Tool
- Generating formatted publication lists for researchers and institutes using KITopen Publication Lists
If you have questions about bibliometric tools in KITopen, please feel free to contact the Bibliometrics Team. If you have questions about your publications in KITopen or your KITopen account, the KITopen Team is here to help.
Additional Services
In addition to basic bibliometric analyses, the following services are available to you:
- Training sessions and presentations on bibliometric analyses and research metrics as part of the information literacy programs
- Customized training and consulting services for institutes and research groups—on-site or virtually
- Customized consulting on academic visibility and publication strategies, as well as author profiles
How Can I Raise the Profile of My Research?
Tips on how to increase the visibility and impact of your publications:
Author profiles such as ORCID help you showcase your publications and activities in a consolidated manner
Share your activities on academic networks (e.g., LinkedIn, ResearchGate)
Use Open Access models for your text publications and the FAIR principles for your research data
Using Metrics Strategically
Bibliometric evaluations are based on quantitative characteristics, known as metrics or indicators. The purpose of these metrics is to monitor, rank, and assess the impact of scientific publications. They are part of research evaluation and can support qualitative evaluations.
Depending on the research questions, different metrics may be used. All indicators or metrics have limitations and should not be used without being placed in context.
Person-based metrics or indicators serve as tools for measuring research performance by making an individual’s publications measurable and comparable through specific metrics. The most common metrics are publication and citation counts, as well as metrics derived from them, such as the h-index.
The h-index, also known as the Hirsch index, was developed in 2005 by physicist Jorge E. Hirsch. The h-index corresponds to the number of publications h that have at least h citations or more. To calculate it, a researcher’s publications are sorted in descending order by the number of citations. One advantage of the h-index is its resilience to outliers. A single highly cited article does not influence the h-index, as it always considers citations in relation to the total number of published articles and their citation counts.
Example: A researcher has published 10 scholarly articles. Of these, 3 articles have been cited 12 times, 1 article has been cited 5 times, and the remaining 6 articles each have a single citation. In this case, this person’s h-index is 4, since 4 of her articles have been cited at least 4 times. Suppose one of these already highly cited articles were to receive even more citations. In that case, this researcher’s h-index would remain 4, even though her total number of citations has increased.
Limits of Bibliometrics and Negative Effects
The basis for selecting appropriate bibliometric indicators depends on the research question underlying the evaluation, particularly whether the goal is to measure the performance of individual researchers or to monitor a specific research topic.
Bibliometric indicators—regardless of the reference variable—can serve as a useful initial measure for assessing the impact and visibility of research output. They do not replace qualitative evaluation. The term “indicator” itself makes it clear that they represent only a part of reality. They should never be regarded as a direct representation of reality. Likewise, it is necessary to always consider multiple metrics and key figures together, as individual numerical values may be distorted due to systematic distortions and bias effects.
The heavy weighting placed on individual bibliometric indicators—particularly the h-index and the JIF—is placing increasing pressure on researchers and institutions to publish—often referred to as the “publish or perish” dilemma. Rankings based on other, comparable principles further contribute to this negative trend. At its worst, this leads to a disregard for the principles of good research practice, for example through fraudulent practices such as predatory publishing and paper mills. These practices harm not only individual researchers or institutions, but also the scientific system as a whole.
The h-index is provided by all major databases. This is also where the weaknesses of this indicator become apparent: Depending on the scope of the indexed articles, the publication counts used to calculate the h-index vary significantly across different databases. Furthermore, it is not a standard tool for comparing researchers across different disciplines or at different career stages. For example, a researcher with many years of experience in research will have a higher h-index than an early-career researcher.
Researchers in disciplines with a high volume of publications, such as medicine or physics, have a significantly higher h-index than researchers in the humanities and social sciences. This means that the academic culture of the discipline must be taken into account when considering the h-index. A comparison across different disciplines is not meaningful.
The increasing pressure to publish—also known as the “publish or perish” dilemma—has negative consequences at various levels. Particularly critical is the (deliberate) disregard for good research practice. Examples include the so-called “salami slicing” method in publishing, in which research findings are not presented in a single, comprehensive publication but are instead split into several small articles. Another example at the individual level is the repeated, unfounded self-citation of one’s own publications.
In addition to these misconducts at the individual level, commercial—and even criminal—methods are on the rise. These are grouped under the umbrella term “predatory practices.” “Predatory publishers” are frequently mentioned in this context. These publishers exist solely for the commercial exploitation of research findings, without regard for any ethical principles of scholarly publishing. Among other things, peer-review processes are faked or simply do not take place. Furthermore, these publishers sometimes employ fraudulent methods by imitating the websites of reputable journals or even hijacking their web addresses. For laypeople, it is difficult to distinguish these fakes from the originals. If you have nevertheless fallen victim to a predatory publisher, do not hesitate to report it.
The phenomenon of “paper mills” operates similarly to predatory publishers but follows a different principle. As the name “paper mills” suggests, the focus here is purely on generating as much output as possible. For a fee, researchers can have academic articles generated for them, complete with co-authorships and references. With the advent of artificial intelligence, this scam has gained momentum and is increasingly flooding the academic system with information that is sometimes plagiarized or false.
Tools like Think.Check.Submit help identify predatory publishers. If in doubt, KIT Library can advise you and is available to answer any questions.
Research Evaluation in Transition
As early as December 2019, the Karlsruhe Institute of Technology (KIT) signed the San Francisco Declaration on Research Assessment (DORA), making it one of the first signatories in Germany. In doing so, KIT has joined the efforts of many research institutions to reform the evaluation of research performance and, in particular, to limit the use of journal-based metrics.
Other initiatives aimed at reforming research assessment include the Coalition for Advancing Research Assessment (CoARA) and the Leiden Manifesto for Research Metrics.
This course provides an introduction to predatory publishers, the warning signs you should look out for, and the tools that can help you identify reputable publishers. In addition, the course offers guidance on what to do if you inadvertently publish with a predatory publisher.
To the course
The course "Introduction to Responsible Research Assessment" provides an introduction to research assessment, bibliometrics, and metrics. Learn about the challenges associated with weighting quantitative metrics, how to use metrics responsibly, and how you can drive change within your institutions.
Link_more
The Helmholtz Open Science Office is actively advocating for reforms in research evaluation, particularly within the Helmholtz Centers. Various task groups are developing new approaches and standards for research evaluation and quality indicators within the Helmholtz Association.
find out more
As part of the KITopen and ORCID campaign, themed “This is me. This is my research.,” the campaign demonstrates how easy it is to link accounts and the added value this provides.
Zur Kampagne
The outcome of the BMFTR project “Development of a Reference Model for Reporting in Research Institutions Based on DORA – ERRED” (2023–2025) is a collection of indicators and measures for the implementation of a reform of research evaluation.