Skip to main content

Research Impact Metrics: Article-Level Metrics

This is an introduction to different research impact metrics and tools for author disambiguation.

Why Journal-Level Metrics?

Declaration on Research Assessment logoResearchers have gradually become concerned about the appropriateness of journal-level metrics for research assessment.  After meeting in Dec. 2012, a group of journal publishers and editors have released the San Francisco Declaration on Research Assessment (DORA) to highlight the need to improve how research impact is evaluated.  In sum, they point out: 

  • the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;
  • the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and
  • the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

They have put forward a list of recommendations for researchers, institutions, funding agencies, publishers, and organizations that supply metrics. Thousands of individuals in different academic communities around the world have signed DORA. Successful researchers have also shared their thoughts about why Journal Impact Factors do not deserve so much weight and attention. Similarly, Randy Schekman, the wininer of the 2013 Nobel prize for medicine, has articulated his concerns over unwarranted significance of publishing in prestigious "luxury journals." 

Benefits of Article-Level Metrics

The Public Library of Science summarizes how article-level metrics (ALMs) are valuable to different stakeholders in the scholarly communication lifecycle:

Researchers

  • Track the impact of your research and share with others (funders, promotion boards, etc.)
  • Navigate and filter research to find what is most suited to your needs
  • Find collaborators based upon the impact of their work and relevance to yours
  • Measure research impact with metrics at the article level, instead of at the traditional journal level
  • Search, filter, and aggregate scholarly content based on research impact with metrics at the article level

Research Institutions

  • Track dissemination of articles published (types of channels, rate of growth, etc.) by members of the institution
  • Access up-to-date information on the research progress of faculty members, useful for tenure and promotion decisions
  • View data on downstream impact of publications
  • Roll up data for custom reporting of department’s research activities

Research Funders

  • Efficiently track progress of the research impact of grant awardees in an automated fashion with the most relevant signals
  • Measure research impact with metrics at the article level, instead of at the traditional journal level
  • Measure evidence of wider engagement
  • Monitor and enhance engagement from a specific audience
  • Identify high-impact research outputs much earlier
  • Analyze trends of past and future funding programs
  • Conduct network analysis to evaluate researcher communities

Publishers

  • Attract authors by offering data on the audience’s research interest
  • Track dissemination of articles published (types of channels, rate of growth, etc.)
  • Provide content distributors data on downstream impact of publications
  • Gain comprehensive understanding of how publications are disseminated and accessed
  • Report on the usage of the article in ways that suit your business needs

Criticism of Journal-Level Metrics

These are some of the discussions about the shortcomings of journal-level metrics:

Article-Level Metrics

Researchers increasingly recognize the significance of assessing research at the article level with both traditional and novel data points.  The SPARC Primer on article-level metrics (ALMs) notes that ALMs:

  • pull from distinct data streams to gauge the "scholarly visibility" and the "social visibility" of research results. 
  • incorporate both shorter- and longer-term data points in the analysis to present a fuller picture of an article's impact over time.
  • can complement traditional metrics because they are more granular and more immediate. 
  • are not proprietary products and thus research communities can develop, distribute, and build upon ALM tools in a manner that unlocks impact metrics.

However, just like other metrics, ALMs can be gamed and cannot evaluate the quality and intent of the comments on an article. 

This video provides an overview of ALMs:

Article-Level Metrics in Action

Here is a snapshot of the metrics for an article published by PLOS ONE:

Altmetrics in action