Measuring your Research Impact

Research Impact Graphic

Image source: J. Priem, D. Taraborelli, P. Groth, C. Neylon (2010), Altmetrics: A manifesto, 26 October 2010. http://altmetrics.org/manifesto

Ever wondered what the impact of your research is? The following guide is a brief introduction to research impact: what it is and how to measure it.

What is Research Impact?

Research Impact Zones Graphic

Research impact refers to the value and benefit associated with using the knowledge produced through research. Research can have an impact both in academia and in society at large. Zones of impact include knowledge, teaching and learning, practice, society and environment, economy and public policy (Emerald Publishing Group Ltd). A variety of metrics can be used to assess research impact. This includes peer review, citations, usage, and non-traditional metrics (altmetrics) such as social media metrics (altmetrics.org, 2010).

There are various ways your research impact can be assessed. This guide will focus on quantitative methods of assessing research impact. Qualitative methods, such as peer-review, are also an important part of research evaluation.

Bibliometrics

Bibliometrics is the statistical analysis of written publications. Citation analysis is a commonly used bibliometric methods. "Many research fields use bibliometric methods to explore the impact of their field, the impact of a set of researchers, or the impact of a particular paper" (Wikipedia). Various metrics exist for journals and authors.

  • Journal Metrics

    • Impact factor (IF)
      • the average number of citations to articles published in the last 2 years in a particular journal (data source: Web of Science)
    • 5-year Journal Impact Factor
      • the average number of times articles from the journal published in the past five years have been cited in the Journal Citation Reports year (data source: Web of Science)
    • Scimago Journal Rank (SJR)
      • the number of citations received by a journal and the importance or prestige of the journals where such citations come from (data source: Scopus)
    • Source Normalized Impact per Paper (SNIP)
      • the ratio of a journal's citation count per paper and the citation potential in its subject field (data source: Scopus)
    • Impact per Publication (IPP)
      • the ratio of citations in a year to scholarly papers published in the three previous years divided by the number of scholarly papers published in those same years (data source: Scopus)
    • Eigenfactor Score
      • the number of times articles from the journal published in the past five years have been cited in the JCR year, but it also considers which journals have contributed these citations so that highly cited journals will influence the network more than lesser cited journals (data source: Journal Citation Reports)
    • Article Influence
      • divide a journal’s Eigenfactor Score by the number of articles in the journal, normalized as a fraction of all articles in all publications. (analogous to the 5-Year Journal IF) (data source: Journal Citation Reports)

  • Author Metrics

    • Number of publications
      • a measure of productivity
    • Citations counts
      • a measure of impact
    • H-index
      • single indicator that combines measures of productivity and impact
      • A researcher’s H-index is the largest possible number N for which N of the researcher’s publications have been cited at least N times (e.g. out of 100 publications, at least 30 publications have been cited at least 30 times; hence, the H-index is 30).

  • Bibliometric Tools

    • Journal Rankings - journal impact is often used as a surrogate for article impact but they are not equivalent
    • Author Identifiers - a unique identifier that distinguishes you from other researchers with the same name, allows you to gather together all your research that has been published with variations on your name (i.e. initials, full name)
    • Author Profiles - allows you to gather together your research in one place and let others see your research outputs

  • Limitations

    • Time favours citations
    • Review studies are cited more often in all fields
    • Many articles are never cited
    • Very important to compare like to like
    • Different databases can give different results
    • No database is perfect
    • Only part of the picture

Altmetrics

"Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments" (Altmetrics Collection, 2012). Although often used to assess impact of an individual article (article level metrics), altmetrics can also be used at the level of journal or author. One of the hallmarks of altmetrics is that they can measure impact in a shorter timeframe than traditional bibliometric measures: days/weeks rather than years.

  • Altmetric Tools

  • Limitations

    • No reporting standards exist yet
    • No way to differentiate between quality and popularity
    • Not currently as user friendly as impact factor (arguable)
    • Vulnerable to manipulation, "gaming the system"
    • Not as easily applied to books or works of art

How-to

Step-by-step instruction for measuring your research impact using bibliometric tools.

Setting up Author IDs/Profile(s)

Google Scholar image

Google Scholar Profile

  • Go to scholar.google.ca and click the “My Profile” link at the top of the page
  • On the first screen, add your affiliation information and Carleton email address so Google Scholar can confirm your account.
    • (option) add any name variations that you use for publishing and research,
    • (option) add your research interests, papers you’ve published, etc.
    • (you can come back to these steps at any time)
  • Click “Next Step,” and that should send you a confirmation email to your Carleton account that will confirm your Google Scholar.
  • Once you’ve confirm registration, you can make your profile public, add articles, etc.

ORCID image

ORCID

  • Does not provide an h-index

Researcher ID image

ResearcherID (Web of Science)

  • Set up ResearcherID account
    • Sign up at ResearcherID.com
  • Add articles to ResearcherID
    • Click Add
    • Search Web of Science
  • Combine profiles if multiples
  • Merge duplicate profiles

Scopus Author Identification

  • Set up Scopus Author account
    • Automatic process, unique identifiers given based on name, affiliation, subject source and co-authors
  • Add articles
    • Automatic, may need to combine Scopus Author accounts or request that articles be added
  • Combine profiles
    • Login to your Scopus account
    • Perform an Author search using your first and last name
    • If two or more profiles appear that you own, select the profiles then click “Request to merge authors”

Finding your H-index/Publications

  • Web of Science
    • H-index
      • Search by author
      • Click Create Citation Report
  • Scopus
    • H-index
      • Do Author search
      • Click on authors’ name(s).
  • Google Scholar (you must have a profile to do this)
    • H-index is available on Google Scholar Citation Profile page

Resources

Content last reviewed: