Tag Archives: impact-factor

How to Measure the Impact of Scholarship

Stemming from a conversation with one of my faculty members, I began trying to define, explain, and provide support to the concepts of impact factors or other journal/article evaluation tools.  Being a smaller, more instruction-focused campus, we don’t currently use impact factor ratings as part of the scholarship evaluation of our faculty.  However, more and more my faculty are collaborating with researchers at other institutions, so they (and I) need to know how to speak the language of modern scholarly communication.  To help expedite your questions in the area, here are a few key terms to be familiar with and tools you can use to support your curious faculty.

Impact factor

  • measures the number of citations from the average article in a journal over a span of about 2 years
  • started in the 1960s by Thompson Reuters
  • originally used as a collection development tool to help identify most popular journals for library purchasing
  • impact factor for a journal is now incorporated as a way to evaluate individual article impact, influencing where authors try to publish
  • Problem: peaked around the 1990s, as the advent and increased utilization of the internet has moved people away from using print resources and only having scholarly communication in journals
Further Readings

The Thomson Reuters Impact Factor

The Demise of the Impact Factor

The Weakening Relationship between the Impact Factor and Papers’ Citations in the Digital Age

Other Factors

Eigenfactor – “measure of the journal’s total importance to the scientific community” – aka big journals=big scores

Altmetrics – tracking system that attempts to note not just the electronic article usage in digital forms like Twitter or CiteULike, but also other information resources like datasets or blogs.  This is tough to tackle but the various tools below are starting to develop some interesting methodologies

So, like many things, the digital age, the increased retrieve-and-shareability of research is changing how we consider the value of research.  In the full circle of things, I wonder how these other metrics, particularly Altmetrics, can impact our collection development, too.  I look forward to discussing these concepts and more at the ACRL Scholarly Communications Roadshow at JMU.

Research impact factors – How can libraries get in on this?

While browsing my Twitter feed, I stumbled across the article in the Chronicle of Higher Education discussing the latest tool for analyzing the impact of a researchers’ publication in a discipline.  Instantaneous feedback from programs like Google Analytics has helped bloggers, website creators, and librarians increase their understanding of who, where, and when their sites are being accessed and read.  Among the journal publishers, Elsevier is now coming to the impact analysis part with their SciVal Spotlight tool.  In contrast to other analyzers like Google Scholar, Reuters Thomson’s citation indices, and Springer’s AuthorMapper (it’s free!), Elsevier attempts to review and categorize articles (instead of journals) into one of their 80,000 clusters, allowing for “a much more precise picture of influential work in emerging fields.”  In conjunction to the bibliometric analyzer, Elsevier also looks to be creating a SciVal Funding database to connect researchers to funding opportunities.

Now, as libraries face budget cuts and collection development demands, tools like these could potentially be of great value in researching and ranking resources of interest and value for our institutions.  Our researchers would also benefit from access to this resource to understand other, related research in the field.  However, this all assumes a level of accuracy in the cataloging of these articles.

Sadly, I feel somewhat disappointed in the premature results, as the same problems that face the keyword indexing of journals in databases remains anything but precise and consistent, continues in the citation indexer.  While I don’t have access to Elsevier’s edition, I did test some searches in Springer’s AuthorMapper.

Image of AuthorMapper webpage, including search bar and Google maps mashup with location of authors
Image of AuthorMapper webpage, including search bar and Google maps mashup with location of authors

Although a variety of subjects are listed below for browsing, I chose to conduct my own topic search on Dante. What I found was that most of my results were not based on the author Dante Alighieri of the Commedia, but others, particularly in the sciences, that had Dante somewhere in their name and primarily were based in South America.

Map and list of keywords retrieved for "Dante" search results in AuthorMapper
Map and list of keywords retrieved for "Dante" search results in AuthorMapper

Like any good librarian, I decided perhaps the fault was my own for not refining my search phrase enough.  So, using his full name Dante Alighieri, I was able to find many more results related to the specific Dante I was after (yes, the results under Mineralogy do related to Dante Alighieri).  However, upon reviewing the various facets, I found that while 18 articles were tagged as Comparative Literature and Linguistics, 12 articles were still related to Medicine and Public Health.  While the concepts of medieval science do arise throughout Dante’s writing, 12 seems a bit excessive.  Upon reviewing the journals list, Deutsche Zeitschrift fur Chirugie (Langenbeck’s Archives of Surgery) is the 4th most popular journal to appear in the list!

Google Maps mashup and keywords listing for AuthorMapper results for Dante Alighieri
Google Maps mashup and keywords listing for AuthorMapper results for Dante Alighieri

These inaccuracies are evidence to just some of the problems bibliometric analyzers have in reviewing research, let alone the issues of citation inflation by colleagues and friends.  Even misspelling Dante’s name as Dante Aligheri still provides another single search results that falls outside of its proper retrieval space.

So, in short, while these tools may be helpful, like the Internet, information literacy will be key in deciphering these results as validity is not guaranteed.