Published Aug 26, 2013 in the Newsletter Issue: The Social Library — 2013
Academic research and publishing have transitioned from paper to online platforms, and that migration has continued to evolve from closed platforms to connected networks. With this evolution, there is growing interest in the academic community in how we might measure scholarly activity online beyond formal citation.
The collection, analysis and presentation of data about how people share and discuss academic papers are known as altmetrics. Over the last year altmetrics has received increasing attention in journals, conferences and social media. (A good starting point for learning about altmetrics is http://altmetrics.org/manifesto.)
Data collected by altmetric platforms come from many sources, ranging from PDF downloads on scholarly platforms to mentions on everyday social websites.
Altmetric usage figures may include whole article data, source data and component data.
- Scholarly usage data: Web page views, PDF downloads
- Scholarly reference: Bookmarking, shares and recommendations from CiteULike, Zotero, Mendeley
- Mass media mentions: NYTimes, BBC, The Washington Post
- Social media mentions: Twitter, Facebook, Delicious
- Data and code usage: Dryad, GitHub
- Component mentions: SlideShare, Figshare
Various altmetrics websites and tools have different sources and specialties: ImpactStory is strong on data, code and components, whereas Altmetric excels in mass media references. Plum Analytics claims to include data on interlibrary loans, and other book-related and library-specific data.
The Altmetric donut is a familiar sight to Scopus users. Each color represents a data source, and the number in the center is computed according to Altmetric’s algorithm.
Although its reports are less colorful than the Altmetric donut, ImpactStory classifies data from different types of sources to identify various activities. For example, it separates public discussions from scholarly ones, and attempts to identify influential tweeters.
The number of use cases for altmetrics is expanding as ideas are shared, but it is clear that many are relevant for librarians and others who curate and appraise published material.
Measuring collection performance
Both ImpactStory and Altmetric allow users to create collections of documents using digital object identifiers (DOIs) or other IDs. ImpactStory also lets users collect data for specific web pages and download data as spreadsheet files. Both platforms have a computer interface (API) for ingesting data directly into a database.
Measuring institutional repositories’ performance
Because ImpactStory allows users to collect data for web pages, it is possible to automatically build a collection of documents from an institutional repository. This potentially allows a librarian to monitor the institution’s relative performance. Altmetric is keen to explore this area too, and has worked with institutions to develop solutions.
Comparing relative performance of institutions and researchers
You can use altmetrics to compare the social usage of the output of both institutions and individual researchers. ImpactStory may make the latter easier by allowing for comparison of registered articles by ORCID (the Open Researcher and Contributor ID repository).
Creating social impact statements
A recent initiative considers whether the social impact of research can be meaningfully assessed with altmetrics. Changes to funding infrastructure in the UK have led to mandatory social impact assessments for Research Council funding, and this trend will likely spread. There are not yet methods for computing social impact — either relative to other research or in absolute terms — but a wider vision of altmetrics could provide a good picture of the extent to which research is communicated to society.
It is important to stress the current limitations of altmetrics. Although it is tempting to use altmetrics for direct comparisons, they actually do not permit anything but cautious evaluation of like-for-like factors. For example, year-on-year growth in social sharing has to be viewed in context of the continued growth of existing platforms, which are naturally attracting new users, and new services still being launched — with the consequential migration of users. Furthermore, we know that different disciplines have very different citation patterns, and early research indicates that social usage is at least as varied. Without a fundamental underpinning from greater research into the field of altmetrics, there is no proven validity in wider comparison.
So why get involved now? As a field of study, altmetrics is only three years old — and remember, it took roughly 20 years for citation analysis to become an accepted measure of a journal’s impact. We don’t think it will take that long for altmetrics to become acceptable, although some issues (for example, understanding different patterns of behavior across disciplines) are complex.
However, these new metrics allow academics to tell stories beyond citations. For example, they can provide evidence of a work’s geographic reach or its impact in a wider population.
Furthermore, it is easy to get involved. Although Altmetric is a commercial operation, it has some free services, and ImpactStory services are all free (it is a grant-backed, nonprofit organization). Furthermore, you don’t need permission to view a document’s altmetrics; a DOI is all you need.
Librarians are one of the key players in this research mix, and we encourage you to use these tools to help academics, institutions and researchers tell their stories.
Note: This article was supported by the Data2Semantics project in the Dutch national program COMMIT.
Priem J., Groth P., and Taraborelli D. (2012). The Altmetrics Collection. PLoS ONE 7(11): e48753. doi:10.1371/journal.pone.0048753 http://altmetric.com/blog/altmetrics-in-academic-libraries-and-institutional-repositories/.
Stacy Konkiel http://www.youtube.com/watch?v=lA-14vXfFwI http://www.swets.com/blog/altmetrics-for-librarians-and-institutions-part-i#.UZtMGOvg-Ik.
Taylor, M. (2013). The Challenges of Measuring Social Impact Using Altmetrics. Research Trends, 33. http://www.researchtrends.com/issue-33-june-2013/the-challenges-of-measuring-social-impact-using-altmetrics/