Where does altmetric data come from?
Altmetric data are compiled from the usage, recommendations, shares and reuse of a common document, which is identified by its DOI, URL or other ID. So when people tweet about a paper, upload it into Mendeley or write a blog post about it and use the reference, they are generating altmetric data.
Different data have different characteristics
Different kinds of data have very different characteristics .The intention of a person tweeting a reference to a paper isn’t going to be the same as that of someone who is uploading a paper to Mendeley. It’s not the same as somebody who is downloading a dataset from Dryad. Nor is it the same as someone writing a newspaper article or a blog post. So although all of these items are classified as altmetrics, one type of data cannot necessarily be equated with another. Likewise, different platforms also have different characteristics including discipline bias.
Tweets tend to be summaries of serious scientific papers that have been written about in the mass media, but volume can be driven by “hot button” topics or contentious issues like policy and gender, stem cell research and climate change. Papers that are highly shared on Mendeley tend to refer to primary research.
Can altmetrics be boiled down to a single number?
No canonical source of altmetric data exists. It may be tempting to think about applying some kind of algorithm to altmetric data derived from multiple sources and coming up with a single number, as an h-index or an Impact Factor. However, altmetrics are not one thing; attempting to express them as one thing will inevitably fail.
Bringing sources together
At Elsevier we believe that it makes more sense to cluster similar kinds of activity intelligently – social activity, mass media, scholarly activity, scholarly comment and reuse – because this more accurately represents similar behaviors and intentions. It tells a better story. We group together social networks like Twitter, Facebook, Pinterest and Delicious. Likewise blogs, F1000 reviews and other kinds of scholarly activities would constitute another group.
The gaming of altmetrics: Will people cheat?
As altmetrics continue to gain acceptance as viable and valuable metrics, the likelihood of cheating also increases. People may try to manipulate the figures to their advantage. This type of behavior has been observed previously within the scholarly environment as evidenced by attempts to manipulate Impact Factors.
The scientific, publishing and library communities have taken these issues very seriously. Although monitoring these activities is complicated because pattern analysis, usage analysis and network analysis have to be taken into account, expertise in detecting fraudulent downloads and tweeting is growing. Working to identify fraudulent activity, the Social Science Research Network has already encountered a situation in which the same paper was downloaded a hundred times to boost its ranking on the site.
In looking at this problem, a key question emerges: Is this generating any impact? I think it is one thing to go onto a website and pay a few dollars to get a thousand tweets or buy 10 blog posts, but is anybody following those Twitter accounts, particularly people within the scholarly community? This is research that can and will be done.
From the trivial to the pursuit of knowledge
One of the biggest criticisms of altmetrics is that they can highlight something deemed worthy of mass media attention, such as a paper published last year about left-handedness in tail-wagging dogs. Although a serious scientific study, the title made it attractive for tweeting and retweeting, so there was a lot of activity generated. But analysis showed that those types of stories don’t tend to hit headlines in the same way that really serious articles do.
Should librarians buy in or build their own?
Librarians and the communities they serve can access altmetrics and make good use of them. There are several options:
- Free-to-use applications like ImpactStory
- Relationships with corporate entities like Plum Analytics and Altmetric.com (higher service levels with large numbers of DOIs and robust analytics)
- Build your own
When it comes to building your own, one approach is to use resources like PLOS and ImpactStory, which have open source code available to download, edit and install on library servers. It’s not something for the faint hearted, but it’s certainly something that a competent techy can manage. Another option is for the library to build its own from scratch using APIs and connections to the servers. Libraries can sign up to these for free and then query those servers for links through to papers of interest. Again, it’s not easy, but it’s a relatively comprehensive thing to undertake if this route will serve the library’s objectives.
Telling the story
Identifying and rewarding reuse of research outputs are essential components of the scholarly communication environment. This will increase in importance over the next few years. Altmetrics data reveal what scholars are using to research and what kinds of research outputs are being reused. They tell us the story of what people in society are talking about.