Subscribe for Free
Send to a Friend
 |
|||||||||||||||||||
Books, bingo, blogs! All the world’s a stage
 Mad libs go to Taupo
How do librarians and publishers add value to content? Library Connect Seminars in Japan and Korea shed some light
|
Research Performance Measurement is revving up
Research Performance Measurement (RPM, also known as bibliometrics) has entered an era of rapid development in terms of indicators. Whether this has been spurred on by the acceptance of the h-index or by science turning towards a global metric-based system, one thing is certain: The status quo of the last few decades is being challenged.
Since the ’60s, when it was introduced, the Impact Factor has dominated RPM indicators. It’s calculated by identifying the number of citations in one year to articles published in the previous two years and then dividing that number by the number of articles published in those same two years. Originally it was intended as a collection management tool, but has since evolved into a metric used for evaluation of science at all levels as well as evaluation of authors. This can have far-reaching consequences for an author’s grant applications, promotion and tenure since the metric is directly influenced by the performance of specific journals and is thus for a large part beyond the author’s control.
The status quo of the last few decades is being challenged.
The h-index, created by Professor Jorge Hirsch in 2005, is a means of evaluating a researcher based only on her or his own published work. When using this metric, one must take care to consider different citation patterns in subject fields and publication periods. Scientists are more than merely the sums of their articles, and when using the h-index for RPM, users must look into the relevancy of an author’s publication history and trends in citations received.
It’s interesting to note when looking at journal evaluation that limitations of the impact factor are being addressed in new journal metrics. One such metric, the Eigenfactor, incorporates five years worth of citation data instead of two and corrects for differences in citation patterns across fields.
An interesting new dimension to RPM is Page Rank. This was introduced by Google to show the link popularity of websites and rank them accordingly. Its roots lie in the RPM of science itself which has embraced the assumption that “good” science gets cited frequently and is thus ranked higher. The incorporation of Page Rank means that journals are now being evaluated not only over a fixed time, but also as an effect of “prestige” afforded by other journals.
There are more indicators available, and I am confident many more, comprising new facets such as usage, will be introduced in the near future. It is promising that the trend is moving towards metrics for each level of evaluation, be it authors, journals or even subjects as a whole. It’s great that researchers and analysts will be evaluated and evaluating against an increasing number of appropriate benchmarks.
The relevance and acceptance of indicators will not be decided by organizations choosing to endorse one or the other, but by the researchers who are the core of Research Performance Measurement everywhere. We at Scopus make sure we stay involved in discussions about new measurements so we can supply the information and tools required by researchers and administrators to perform evaluation effectively, efficiently and as free as possible from potential bias. 
Explore More
http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor
R&D performance evaluation: We need to look at diverse metrics
Seogwon Hwang
When assessing the performance of research and development, we can see that evaluation has evolved along with the times. For example, in the past, integrity and ethics were largely neglected. But now, having experienced various scandals, government agencies, research institutes and publishers are establishing “Research Integrity and Ethics” committees and guidelines to reduce incidents of forgery, falsification or plagiarism. Today we hope that integrity and ethics are considered more carefully as papers are published and patents awarded.
Improving R&D performance evaluation can help us keep a focus on research integrity and ethics and can help address other problems. One notable such problem is that when relying on only one or two citation databases to evaluate research performance, the output of non-Western researchers is adversely affected because the databases encompass limited numbers of domestic journals (e.g., journals published in Korean).
What’s the best way to enhance the R&D evaluation process? Widen the scope through measures like these:
- Besides considering impacts of papers, consider impacts of patents and other activities such as technological consulting for small- to mid-sized enterprises.
- Develop and utilize R&D-productivity-measurement methodologies using appropriate tools such as Data Envelopment Analysis, Analytic Hierarchy Process or Balanced Scorecard.
- Increase researchers’ additive scores on education and external activities.
Improving R&D performance evaluation can help introduce more fairness for researchers worldwide and can boost R&D effectiveness. There’s a lot at stake. As we all know, the output of R&D includes publications, patents, prototypes, products, processes, services and standards — as well as increased knowledge and skills. Other outcomes include economic and social effects, and impacts in science and technology.
