Subscribe for Free
Send to a Friend
 |
|||||||||||||||||||
Books, bingo, blogs! All the world’s a stage
 Mad libs go to Taupo
How do librarians and publishers add value to content? Library Connect Seminars in Japan and Korea shed some light
|
Evaluating research from the university administrator perspective
Why is evaluating research output important from the university administrator’s perspective? On March 5, 2007, I had the opportunity to gain some insight from Charles Zukoski, the vice chancellor for research at the University of Illinois at Urbana-Champaign. “Measuring impact is fundamental to the university strategy,” noted Zukoski, but he was not talking about impact factor. “Citations are fine for impact [of publications], but in other areas it doesn’t work as well…. In new emerging technology there’s not enough time to cite anything. New technology may only last two years, but the impact can be huge.”
Instead, Zukoski emphasized the importance of measuring the impact of UIUC faculty as they juggle international multidisciplinary collaborations, participate in blue ribbon task forces and serve as spokespeople to the media for the research enterprise. An institution, its faculty and the research they produce impact the global community in ways beyond published papers, and the UIUC Research Office is keen on measuring that impact. At one time, researchers were evaluated primarily on the quantity of papers they published and the quality of the work. But paper output, citations and impact factors increasingly are unable to capture the whole picture. Are researchers known? That is, are they known and recognized via awards and in the popular press? What is the impact of social networking?
“Funding does not regenerate funding. But reputation does.”
At the end of the conversation, Zukoski affirmed that what is important to the university is its reputation. And that reputation ultimately leads to more funding. When I asked then why funding itself was not higher on the priority list for the institution, the answer was simple: Fame continues to generate funding. Said Zukoski, “Funding does not regenerate funding. But reputation does.”
COUNTER keeps innovating
Since its start in 2002, COUNTER (Counting Online Usage of NeTworked Electronic Resources) has developed and registered a number of standards for measuring and reporting the usage of electronic resources. The first phase concentrated mainly on the development of standards for journal, database and book usage reports and produced Codes of Practices. These basic activities, focusing mainly on defining the content and layout of usage reports, were quite urgently needed. About 65 vendors, publishing about 70% of all available STM journals, are currently COUNTER-compliant.
Recently, COUNTER entered a new phase by implementing audit tests to ensure that usage data reported by vendors is reliable and sustainable over time. As a major step forward COUNTER made a nonexclusive agreement with the auditing company ABC-E in the UK. A first batch of vendors was audited in June 2007, and it’s expected that all additional compliant vendors will be audited in the remainder of 2007 and 2008.
Another milestone in the works is the development of a so-called COUNTER Usage Factor, a measure in its definition quite similar to the well-known Impact Factor for citations, but quite different as it expresses the informal usage-based value of articles and journals to users. A working group has been installed and cooperation is sought with other bibliometric initiatives and institutes.
Another milestone in the works is the development of a so-called COUNTER Usage Factor.
Finally, work is being done regarding standardization of report formats to allow librarians to easily import COUNTER-compliant usage reports in XML format into local library management systems. The most productive cooperation in this area to date has involved COUNTER and the NISO-SUSHI initiative.
It’s significant that all this work has been accomplished by a single organization consisting of librarians and vendors both. While serving as Elsevier’s representative to COUNTER, since its founding in fact, I've participated in hefty discussions within the COUNTER Executive Committee regarding what to report or how to ensure reliability of data. However, the major aim of COUNTER, to cooperatively work on standards for usage reports, always proves to be stronger!
www.niso.org/committees/SUSHI/SUSHI_comm.html
www.projectcounter.org
