The Research Information Network (RIN), in collaboration with the Centre for Information Behaviour and the Evaluation of Research (CIBER) at University College London, has been investigating the use, value and impact of e-journals in the UK for the past couple of years. We've shown a rapid rise in usage across UK universities, as well as notable differences in patterns of use in different subject areas and different institutions. But one of the most interesting features of the analysis investigates relationships between levels of usage and research performance.
We brought together data for 112 UK universities on serials expenditure and usage; numbers of PhD awards; income from research grants and contracts; and articles published and their citation impact. Our aim was to investigate whether relationships exist among any of these variables. Table 1 shows some partial correlations between aspects of library provision and research outcomes for 2007–2008, with article downloads correlating positively with all four measures of research performance. The correlations are significant and independent both of institutional size and the balance of research activity across different disciplines.
But correlations are not causes. So we tried to build a more dynamic model, using data from a five-year period rather than a single year, to test a series of six hypotheses:
H1: Spending drives use (as in Figure 1)
H4: Use drives spending
We tested these hypotheses using a structural modeling technique, introducing a time lag of three years so that we could ask such questions as “Is spending on e-journals in Year 1 a good predictor of research outcomes in Year 3?” (Hypothesis 1).
A positive answer to that question still doesn't necessarily imply cause and effect. But it does suggest that if there is a change in the driver (in this case, expenditure) in Year 1, it's likely that the target (in this case, usage) will change in Year 3. And because we can test the reverse hypothesis - that use drives spending (H4) - we can get closer to understanding directionality as well.
The results are summarized in Figure 2, which shows three strong driving relationships:
- Expenditure drives use. Indeed, it's a precondition for use since you must purchase a license or make some other payment to gain access to any content that is not open access.
- Most powerfully, the use of e-journals drives subsequent research performance.
- Research success drives more usage of e-journals. There is thus a strong positive feedback loop between levels of usage and research outcomes: They each feed off each other.