Get the latest articles and downloads sent to your inbox in a monthly newsletter.

Get the latest articles and downloads sent to your inbox in a monthly newsletter.

The use of e-journals and research outcomes: Are they related?

Michael Jubb, Research Information Network | June 01, 2011

The Research Information Network (RIN), in collaboration with the Centre for Information Behaviour and the Evaluation of Research (CIBER) at University College London, has been investigating the use, value and impact of e-journals in the UK for the past couple of years. We've shown a rapid rise in usage across UK universities, as well as notable differences in patterns of use in different subject areas and different institutions. But one of the most interesting features of the analysis investigates relationships between levels of usage and research performance.

We brought together data for 112 UK universities on serials expenditure and usage; numbers of PhD awards; income from research grants and contracts; and articles published and their citation impact. Our aim was to investigate whether relationships exist among any of these variables. Table 1 shows some partial correlations between aspects of library provision and research outcomes for 2007–2008, with article downloads correlating positively with all four measures of research performance. The correlations are significant and independent both of institutional size and the balance of research activity across different disciplines.

But correlations are not causes. So we tried to build a more dynamic model, using data from a five-year period rather than a single year, to test a series of six hypotheses:

H1: Spending drives use (as in Figure 1)
H2: Use drives research success
H3: Spending drives research success

H4: Use drives spending
H5: Research success drives use
H6: Research success drives spending

We tested these hypotheses using a structural modeling technique, introducing a time lag of three years so that we could ask such questions as “Is spending on e-journals in Year 1 a good predictor of research outcomes in Year 3?” (Hypothesis 1).

A positive answer to that question still doesn't necessarily imply cause and effect. But it does suggest that if there is a change in the driver (in this case, expenditure) in Year 1, it's likely that the target (in this case, usage) will change in Year 3. And because we can test the reverse hypothesis - that use drives spending (H4) - we can get closer to understanding directionality as well.

The results are summarized in Figure 2, which shows three strong driving relationships:

  1. Expenditure drives use. Indeed, it's a precondition for use since you must purchase a license or make some other payment to gain access to any content that is not open access.
  2. Most powerfully, the use of e-journals drives subsequent research performance.
  3. Research success drives more usage of e-journals. There is thus a strong positive feedback loop between levels of usage and research outcomes: They each feed off each other.
Other linkages are much weaker. Thus any direct relationship between expenditure on e-journals and subsequent research success is weak. This is probably because additional expenditure on e-journals is unlikely to lead to tangible improvements in research performance in as little as three years. The reverse relationship is stronger, suggesting that some decisions on expenditure are related to success in winning research grants and contracts.
We have not, of course, conclusively established cause and effect: Many factors in the wider environment are not included in the model, and some third element may be at work as we demonstrate that usage is a strong predictor of research success. More detailed modeling, for individual and groups of universities and over different time periods, is required to test a range of hypotheses. Nevertheless, both libraries and universities should consider this evidence carefully in reaching decisions on both the future development of their e-content collections and how their services can support the effective use of e-journals.