Digital Repositories Roadmap Review

Charting progress

Over the last two years the level of repository activity has increased within the UK and beyond. Progress against the original Roadmap has been made on several fronts. A majority of UK HE institutions have implemented repositories. Institutional repositories (IRs) have been populated with content of various types, albeit much of the attention and discussion has focused on open access to peer-reviewed journal articles. At the national level six out of the seven RCUK funders are encouraging open access to research outputs by means of mandates (for details see JULIET).¹ Perhaps encouraged by such mandates, researchers are now contributing at least some of their output to institutional repositories, national funders repositories (notably UK PubMed Central), and subject repositories (such as arXiv). Intermediaries are also populating repositories on behalf of authors.

One of the motivating drivers for establishing repositories has been to support open access in order to disseminate research outputs more effectively. The open access agenda is influencing publishers who are to a limited extent accommodating the ambition to make research outputs open access, while attempting to cover their costs, for example by payments from Wellcome’s Value in People² awards. Open access journals are establishing a business model that is beginning to be absorbed by the more innovative publishers, for example the recent acquisition of the Biomed Central Group by Springer.

It seems likely that repository activity has introduced a level of organisational and cultural change within implementing institutions, and that technology capacity has grown with the spread of skill sets among institutional repository staff. IRs were used to support the 2008 RAE, and several institutions are establishing repositories motivated by the possibility of metric based indicators within the forthcoming Research Excellence Framework (REF) assessment.

Wider interest in the strategic importance of research data management has emerged partly driven by the DCC. There has been a growing interest both in the curation and preservation of data. Initiatives are underway to provide a UK Research Data Service.³ Within HE institutions, where research is increasingly data-centric, there is growing awareness of the need to preserve and re-use data.

Though various statistics are available from both ROAR and OpenDOAR, the figures to measure this progress are less than precise. Whilst these statistical services show that approximately 97 UK HE institutions now have active repositories (whilst 72 do not) there are gaps in the information available about content, in particular how many full text items are accessible. Recent estimates from repository managers suggest that their IRs contain at least 35K full text items available on open access, but these figures are not very useful even as an estimate as they are based on responses from only 14 institutional repository managers.  The figures result from a one-off request made by a JISC Programme Manager to IR managers in mid-2008.

Also, there is little information available about level of deposit and usage of repository content. Detailed analysis of trends by type of content being deposited and level of usage would be welcome. Similarly more understanding of who is depositing content, their motivation and behaviour patterns, would assist fruitful approaches to encouraging deposit.

At present there is no indication as to the overall target for IR content by which success criteria could be set. In particular, the lack of information regarding learning material was highlighted during the consultation exercise. It may be possible to estimate publication numbers for scholarly works in the UK (for example using existing figures from Evidence Ltd derived from Thomson Scientific data).⁴ If so these could be used to set targets for deposit of content. As more attention is given to opening access to research data then some baseline analysis of targets for data content, by type and size, would be useful. Whilst simple totals of the number of items in repositories could be misleading, particularly when legacy content is being gathered, some measures need to be in place to understand the patterns of growth in repository content e.g. by content type, whether content includes full text, etc. JISC may want to consider working with the community to set targets for a certain percentage of content to be made available. This could serve as a focus for funding, advocacy and support. JISC would be unable to set such targets unilaterally and would need to design a process for producing targets in collaboration with the HE community.

To justify the effort involved, any performance metrics would need to provide value to the institution as well as to funders. Metrics need to be tied to the benefits provided by repositories, and therefore should be associated with the objectives that JISC funding and institutional investment is aiming to achieve. As well as metrics for open access material, value needs to be placed on collaboration within subject groups or institutions (sometimes in closed ways). In addition the benefits of content being well managed needs to be factored into any metrics.

As well as statistical metrics, JISC should consider softer evidence such as testimony from senior managers and researchers as to how good content management has improved their working lives.

Recommendation 1: Chart progress of the implementation and usage of repositories by measurable indicators, for example

• Measure cultural and organisational change by whether relevant repository related deliverables are included in institutional information strategies (in particular open access, integration of institutional systems, digital preservation and curation, well managed digital collections).
• Produce baseline analysis of current repository content and analysis of potential repository content.
• Enhance repository statistics to measure the availability of open access full text items (e.g. using ROAR ⁵ or OpenDOAR ⁶ or Intute).⁷
• Analyse patterns of deposit in those institutions with an institutional mandate compared to other institutions. Who is self-depositing and why?
• Scope metrics, qualitative as well as quantitative, to demonstrate the value of repositories to institutions and their members.

1. Sherpa JULIET http://www.sherpa.ac.uk/juliet/ []
2. Wellcome Trust open access funding http://www.wellcome.ac.uk/About-us/Policy/Spotlight-issues/Open-access/Guides-and-FAQ/WTX036803.htm []
3. UKRDS. The UK research data service feasibility study. July 2008. http://www.ukrds.ac.uk/ []
4. PSA target metrics for the UK research base. Evidence Ltd. for Department of Trade and Industry. Office of Science and Innovation, March 2007. http://www.berr.gov.uk/files/file38817.pdf []
5. Registry of Open Access Repositries: http://roar.eprints.org/ []
6. The directory of open access repositories: http://www.opendoar.org/ []
7. Intute Repository Search: http://www.intute.ac.uk/irs []