Sunday 16 May 2010

Visualising Repository Contents

Repositories are great for acquiring material, providing access to that material and securing it for the future. There are also other systems that purport to deal with scientific and scholarly communication of one sort or another: journals web sites, Google, email, blogs, twitter, social networks.

Scholarly communication operates on a grand scale: we have monthly and quarterly issues of journals, annual cycles of conferences, three-year cycles of project management, decade-long cycles of employment and forty-year long careers. The published literature is vast, and growing at an alarming rate. Our informal electronic communications are even more prolific - with hundreds of tweets, blogs and emails to keep up with each day.

The problem with all of the systems that supposedly support scholarly communication is that they help us keep up with approximately 1 hour's worth of recent material - a couple of dozen emails or tweets, a page of search results, a single screen of professional social network commentary. What about making sense of that material in the context of the other 99.999% of the last decade's scientific discussion in that area? What we need is really good ways of visualising really (really) large amounts of material, and exploring it in real time, so that we can make sense of and contribute effectively to current discussions. Especially when we come to topics that are just outside our areas of expertise.

I have been really excited to discover Microsoft Labs' Pivot project (see, a system for interacting with huge amounts of visual data. You can see an example of using Pivot to view the contents of an EPrint repository below.

This example only demonstrates using Pivot with the outputs of a part of a single repository, so it's not exactly showing off "the grand scale of scholarly communication". But it is a compelling example of how our respositories might be able to show us the wood and the trees of scientific endeavour, both at the same time.

(This example was constructed by Jiadi Yao, the EPrints-sponsored postgraduate student from the Web Science Doctoral Training Centre. The rest of his time is spent in investigating the ways that social networks underpin citation networks.)