EPrints Evaluation - assessing the usability

Andy Powell has been looking at repository usability on his blog, and his latest posting uses a paper in the ECS school repository as an example. He makes some very good points to which I ought to respond.

First, Andy comments on the URL of the splash page. 

The jump-off page for the article in the repository is at http://eprints.ecs.soton.ac.uk/14352/, a URL that, while it isn't too bad, could probably be better. How about replacing 'eprints.ecs' by 'research' for example to mitigate against changes in repository content (things other than eprints) and organisational structure (the day Computer Science becomes a separate school).

This is certainly a point to consider, but there are two approaches to unchanging cool URIs. One is to try to make the URI as independent of any implementation or specific service as possible - "research" instead of eprints.ecs. Unfortunately, there are some things that are givens - we are the School of Electronics and Computer Science and ecs.soton.ac.uk is our subdomain. We can't pretend otherwise - and we do not have the leeway to invent a new soton.ac.uk subdomain. We are very aware of the impermanence of the university structure (and hence the URL and domain name structure). Five years ago the whole University was re-arranged from departments into new amalgamated schools. Luckily, we and our URLs survived unscathed. Even worse, last year the University's marketing office almost rebranded every official URL and mail address from soton.ac.uk to southampton.ac.uk!

The ultimate in insulating yourself from organisational changes is to adopt an entirely opaque URI such as a handle. The alternative is to admit that you can't choose a 100% safe name, and have policies and procedures in place to support old names whatever changes and upheavals come to pass. For example, our eprints service itself replaces an older bibliographic database called jerome, whose legacy URIs are redirected to the corresponding pages in eprints. That is the approach that we take with our services - adapt, change, move and rename if necessary, but always provide a continuity of service for already published URIs.

The jump-off page itself is significantly better in usability terms than the one I looked at yesterday. The page <title> is set correctly for a start. Hurrah! Further, the link to the PDF of the paper is near the top of the page and a mouse-over pop-up shows clearly what you are going to get when you follow the link. I've heard people bemoaning the use of pop-ups like this in usability terms in the past but I have to say, in this case, I think it works quite well. On the downside, the link text is just 'PDF' which is less informative than it should be.

The link text is a bit curt: it should say "View/Open/Download PDF document"

Following the abstract a short list of information about the paper is presented. Author names are linked (good) though for some reason keywords are not (bad). I have no idea what a 'Performance indicator' is in this context, even less so the value "EZ~05~05~11". Similarly I don't see what use the ID Code is and I don't know if Last Modified refers to the paper or the information about the paper. On that basis, I would suggest some mouse-over help text to explain these terms to end-users like myself.

Author names are linked to the official school portal pages externally to the repository. Presumably keywords should be linked to a keyword cloud that groups all similarly-themed papers. The ID Code and Performance indicators are internal, and should be low-lighted in some way. The last-modified information refers to the eprint record itself, and so the label should be mae more informative.

The 'Look up in Google Scholar' link fails to deliver any useful results, though I'm not sure if that is a fault on the part of Google Scholar or the repository? In any case, a bit of Ajax that indicated how many results that link was going to return would be nice (note: I have no idea off the top of my head if it is possible to do that or not).

The Google Scholar link failed to work on this item because the author/depositor changed the title of the article in its final version and left the original title in the eprint record. I have revised the record and now the Google Scholar link works properly. (The link initiates a search on the title and first named author.)

Each of the references towards the bottom of the page has a 'SEEK' button next to them (why uppercase?). As with my comments yesterday, this is a button that acts like a link (from my perspective as the end-user) so it is not clear to me why it has been implemented in the way it has (though I'm guessing that it is to do with limitations in the way Paracite (the target of the link) has been implemented. My gut feeling is that there is something unRESTful in the way this is working, though I could be wrong. In any case, it seems to be using an HTTP POST request where a HTTP GET would be more appropriate?

You are right to pull us up on this. ParaCite is a piece of legacy technology that we should probably either revise or remove. The author left the University several years ago. I will check to see what genuine usage it is getting.

There is no shortage of embedded metadata in the page, at least in terms of volume, though it is interesting that <meta name="DC.subject" ... > is provided whereas the far more useful <meta name="keywords" ... > is not.
The page also contains a large number of <link rel="alternate" ... > tags in the page header - matching the wide range of metadata formats available for manual export from the page (are end-users really interested in all this stuff?) - so many in fact, that I question how useful these could possibly be in any real-world machine-to-machine scenario.

We should definitely add a meta entry with the unqualified keyword tag. The large number of exports are for bibliographic databases (BibTeX, EndNote and the like), metadata standards (METS, MODS, DIDL, Dublin Core) and various services (e.g. visualisations or mashups). The problem with that list is that it is undifferentiated and unexplained - it should at least have categories of export functionality to inform users. As for the large number of <<link rel="alternate" ...> tags, I'm not sure I understand the criticism - will they bore the HTML parser?

Overall then, I think this is a pretty good HTML page in usability terms. I don't know how far this is an "out of the box" ePrints.org installation or how much it has been customised but I suggest that it is something that other repository managers could usefully take a look at.

I am happy to have received a "Good, but could do better" assessment for the repository. This is a configured EPrints repository, but not a heavily customised installation, so it shouldn't be too different an experience for other EPrints 3 repositories.

Usability and SEO don't centre around individual pages of course, so the kind of analysis that I've done here needs to be much broader in its reach, considering how the repository functions as a whole site and, ultimately, how the network of institutional repositories and related services (since that seems to be the architectural approach we have settled on) function in usability terms.

I agree with this - repositories are about more than individual files, they are about services over aggregations of documents - searches, browse views of collections. We need critiques of more of a repository's features - perhaps something that SHERPA/DOAR could implement in future?

It's not enough that individual repositories think about the issues, even if some or most make good decisions, because most end-users (i.e. researchers) need to work across multiple repositories (typically globally) and therefore we need the usability of the system as a whole to function correctly. We therefore need to think about these issues as a community.

This is a key point that we need to bear in mind. So is it possible to be more specific about the external players in the Web? What services exactly do we need to play happily with? Various commentators have mentioned different aspects of the Web ecology - Google, Google Scholar, RSS, Web 2.0 services, Zotero etc. Can we bring together a catalogue of the important external players with which a repository must interoperate and against which a repository should be expect to be judged/accredited?