DateRanger – a new tool to share

This is a post by Nat Buckley, of Buckley Williams, our recent collaborators on the new Postal Museum Touch Table project.

As humans we find it easy to quickly read a variety of date formats and almost instantly understand what they represent. We can glance at dates written as “23rd Jan 1894”, “Jan-Mar 1856”, “c. 1960” or “1945-1949” and we know how big the date range is and how accurate it might be. These are just a few examples of dates formats we found in the Postal Museum collections.

To work with those dates effectively in our software we needed to find a way to parse them and represent them in a single format. There are many software libraries designed for translating between standardised date formats (eg. ones used by different countries), but parsing formats commonly used in archives is a slightly less popular problem to solve. Archives tend not to have an enforced, fixed way of writing down dates, so there can be a surprising variety of notations. This isn’t a bad thing — it gives the archivists the flexibility they need to represent their knowledge about the objects under their care. Each collection might have its own quirks.

Some smaller software libraries do take on a challenge of parsing dates from more natural, human-readable formats, but we decided to devise our own way. We had a very specific set of formats and couldn’t find an existing solution that could deal with all of them easily.

I wrote DateRanger, a Ruby library which takes in those formats and translates them into a data structure which represents the start and end of the date range. It makes it straightforward to understand the accuracy of the date — the wider the range, the less accurate or specific the date was to begin with. We’d love to see contributions from anyone interested in expanding how many formats DateRanger can work with.

I used automated tests to build up the code in stages, starting from parsing really simple dates, and culminating at testing even obscure format combinations that we didn’t quite encounter in our data sample. The tests-first approach meant I managed to notice and catch some pretty confusing bugs really early on.

We used DateRanger on the Postal Museum touch table, to help us determine where on the timeline to place the collection records. We did, however, use the original date formats from the archive to display to the viewer. After all, those are already perfectly human-readable.

Advertisements

It’s Made of People

Today I’m hosting a panel at the British Library Labs Symposium. I’ve called it ‘The Ups and Downs of Open’ and invited Paul Downey, Mia Ridge and Jenn Phillips-Bacher to be my panellists. I worked with Jenn and her team on our recent What’s In The Library project, and we swapped wrap-up blog posts (which is an emerging trend I’m enjoying).

This post is from Jenn, Web Manager at the Wellcome Library, and was codenamed “the aftermath”.

---

While most (normal) people spend the summer looking for excuses to get out of the office, I invited Good, Form & Spectacle to work with Wellcome Library on the month-long project that became What’s In The Library? Concentrating our work in July was both a blessing and a curse: a blessing, because we had very little meddling of the type that plagues web projects – but also curse, because as my excitement grew about the potential of Good Things To Come, the colleagues I really wanted to share with were sunning themselves on distant shores.

The team produced scads of working prototypes over four weeks. George summed it up best. It was so worth doing – creative, experimental and loads of fun. But now that the dust has settled and the team has dispersed, I’ve had a bit of time to think about how this project will push conversation and (hopefully) change at the Wellcome Library.

Four big themes stick in my mind. Pull up a chair.

1. It’s true what they say. Silos suck.

Wellcome is like any other massive institution that accretes collections data, systems, and people. We are working with four separate silos of collections data [we probably have more, she sobs] – MARC21 and local metadata in the library catalogue; archival collections described in ISAD(g) in our archives catalogue; keyword descriptions of images in our picture library system; and all of the lovely, full-fat full-text from OCR plus all other data related to our digitisation programme.

But when I look at these data silos, I’m not seeing the problems related to cleanup or systems maintenance – I’m seeing the messy humanity behind it all. Someday in the future when we welcome our fans with a nicer-to-use interface, it won’t be made by magic and clever data manipulation. It’ll be a result of all of the human effort expended on changing the way metadata is collected, created and stored in the first place.

Silos suck because they have to be broken down in multiples. It’s rarely just one; sometimes it’s four. Or more. Time to beef up those influencing skills!

2. It’s hard for us to give stuff away even though we want to.

So here we are, sitting on a goldmine of repurposable metadata, an institutional imperative for openly accessible and free content, and surprisingly low usage. For an organisation whose remit is to GIVE STUFF AWAY, we’re not quite set up to be so magnanimous. Why is it so hard for a digital library like ours to be open?

The glib answer would refer back to Theme 1: Silos. I might also call out black box library management systems, which allow us to get data out, but make it difficult to see it until then, and good luck using it to clean up the metadata en masse.

My more sincere answer is that we like a bit of polish, uncomfortable with being a bit rough around the edges. Let’s keep it all on the down-low until our website and search tools are perfect, yeah? I’m really keen to take our imperfections out to our audiences and find out what they think. Is there utility and charm in what we have? Can we think about ‘open content’ meaning more than just free to use? What about making it open to corrections, feedback and conversation, too?

We’re also very proud of our role as collection stewards. Some (myself, for example), might say we take it a bit too far, as evidenced by this gem I stumbled upon in our Scale prototype

Catalogue record for a piece of Wellcome ephemera, a bookmark for the Research Library. Will we digitize it? Only time will tell.

But what do we lose when we let people do whatever they want with the digital content? I’m inclined to say “not much”, but when trying to open our content for widest possible use, we often find ourselves having to negotiate the anxieties around perceived loss of ownership.

3. Showing the Thing is essential.

Week Two’s work was where we Showed the Thing. At the time I was thinking merely of showing the object, whether it’s a single image, a 280-page book or data – what can be shown digitally to represent our collections. But it ended up being more than just Object-as-Thing. We don’t normally have daily access to an multi-disciplinary development team, so it was really exciting to develop Way-of-Working-as-Thing. With new stuff being built every day, it felt so dang liberating to see things actually working. My colleagues could see what I’d been yammering on about, in real life, no abstractions or even wireframes. Working code. Yay!

The other thing we showed was progress. G,F&S blogged prolifically about the process, tracing how and why decisions were made, and asking questions of us. It’s hugely rewarding to see the project documented this way. We’re not just showing you the Thing. We’re showing the workings behind the thing. Awesome stuff; we need to do more of this.

4. Metadata is our first stab at web content. Let’s edit it.

With over 962K MARC records in our bibliographic database representing our whole collection (with some added subscription content), there’s no way on earth we’d ever be able to create interesting, contextual, editorial content for everything.

But we have a bit of content already, by virtue of it having been described in a catalogue record. What happens if we start thinking of descriptive metadata as editorial content? Are there improvements the editorial team can contribute to the catalogue records, to make the data friendlier and easier to read on screen? Can we get cataloguers to think of themselves as editors or writers who are already have expertise in creating structured content? (BTW, very interested to hear from anyone who’s worked with cataloguing teams this way).

…
So what started out as a ‘digital’ project, in the end, became – and will continue to be – a people project. Go figure. I have at least 23 more ideas to explore with Wellcome Library teams, all of which can be prototyped and maybe even developed for real. And the faster we can do it, the better.

*not an exaggeration.