Preservation Watch: suggested change in OAIS review

preservation-watchAs a late Christmas present I submitted a change for the OAIS review  by suggesting to introduce the term Preservation Watch into the standard as part of the Preservation Planning module.

During the European project Planets (2007-2011) where we investigated the Preservation Planning functional entity, we had a feeling that the monitoring functionality in OAIS should be extended to other areas. In the standard the monitoring is very much focused on monitoring the Designated Community (of course, these are the people that will use your archive) and the Technology changes. Well it is generally accepted that the rapid changes in technology are seen as a threat to our collected digital files. But there are more threats, that should be monitored systematically and so need to be part of a Preservation Planning function. Changes in your organisation (budget cuts, staff cuts, a merge with another organisation), changes in the environment you’re operating in: political changes for example. The election of Donald Trump was one of the reasons Brewster Kahle of Internet Archive decided to have an extra copy of their archive outside the US.

After the Planets project was finished, the concept of Preservation Watch is taken up by the digital preservation community, for example in the SCAPE project and in various papers and presentations. This justifies for me the concept to be introduced into the OAIS standard. But it will be up to the community to better define this concept and to describe for example the relationship with risk management. This could be done in the next 5 years, so that after a soft introduction in 2017,  a more profound description of the concept of Preservation Watch can be part of OAIS 2022. Let’s wait and see what the CCSDS people think of it.

OAIS: a cage or a guide?

Last week I gave a presentation at the Pericles conference Acting on Change: New Approaches and Future Practices in LTDP in London. This is what I told during the panel about OAIS.

OAIS as a cage?

Is the OAIS standard a cage, with the preservation archive inside as a captivated bird? With clipped wings, unable to fly away, but kept inside by the functional model, the data model and metrics in OAIS and the related standards like the audit and certification standard?oais-cage-klein

 

Lees verder

Experts bediscussiëren OAIS

Veertien Nederlandse en Vlaamse experts bespraken op uitnodiging van de NCDD hun dilemma’s bij de vertaling naar de praktijk van dé standaard in digitale duurzaamheid: OAIS (ISO 14721). Ze deelden een breed scala aan visies op OAIS. Is OAIS een bijbeltekst? Een magische tempel der waarheid? Een kompas om op te varen? Een donkere dreigende wolk of een wolk met af en toe een verkwikkend buitje? Een venster op je organisatie? Op de buitenwereld? Een vliegtuig, de machinekamer van een schip?

Vertaling naar de praktijk

OAIS is al ruim 15 jaar de internationale standaard die we gebruiken als we het hebben over digitale duurzaamheid. De gemeenschappelijke taal helpt ons bij het communiceren over complexe problemen. OAIS is de beschrijving van een conceptueel model voor digitale duurzaamheid, geen reeks van voorschriften. Je moet het model dus naar je eigen omgeving vertalen. Hoe weet je of je de standaard goed interpreteert? Als de groep van experts het ergens over eens was, dan was het wel de behoefte aan praktijkvoorbeelden. In het Engels is daar een begin mee gemaakt via een wiki OAIS community. Deze NCDD-bijeenkomst zou wel eens de opmaat kunnen zijn voor een Nederlandse variant [daar wordt aan gewerkt].

img_0445

OAIS in aluminiumfolie

De experts hadden vooraf hun visie op OAIS gegeven. Wat betekent OAIS voor hen? Dit werd met behulp van aluminiumfolie door iedereen in 2 minuten in een kunstige sculptuur samengevat. Men was het er vrijwel unaniem over eens dat de standaard een goede richtlijn was, goed in elkaar zit en zeker als leidraad bij de inrichting van zowel je organisatie als je systeem gebruikt kan worden. Kritiek komt vaak voort uit onkunde, omdat men stokt bij de beschrijving van het functionele model, of de teksten te letterlijk wil interpreteren. En niet verder leest in de standaard, terwijl het bijbehorende datamodel ook van belang is voor het begrip van de standaard. Er was een gedeeld pleidooi om meer aandacht te geven aan een klein, maar uitermate belangrijk stukje tekst in OAIS: de verantwoordelijkheden van het digitale archief (paragraaf 3.1. voor de kenners).

Voorbeelden gevraagd

Naast hun visie hadden de expert ook hun dilemma’s opgeschreven. Hier kwam vooral naar voren dat men vaak met de praktische vertaling worstelt. Omdat OAIS niet voorschrijft hoe het moet, kan er verschil van interpretatie zijn. Meer voorbeelden van collega’s hoe ze bepaalde richtlijnen naar de (vaak weerbarstige) praktijk vertalen, zou de digitale duurzaamheidscommunity enorm kunnen helpen.

AIP, Pre-Ingest, Designated Community en Access

Is het bijvoorbeeld zo dat een Archival Information Package dat is opgeslagen altijd ongewijzigd moet blijven, of mag men in de loop van de geschiedenis metadata toevoegen en wijzigen? Zou er niet een Pre-Ingest-functie moeten worden toegevoegd aan het OAIS model, omdat alle organisaties worstelen met het verschil tussen wat ze rechtstreeks aangeleverd krijgen en de tussenstappen die nodig zijn voordat het een Submission Information Package is (en dus weer past in het model)? En kunnen we hier dan een onderscheid maken tussen het moment waarop de organisatie de digitale objecten onder zijn hoede genomen heeft en het moment waarop de keuze nog gemaakt moet worden of het materiaal wel geaccepteerd wordt (met andere woorden zit “appraisal” in de pre-ingest of niet)? Het begrip Designated Community – feitelijk de mensen voor wie je het materiaal toegankelijk houdt – is redelijk vaag en ook de Access-functie kent in de praktijk veel meer haken en ogen dan nu in de OAIS standaard verwoord zijn. Via een buitengewoon democratische procedure (je schreef niet alleen op waar je wél maar ook waarover je persé niet wilde discussiëren) werden groepen samengesteld om een selectie van de stellingen nader uit te diepen en om te zetten in een scherp geformuleerde aanbeveling. Waarom dat dan?

img_0429

Voorbereiding op de review

Wel, er is op dit moment een review proces gestart voor OAIS en dat is bij uitstek de gelegenheid voor de gebruikers van de OAIS-standaard om verbeteringen en wijzigingen voor te stellen. De standaard is oorspronkelijk in de jaren negentig opgesteld door en voor ruimtevaartdeskundigen. De variatie aan gebruikers is echter sindsdien toegenomen: archieven, bibliotheken, data centers, medici, universitaire repositories en ga zo maar door. Wil de standaard ook voor die gebruikers relevant blijven, dan zal die moeten mee veranderen als de wereld van de gebruikers verandert.

Vandaar dat er in internationaal verband een wiki is ingericht om praktische en theoretische kennis over OAIS te verzamelen. Praktijkvoorbeelden dus. Daarnaast kan deze inbreng, net als de resultaten van de NCDD-workshop, leiden tot aanbevelingen voor de nieuwe versie van de standaard. Iedereen kan deelnemen aan de discussie, zodat het proces transparant en open is. Uiteindelijke verzoeken tot aanpassingen dienen via het officiële kanaal ingediend te worden bij review.oais.info.

Maar voordat het tot een nieuwe versie van de standaard komt, duurt nog even omdat eerst alle officiële processen en inspraakrondes doorlopen moeten zijn, zodat de verwachting voor een nieuwe versie van OAIS nu rond 2020 ligt. Genoeg tijd om te discussiëren en te analyseren waar de knelpunten liggen en samen te werken aan een online kennisbron over OAIS in de praktijk – in het Engels of in het Nederlands (zie de nieuwe informatiepagina over OAIS van de NCDD).

Wordt vervolgd

Meer nieuws volgt binnenkort: de resultaten van deze NCDD-workshop brengen we in bij een OAIS-paneldiscussie op de iPRES Conferentie in Bern op 2 oktober. Wordt vervolgd!

iPRES 2015 in Chapel Hill

Afbeelding1Audit, CD-ROMS, Emulatie, Ingest, OAIS en Web, dat waren in alfabetische volgorde de meest besproken onderwerpen tijdens de jaarlijkse conferentie iPRES 2016, die vorige week plaatsvond in Chapel Hill, North Carolina. Dit is mijn persoonlijke indruk, want natuurlijk kwamen in de lezingen, posters en workshops nog veel meer onderwerpen aan bod. Het is tenslotte een jaarlijkse reünie waarbij iedereen probeert zijn resultaten en toekomstplannen te presenteren. Lees verder

Let’s discuss OAIS!

Today our “gold standard” in preservation will have its own place on the Internet: the OAIS wiki

OAIS-logo-wiki

Since its first publication the OAIS standard has become a crucial guidance in our digital preservation community. It is in our own interest to keep this standard up to date and to monitor its connection with our daily practices. We are now better informed how to preserve our digital collections. But we need to be alert to keep the OAIS standard connected to our daily practices.

A wiki for OAIS
The idea to create a wiki for OAIS was raised last year at the 4C/DPC Conference and was realised in close cooperation by William Kilbride (DPC), Hervé L’Hours (UKDA), Paul Wheatley (DPC) en me (KB).

What are we heading for? A place to discuss OAIS and to share experiences. For everyone working in digital preservation, in our “community of practice”. Despite the advantage of using a shared OAIS terminology, translating the OAIS concepts into daily practice often raises questions. Implementing these concepts can lead to different interpretations of the standard and to confusion. Sharing examples and real life practices can help everyone in their situation. This wiki is intended to become a central place where everyone can start to discuss OAIS aspects. This way we can keep the OAIS standard relevant in our daily work.

ISO 5- year review in 2017
There is another reason to discuss the OAIS standard. In 2017 the ISO process will be started to review the standard, which happens every 5 years. This review will offer us a chance to propose changes to the standard. Changes we think are necessary to keep the standard relevant.

Therefore we put the integral text of the standard on-line with an opportunity to add your commentary and annotations and the possibility to discuss this. Based on this feedback we will draft an official proposal for the review. Apart from that we will investigate what will be the best way to take part in this official review process.

Interested?
Join us and help to keep the OAIS standard relevant. Go to the OAIS wiki, register and contribute your bit!

 

Praat mee over OAIS!

Vandaag krijgt de duurzaamheidsstandaard OAIS ISO 14721) een eigen plek op internet: de OAIS wiki

OAIS_Lgo_CMYK

De OAIS standaard  is in de loop der jaren een cruciale leidraad in onze “digital preservation community” geworden.  Het is in ons belang dat deze standaard blijft aansluiten bij onze dagelijkse praktijk. Inmiddels weten we steeds beter hoe we duurzame toegankelijkheid tot onze digitale collecties moeten organiseren. Sluit de OAIS standaard nog wel aan bij deze ontwikkelingen?

Een wiki voor OAIS
Het idee voor deze wiki ontstond vorig jaar op de 4C/DPC Conference en is verder uitgewerkt door William Kilbride (DPC), Hervé L’Hours (UKDA), Paul Wheatley (DPC) en mijzelf (KB). Wat staat ons voor ogen?
Een plek voor discussie over OAIS en voor het delen van ervaringen. Voor iedereen die bezig is met digitale duurzaamheid in onze “community of practice”. De OAIS standaard mag dan algemeen bekend zijn, de toepassing ervan roept nogal eens vragen op en interpretatieverschillen leiden soms tot verwarring. Het delen van visies kan helpen de praktische invulling te realiseren. Voorbeelden en oplossingen van collega’s kunnen inspirerend werken in de eigen omgeving. Deze wiki wil een centrale plek worden waar iedereen terecht kan om OAIS gerelateerde kwesties te bediscussiëren. Op deze manier kunnen we de OAIS standaard levend houden en blijvend laten aansluiten op onze dagelijkse werkzaamheden.

ISO 5-jaar review in 2017
Er is nog een reden om nu over de relevantie van de OAIS standaard te discussiëren. In 2017 start het proces van de reguliere 5-jaarlijkse ISO review van de OAIS standaard. Dit geeft ons de kans wijzigingen op de standaard voor te stellen. Daarom hebben we de volledige OAIS standaard tekst op de wiki gezet, met een mogelijkheid om commentaar te geven en hierover onderling te discussiëren.
Wij willen deze bijdragen gebruiken om goed beslagen ten ijs te komen voor deze review. Op basis van het commentaar op OAIS, zal een commissie voorstellen voor aanpassingen in de ISO standaard doen. Daarnaast zullen we goed uitzoeken welke officiële kanalen bewandeld moeten worden om deze voorstellen bij ISO in te brengen.

Meedoen
Doe mee en help de OAIS standaard blijvend actueel te houden. Wat let je nog? Ga naar de wiki, registreer je en draag je steentje bij!

Crystal clear digital preservation: a management issue

Digital Preservation of Libraries.final.final.inddRaising awareness for digital preservation was a frequently used phrase when I started in this field ten years ago (never regretted it, hurray!). We preservationists have made progress. But the story is still not explaining itself. So I like reading how others persuade and convince people. Recently I found a book that really does the job. In crystal clear language, without beating about the bush and based on extensive up to date (until 2014) literature, digital preservation is explained and almost every aspect of it is touched upon. Edward M. Corrado and Heather Lea Moulaison have done a great job with their Digital Preservation for Libraries, Archives and Museums , Rowman and Littlefield, 2014. ISBN 978-0-8108-8712-1 (pbk.) — ISBN 978-0-8108-8713-8 (ebook)

In fact, I should start this blog post with “Dear manager, I have found a book that tells you all you need to know about digital preservation. Spare some time and read the chapter that is dedicated to you (part II) , the sooner the better” [preservationist, please forward this to your manager, they might even read the rest of the book!]

The book starts by explaining what digital preservation is not ( like “backup and recovery”, access, “an afterthought”). Followed almost immediately by the (positively phrased) starting point, that guides the whole book:

“ensuring ongoing access to digital content over time requires careful reflection and planning. In terms of technology, digital preservation is possible today. It might be difficult and require extensive, institution-wide planning, but digital preservation is an achievable goal given the proper resources. In short, digital preservation is in many ways primarily a management issue”.

The red line/ metaphor in the book is the authors “Digital Preservation Triad”. The triad is a new variety of the Three legged stool of Nancy McGovern and is symbolized by a Celtic knot. The knot is used in order to better symbolize the interrelated activities.

triad

These activities are divided into :

  • Management-related activities,
  • Technological activities and
  • Content-centred activities.

Each set of activities is further explained in a dedicated chapter. The chapter about Management activities immediately starts to explain the basics of the OAIS model. Clearly showing that this is the essence of digital preservation. Knowledge of OAIS should be present on management level of an organisation. Only then management can deal properly with aspects like human resources (skills and training), and sustainable digital preservation (costs etc).

The Technology part is more concerned with metadata and file formats and the technical infrastructure or repository, which is closely related to mechanisms of trust (audit and certification).

The last part of the book discusses aspects related to the Content, like collection development.

The text is based on a large literature list in which many recently published conference papers, (EU) project results and reports are used. The authors are well informed about what is going on and do not restrict themselves to the US.

What I liked in this book is the very practical approach and the unvarnished description of digital preservation (‘not easy but doable’). The authors stress that preservationists should convince over and over again management “that digital preservation is important to the overall mission of the organization”, and not just “an experimental technology project” and “communicate the multiple ways in which digital preservation brings value to the organization.”

One of the barriers in this process, at least in my experience, it that people often try to connect their experience in analogue preservation with that of digital preservation. Sometimes this leads to monstrous analogies. This book does not try to map the two worlds, but clearly states:

“The digital item created and made accessible as part of a digital preservation system is fundamentally different from an analogue item. Period.”

Unavoidably some recent developments are missing, like the Cost model work that was done in the 4C project and the work on Preservation Planning and Policies in SCAPE.

But if you still need to convince your management, point them to this book – also available as an epub!

The gold standard

This is the text of a presentation I held at the combined 4C/DPC conference ‘Investing in Opportunity: Policy Practice and Planning for a Sustainable Digital Future’ in London 17-18 November 2014 at the Wellcome Trust.

WP_20141118_006

A few months ago I was in Copenhagen and as I like shopping in a foreign country and take something with me to remind me of my trip , I bought a golden ring. Well, not gold, it was gold plated and after a few months of wearing is was a silver ring. Was that a disappointment? The design was still nice and the ring still fitted. But it did no longer match my other rings. And I was not expecting this to happen so soon. And if instead of silver, the next layer would have been brass or nickel or copper, I really would have been disappointed and felt betrayed. So it was gold, turned to silver and yes that did matter.

No one would appreciate a trustworthy digital archive turning from gold to silver…

What are we talking about?

The title of this presentation Is there a gold standard and does this matter? was made by William [Kilbride] when inviting me for this panel. But what we are talking about here of course is the ISO 16363 standard on Audit and Certification of trustworthy digital repositories, officially an ISO standard since 2012. Recently the accompanying standard for accreditation of auditors was officially published by ISO and now certification against this standard can start.

The ISO standard is the highest level in the European framework of certification, starting with the basic level of the Data Seal of Approval, a middle layer of the nestor/DIN standard, and the highest level of this ISO standard. We currently don’t speak about layers like bronze, silver and gold, but if you would do, the ISO is the gold level in this model, the gold standard if you wish.

History

This audit standard has a history of more than 10 years and to really value its importance, we should know a bit of its history. Already in the OAIS model, a reference is made to a systematic approach of checking the conformity of organisations to this model, a standard for accreditation of archives. The first draft of TRAC criteria is based on this OAIS model – our shared view how digital preservation should be done. Ten years ago a draft version of TRAC was tested by performing audits on different repositories, among them the KB, and based on this feedback the Checklist of TRAC was published in 2007. 5 years later, with TRAC as a starting point, the ISO standard was published, which I will call here the TDR-standard. In summary, this standard is based on preservation insights of a wide variety of organisations and people, all knowledgeable in digital preservation and with different background. We sometimes might forget who is behind this standard and I tried to create an overview of the people and organisations that were involved in TRAC 2002, 2005, 2007, ISO 16363 and ISO 16919. A wide variety of organisations, and of people, some already retired, some still going strong and others working daily with digital preservation. The standard was created really with input from the preservation community.

Woordenwolk-personen woordenwolk-instellingen

Is a gold standard also a perfect standard?

That need not to be the case. Although a lot of experience is woven into the standard, this standard is not a static, carved in stone, document. For the TDR standard, I think it is too early to judge. No audits did take place with this standard yet.

Let me give you an example of another “gold” standard we all share: the OAIS model ISO 14721, out there for more than 10 years. The OAIS standard has proven its value, on various areas but especially as the Esperanto of the digital preservation community. However, standards operate in a changing world and that has an influence on the value of that standard. If a standard is not adapted to developments in the changing world – and digital preservation is an evolving area – it will lose its value.

Some people think the OAIS model is outdated and does not keep up with the developments in preserving digital material. Since 2002, when the OAIS was first published, the preservation insights have changed. As an example: Emulation is now an accepted approach, migration less then 10 years ago. Quite often, in practice,  the well know functional model of the OAIS is extended with a function “pre-process”. Sometimes the OAIS standard is interpreted more as a straitjacket then as a conceptual model. This will lead now and then to strong twitter opinions, like suggestions to burn OAIS, never to mention it in a presentation etc. Other people contribute in a more constructive way by blogging about specific shortcomings of OAIS or by further developing the model. Like we did in the Planets project, where Paul Wheatly and I added the Preservation Watch concept, extended in the SCAPE project. Others created a framework to apply OAIS to distributed digital preservation. These contributions will help to keep a standard alive and we, as a preservation community, should try to incorporate new insights into the standard.

That is why ISO standard has a process of reviewing the standard. Every 5 years, theoretically at least, changes can be made to the standard. The first opportunity for the OAIS model (and coincidentally for the TDR standard as well) is 2017.

The ISO process is not quite clear, but there is enough reason for the preservation community to investigate how to be involved in this process and keep the standards up to date. I’ll talk with the Dutch Standard Organisation NEN soon and ask them what is the official procedure and how we can collaboratively act. I’ll report on this at our KB research blog

Back to the TDR standard.

The TDR standard was developed to audit trustworthy digital repositories. A big difference with the OAIS model, where the individual institute decides whether it is compliant or not, is that it is a group of auditors who are making this judgement. As currently everyone can say that his repository is conform OAIS, this will no longer be correct if we speak about the TDR standard. There an external body will make the decision whether your repositoriy is a Trustworthy Digital Repository, according to the “gold” standard and based on the information you gave the auditors. And, not less important, based on the knowledge and experience of the auditors. And this is a crucial factor in the process: who are these auditors? Sometimes the more cynical people tell me that an audit process starts with giving the auditors a good dinner, have a chat and then you’ll get the certificate. Needless to say that this is not a good auditing process. Certainly not one that is covered by the rules and regulations that are described in ISO 16919. This standard describes the qualities an auditor need to have in general, but the standard is specifically extended with qualities related to digital preservation. Despite this, how do we, as a community get to the point that we trust the auditors and that the certificate really reflects a trustworthy repository? Two elements contribute to this: transparency and the time limited validity of the certificate.

One of the measures that were taken is “transparency”: all documentation, except for really confidential material, will be publicly available after the certificate is given. There are already a few examples out there based on TRAC audit by CRL, LOCKSS, Portico, Hathi Trust, Chronopolis and Scholars Portal. This transparency will help to foster discussion in the community about the certification.

The other element is that a certificate, in contrast to the repository itself, is not for “long term” . Instead it is lasting 5 years or so. Then the repository will need to start a new audit process to get the certificate again, based on changed conditions no doubt.

Will a certified repository be a successful preservation environment?

Chances are yes, but no guarantee can be given. As a community we agreed on a approach to digital preservation by accepting the OAIS model and the derived audit standards, described and hopefully regularly discussed and updated in this golden approach so to speak. By the way, the monetary gold standard is not in use any more, but replaced by other mechanisms. Whether our gold standards will lead to success, time will tell.