Digital Preservation Seeds

by Barbara Sierman

Self-assesment tools in DP

The Score Model for self-assesment

In the Dutch Digital Heritage Network we are working with a set of tools, usable by all Dutch organizations with a preservation task. One of those tools is the Score Model. The Score model is an easy self assesment tool, that was developed by the Dutch organisation DEN in cooperation with the Flemish organization Packed a few years ago. The idea is that this Score Model can be used to get an impression of the status of digital preservation in an organization. By identifying the weak and strong areas, the organization will be better equipped to plan a roadmap for improvements.

At the basis of it is a survey. The organization will answer 52 questions, related to seven topics. The questions are characterized by their importance for preservation by a weighing factor. The user is guided in preservation topics by the explanations by each question (the context) and typically related actions to achieve the “good practice”. At the end of the survey the user can print a report. In this report a spider diagram is shown, based on the answers and including the weighing factor. To stimulate improvements in the weak areas, suggestions about action to undertake are given. The Score Model is available in 2 languages, both Dutch and English.

The DPC-RAM for Self-assesment

Recently the DPC published their Rapid Assesment Model, created with a similar goal: to support organisations, who want to do a self-assessment and to improve themselves in digital preservation. The model is based on the concept of an assessment model by Adrian Brown, as described in his book Practical Digital Preservation: a how-to guide for organizations of any size, Facet Publishing, London 2013

In this model, the organization will fill in a Worksheet, describing and giving evidence of the current status and giving themselves a score, ranking from “Minimal Awareness” to “Optimized”. There are 11 topics that are taken into account. For all of them the organization also set their ambition to improve themselves in achieving a higher score. The results are visualized in a spider diagram, showing clearly the difference between the current situation and the target situation. The results are also given as bar charts. The tool comes with a separate document with background information and suggestions for each topic. Although the tool is freely available, there are some bonuses for DPC members.

Comparing the models

When comparing the topics in both models, they are mostly overlapping each other, with some exceptions, like for example the DPC-RAM “Continuous improvement” topic, that is not in the Score model. Also the DPC-RAM distinguishes between Organizational Capabilities and Service Capabilities.

The intended audience of the Score model are “small and medium sized organizations, managing a digital collection […] for a long time”. The DPC-RAM model “can be used by any organisation with a need to preserve digital information for the long term”.

Score Model DPC-RAM
  Organizational Capabilities
Mandate and policy Organizational viability
Preservation Strategy Policy and strategy
Knowledge and Organization Legal basis
Technical Infrastructure IT Capability
Ingest Continuous improvement
Planning and Quality control Community
Access Service capabilities
  Acquisition, transfer and ingest
  Bitstream Preservation
  Content Preservation
  Metadata and Metadata management
  Discovery and Access

Conclusion

For the sake of simplicity, it would have been nice if there was one tool to do a basic assesment. The more, while the Score Model and the DPC RAM have an overlap in the topics they address. But the fact is now, that there are two tools. Both support the organization to have a critical look at their own environment, both cover similar topics and give supporting information for improvement. And both come with a result that is visualized in a spider diagram (the DPC RAM with an additional bar chart with a score in numbers and I know that people like that particularly).

The Dutch and Flemish people having the advantage that it is also in their own language (apart from the English version) and that it is incorporated in the Dutch Digital Heritage Tool set. The DPC members having the advantage of the bonuses of the DPC membership. And all the other DP organizations will have a choice!

3 Comments

  1. Robert Gillesse November 6, 2019

    And one more difference: the DPC model has the advantage of the possibility of giving comments on the background of the answers. This is a helpful feature as answers aren’t always so straightforward (see my earlier comment) and will offer some necessary context to the answers. The current Score model has no feature like this and it is high on the list of one of the improvements.

  2. Robert Gillesse November 6, 2019

    As co-creator of the Score Model I may be a bit biased but I see value in both models. To add a few things to the comparison that Barbara has made:
    As to what will to take the most time and effort filling in:
    The DPC-MM model has (only) 12 question against the 56 of Score Model. But behind the DPC models there is a whole range of meanings behind the answers of minimal awareness, awareness, basic, managed, and optimized: https://dpconline.org/docs/miscellaneous/our-work/dpc-ram/2006-dpc-ram-v-1-0/file. So that makes up for 12 x 5 possibilities = 60 items to choose from. So really there is not much difference there as the Scoremodel has only yes or no answers for each question.

    Coming to the yes and no answers of the Scoremodel: these can be seen as simplistic as answers will mostly be somewhere in between a binary yes/no. So there is probably room for improvement there. But as I always explain to current users: as long as you fill in the model in a consistent way (for instance: when in doubt/or not finished the answer is no) you are mostly ok.

    A more fundamental difference is that the Score Model more or less follows the clusters of ISO 16363 and the DPC model does not. As to richness of content I think the models are more or less the same. So the question is really how important these clusters are for the end users.

    Another difference is that the Scoremodel works with risk levels (low, average, high) thereby indicating that some issues are more important than others. The different risk levels have a different wight in the end score and in the end report the high risk questions are seen as things the organisation needs to solve first. In this way the organisation can use the end report as a practical action plan. This is maybe an advantage of the score model. Of course these risk levels can be seen as subjective as they might differ between organisations and the context in which they operate. I have no easy answer for that one other then that a model like this gives an indication and is not all knowing and needs to be used with care and common sense. I guess this – looking at scores, questions and answers – also is the case for the DPC model. Every self evaluation model is by its nature a simplification of a more nitty gritty reality….

    Looking at the form the Score Model is a – hopefully – easy to web application in which the user has to create an account (but it can also be used anonymously) and where you can go back to earlier reports. The DPC Model uses a spreadsheet which is less fancy but is really very easy to use and of course can also be saved in different version through time. So I would say the form of both models is not really an important difference.

    Another advantage of the Scoremodel is I think the the way the scores are connected to the Core Trust Seal certification (https://www.coretrustseal.org/). By filling in the Score Model the user gets an indication in the end report how far the organisation is evolved in the direction of being ready for a CTS certification. As CTS seems to be gaining in importance in the cultural heritage sector (at least in Netherlands) this feature is helpful I guess.

    • B Sierman November 6, 2019 — Post Author

      Thanks Robert, this is very helpful. At least people have a choice!

Leave a Reply

© 2019 Barbara Sierman

Theme by Anders Norén adapted by Bob Koeman