• nl
  • en

On trust and risk and, more importantly: Can we measure any of it? – #iPRES2012 (3)

4 oktober 2012 Gepubliceerd door Laat uw gedicht achter

It is much to the credit of René van Horik of the Dutch Research Data Archive DANS that he did not just prepare a one-way presentation about the work the European project APARSEN is doing on trust, audit and certification (which would have been the easy way out), but that he brought together a jury of his peers to discuss the project from different points of view. Such an approach takes a lot more work, but it leads to lively and informative confrontations for the audience – and the readers of this blog. From iPRES2012 – by Inge Angevaare

The panel included, from the left, Seamus Ross, Andreas Rauber, Helen Tibbo, Silvio Salza, and Maurizio Lunghi.

For those new to the APARSEN work on Audit and Certification, you can find a summary (from Screening the Future 2012) here. The EU is building a three-tiered framework for audit and certification of trusted digital repositories in Europe and the APARSEN work on trust feeds into that. The first level is a self-assessment against very broad, general criteria (Data Seal of Approval), then comes a self-assessment against a real ISO standard, ISO 16363, and the highest level is a third-party audit against ISO 16363. The work of APARSEN carries on from the TRAC project by the US Center for Research Libraries.

After René’s introduction, the first speaker from the panel, our host in Toronto Seamus Ross, wasted no time at all to get his point across. “There is no such thing as trustworthiness,” he said. “There is so much involved in running a digital repository, that a) we cannot measure trustworthiness, and b) we cannot maintain it over time.”

Ross: “There is no such thing as trustworthiness.”

So much for a friendly kick-off …

“Instead,” Ross continued, “I believe in a risk management approach. I can measure risks to the security of digital objects – the risk load. And I can measure the risk resilience of an organization. Typically, a small, young organization can adapt easily; an old, large organization, with many more actors in play, will be much slower to react.” Seamus did not mention it, but a good example of such an approach is the Drambora toolkit.

Helen Tibbo, of the U. of North Carolina School of Library and Information Sciences, agreed with much that Seamus Ross said, but, she added, “that is not the whole story.”

Tibbo: “In a restaurant you can measure objectively whether food is being kept at the right temperature.”

Her main argument was with the metrics involved in ISO16363. “For instance, ISO 16363 prescribes that you have to have a mission. But it says nothing about the quality of that mission” (again, see previous post). Helen Tibbo also questioned the role of the auditors. “If auditing becomes a career, what will happen to objectivity?” Tibbo’s comments come from a “trusted” source, because she was part of the team that carried out test audits for the APARSEN project.

The panel went on to discuss research data quality and authenticity. Silvio Salza presented the recommendations of the APARSEN project that, really, the entire process of data creation and all subsequent actions which could possibly harm the data should be monitored. Helen Tibbo was quick to react to that: “Yes, that would be an ideal world. But it is not going to happen. Nobody has the money and staff to do that.” From the audience Bjarne Anderson (Denmark) agreed with Helen. “Counting the rows and the columns is about the best we can do.”

Session Chair Peter Doorn (Director of DANS) harvesting the panel’s comments

Which prompted session chair Peter Doorn (Director of DANS) to conclude: “The world is messy, the digital world is messier, and the research world is the messiest,” – no doubt thinking of a recent scandal in the Netherlands whereby a psychologist faked all of his survey data and got away with that for years before being caught out.

At the end of the session, Seamus Ross concluded that it had been “fun”, with those different points of view. All agreed, however, that the goal of audit and certification must be to make digital archives perform better. And that, somehow, sometime, we must come up with more objective metrics to benchmark digital archives, whether those be trust-based or risk-based. But preferably automated, embedded in the systems, because if we have to do all of this manually, we will never find enough funding.



Gecategoriseerd in :Geen categorie

Geef een reactie

Het e-mailadres wordt niet gepubliceerd. Vereiste velden zijn gemarkeerd met *

Translate »