Digital transfer preparation analysis
At this stage of the digital transfer process, we’ll check your digital records to identify any issues that may affect their transfer into the Government Digital Archive.
We need to analyse your digital records before transfer
When we receive a test sample or extract of your digital information and records (digital records) and their transfer metadata file (TMF) for potential transfer, we’ll run several analytical processes to identify any issues that may affect ingest or transfer into the Government Digital Archive.
Read more about how to prepare an extract
We’ll also do this to analyse all your eligible digital records when we agree with you to proceed with a full transfer.
Useful links
Integrity check
We use the checksums you supply to ensure the digital records are not altered or corrupted from the time they’re exported or copied from your systems until we receive them. We do this by generating new checksums and comparing them to those in the TMF.
If they match, this reassures us — and you — that:
nothing has changed in the copying and transit processes
the integrity of the records has not been compromised.
Useful links
Content analysis
We use a number of automated tools like DROID (Digital Record Object Identification), SQLint or Demystify to assess the accuracy of the sentencing or selection of eligible digital records against your organisation’s disposal authority and the General Disposal Authorities, GDA6 and GDA7.
These tools can also be used to check for:
duplicate records
obvious sensitive, non-business related or draft material
missing records
empty folders
layers of content, such as embedded objects like images or videos, that you may not have picked up during the readiness assessment step.
DROID
DROID is a file format identification freeware created by The National Archives in the United Kingdom. You can download it from their website.
SQLint
SQLint is a simple command-line linter which reads SQL files and reports any syntax errors or warnings it finds. A ‘linter’ or ‘lint’ refers to tools that analyse source code to flag:
programming errors
bugs
stylistic errors
suspicious constructs.
Demystify
Demystify is a way to analyse DROID CSV and Seigfried export files. Demystify:
breaks the export into its components and stores them within a set of tables in an SQLite database
creates additional columns to augment the output where useful
queries the SQLite database, outputting results in a readable form useful for analysis.
Demystify provides an easily readable overview and statistics of the digital records in a transfer.
Useful links
Digital transfer preparation readiness
Technical analysis
We run file format identification and validation tools over the extract. This allows us to do a closer manual analysis of any accessibility and preservation issues which need addressing, such as:
obsolete file formats
unknown file formats
broken files
invalid file formats, and so on.
Other checks are done to identify duplicate records and system files which might need to be excluded from the transfer after discussing with you.
Metadata analysis
We’ll manually map the metadata fields in the TMF to:
the descriptive metadata fields in Collections search (which provides access to the digital public archives)
the technical metadata fields in Rosetta (which stores and preserves the Government Digital Archive).
We’ll check and confirm this manual mapping with you before importing the extract into the test environments of Collections search and Rosetta as appropriate.
Useful links
Analysis results
We’ll consolidate our analysis results in an extract analysis report or ‘warrant of fitness’ for discussion with you. This report assesses the:
transfer readiness of your digital records (the quality and consistency of sentencing decisions)
current digital health of your digital records (the identification of unique file formats and potential digital preservation issues).
The extract analysis report concludes with a recommendation for you to either:
undertake more transfer preparation and repeat the eligibility selection and readiness assessment steps
proceed with planning and preparation for a full extract, or
postpone the proposed transfer.
To document this analysis, we’ll use the Extract analysis report template.
Download the Extract analysis report template 23/Fm9 (PDF, 195KB)