Te tīmatanga whakawhiti matihiko - Ngā āhuatanga

Digital transfer initiation eligibility

We have developed methods for processing digital information and records into the Government Digital Archive and making them accessible through our Collections search tool. However, current system limitations mean that digital information and records need to have certain characteristics in order to be transferred successfully.

If a public sector organisation considers that they have a set of digital information and records that meet the transfer characteristics listed below, please contact us. We will work with organisations on a case by case basis for digital transfers as our tools and methods continue to evolve.

The aim is to complete transfers, however, after initial investigation it may turn out that a transfer is not possible. Each transfer is a collaborative effort which both the transferring organisation and we will benefit from by gaining experience and knowledge testing respective processes and systems.

The purpose of this document is to give organisations a starting point to determine whether a digital transfer might be feasible. It is intended to assist organisations in initiating discussions with us about a potential digital transfer. The next step will be a comprehensive information gathering exercise which will assist us to assess the digital transfer readiness of the organisation.

4.1 Disposal

If a set of files has the following characteristics, then it may be a possible candidate for digital transfer:

• all files have a disposal action of ‘Transfer to Archives New Zealand’

• all files are considered to have been accurately sentenced and ideally will have met their minimum retention period

• all files are the ‘authoritative’ version.

It is important that only files that have been sentenced as ’Transfer to Archives New Zealand’ are included in the transfer. This means that transferring organisations need to have a current disposal authority. Depending on the organisation’s systems and knowledge of their digital files, sufficient resources should be made available in order to sentence the files accurately against this disposal authority. Ideally files will have met their minimum retention period, but we are open to discussing early transfer.

There may be some duplication in the files. Transferring organisations are encouraged to assess the risk of this and dispose of duplicates prior to transfer. Duplication can also occur if the files were created at a time when an organisation’s information management policy was ‘print-to-file’. This can create considerable work for organisations in identifying what constitutes the ‘authoritative’ version, ie, the digital or paper file.

It is also important for organisations to be aware of their post transfer responsibilities. Remaining in-house copies of the transferred digital information and records need to be destroyed once the transfer has been fully processed and confirmed by us. Transferred digital public archives must not be downloaded from Collections search and resaved back in systems as this will create more copies of the transferred files. The long term effect of this is that duplicate digital versions of documents may be transferred to us at a later date.

If files are still required for business use by the organisation this means they are not yet ready for transfer. Organisations should seek advice from us on deferral of transfer if records have reached their recommended transfer date but are still required.

4.2 Access

Our preference is that all files should be identified with a public access status of ‘Open’ on transfer, so that the public can access them directly online through Collections search. Restricted records may be considered on a case-by-case basis.

Currently, potential digital transfers should contain only files classified under the Public Records Act 2005 as ’Open‘ access on transfer. Transferring organisations are encouraged to do a thorough sensitivity review in advance - see Section 43 of the Public Records Act 2005. This is not the same as organisations having an internal ‘open-by-default’ policy for staff access to files. Once fully processed by us, files classified as ‘Open’ will be viewable immediately by members of the public online. Focusing on ’Open‘ access files makes it easier for us to share any lessons and experiences from a particular digital transfer with other organisations, and organisations can share these with their stakeholders.

Files that are known or believed to have password protections on them must be identified ahead of transfer.

4.3 Size

We have accessioned transfers in excess of 20,000 files to date. From experience it has been found that the complexity of technical analysis required by both parties is not dependent on the number of files in the transfer. There can be a high number of technical issues found in a small transfer, or only a few issues found in a transfer of thousands of consistent files. The quantity of issues and possible solutions will influence any decision on whether to process the transfer at that time or not.

4.4 Metadata and file formats

A file manifest or list that includes file paths and checksums for each file must be produced.

A file manifest is a list containing metadata for a group of accompanying files that are part of a set or coherent unit. We have set no fixed requirements on the format or structure of the metadata. We do have requirements for coding and file naming of the files, though – see 18-F1-Digital Transfer Initiation – readiness assessment guidelines. However, it is important that the metadata is structured consistently and that the transferring organisation has the expertise to understand the metadata and can assist us to understand it.

It is important for us to be made aware of any specialist file formats, or any deliberate format modification that may have taken place: for example, if Microsoft Word documents have been migrated to Adobe PDFs. However, there are no restrictions on the file formats that we will accept. As a minimum, we do prefer to have the original format.

The context or relationships of files and items (ie, folder structures or other contextual metadata) can be represented in Collections search. However, current system limitations mean that record-to-record or item-to-item relationships for example, linked spreadsheets and embedded objects cannot be represented. Therefore, files with relationships that need to be maintained after transfer are not suitable for transfer at this time.