The PDF Version of this document can be downloaded here: The Chief Archivist's Annual Report on the State of Government Recordkeeping 2010
This is the sixth report issued by the Chief Archivist under section 32 of the Public Records Act 2005 (Act) on the state of government recordkeeping.
Since the passing of the Act, Archives New Zealand has worked with public offices to support the development of recordkeeping across government. This report reflects on the progress that has been made since the passing of the Act and examines the current state of recordkeeping in public offices. The report identifies key opportunities for development and makes several recommendations to public offices.
Chief Archivist and General Manager
Archives New Zealand
Department of Internal Affairs
Archives New Zealand and public offices have continued to work together over the past year to improve the state of government recordkeeping. This report provides a summary of the current environment and makes recommendations aimed at ensuring the continual improvement of government recordkeeping and information management in New Zealand.
Public offices have made good progress in developing their recordkeeping practices since the passing of the Public Records Act (Act) in 2005. In the section, Five years of government recordkeeping under the Public Records Act 2005 I reflect on the progress that has been made in areas identified as enablers of good information management:
Archives New Zealand will continue to work with public offices to support continual improvement of records and information management in government.
Analysis of information gathered by Archives New Zealand highlights five key themes in the government recordkeeping environment for 2010:
Based on analysis of these themes I make the following recommendations:
Investment in records and information management helps guarantee that the New Zealand Government continues to have a comprehensive public record well managed both now and in the future.
This report draws on a number of different data sources:
This report is also informed by monitoring the advice and guidance Archives New Zealand has provided to public offices and examining the current recordkeeping environment both within New Zealand and internationally.
Information management has a crucial role in improving the efficiency of government. Effective business decision making relies on timely and accurate information. Well integrated information management reduces the time spent finding information and
supports the reliability and authenticity of the information. Significant administrative and storage costs are reduced by ensuring records are kept only as long as they are required for normal prudent business practice.
Poor information management exposes organisations to a number of risks including:
The Public Records Act 2005 (Act) establishes a framework for supporting, monitoring and improving information management across public offices and local authorities.
Core elements of this framework include:
Since 1957, New Zealand has had legislation to support the management of records of long-term value to government. Over the last decades of the twentieth century the high pace of change in administrative structures, work practices and technology had a
detrimental impact on the state of records kept by government. The 1957 Archives Act did not provide a sufficient framework to ensure all records of government were being consistently created and maintained across government.
In 2005, the Public Records Act (Act) was enacted to support the effective management of all public records and the recordkeeping practices of public offices and local authorities as well as the long-term preservation of public archives. The Act has enabled Archives New Zealand to develop a framework to support the administration of public records and ensure systems and processes are in place for the efficient management of all public records, regardless of format, across government.
As part of the framework Archives New Zealand provides products and services to people working with public records across government.
Key support products and services include:
To ensure Archives New Zealand’s programmes are targeting the most appropriate areas we regularly monitor the state of recordkeeping in government. Since 2006, we have surveyed public offices annually to gather information on recordkeeping within
their organisation. This information is then analysed to determine the overall state of government recordkeeping. The findings of the annual government recordkeeping surveys have formed the basis of the Chief Archivist’s annual report on the state of government recordkeeping tabled in Parliament yearly since the passing of the Act.
In 2010, the audit provisions of the Act came into effect and Archives New Zealand began implementing a programme to audit the recordkeeping practices of public offices. Public offices are being audited on a five-year cycle to validate public offices compliance with recordkeeping standards and with the Act.
The purpose of the audits is to provide assurance that good information management behaviour and practices are in place across government to support accountability and good business practice. The audit programme is based on compliance measured from a capability, risk assessment, and continuous improvement perspective, focusing on achieved outcomes and practices.
The results of the 2010/11 audits will be presented to Parliament in a separate report prepared under section 35 of the Act.
The monitoring and reporting activities of Archives New Zealand provide a comprehensive and complementary view of recordkeeping in government and assist us in supporting the continual improvement of the information management practices of
In future years we will continue to ensure that reporting to Parliament under the various provisions of the Act is efficient and well coordinated to provide a clear and evidence based assessment of recordkeeping in government.
Analysis of the data collected by Archives New Zealand has revealed positive trends in the development of the infrastructure and capability needed to support good recordkeeping practices in public offices. Data indicates public offices are increasingly
aware of the benefits of effective recordkeeping and are making continual progress towards realising these benefits.
In areas identified by Archives New Zealand as enablers of good information management public offices have made good progress. Namely:
In order to achieve the business benefits of good information management, public offices must take a coordinated and organised approach towards managing their information.
In 2005, the Chief Archivist’s Annual Report on the State of Government Recordkeeping recommended that all public offices should have a formal recordkeeping programme. Since the 2005 report, Archives New Zealand has worked alongside public offices to support this recommendation.
It is pleasing to see that in 2010, 93 per cent of public offices either have a formal recordkeeping programme or are working towards implementing a formal recordkeeping programme. Figure 1 below illustrates the significant improvements made in the years following the initial recommendation in 2005.
Question 2: Which of the following statements best describes the type of recordkeeping programme in your organisation? (n=176) from Government Recordkeeping Survey Report 2010: Public Offices – Final
Note: ‘Other’ option was replaced with ‘Working towards implementing a formal programme’ in 2008.
Public Service departments are the most likely sector to have developed a formal recordkeeping programme. Figure 2 below shows that 83 per cent of Public Service departments report they have a formal recordkeeping programme in place. Just two per cent are still operating an informal programme. Archives New Zealand will continue working with other sectors to ensure that comprehensive and appropriate recordkeeping programmes are operating across all of government. We would like to see all public offices performing at the level of Public Service departments.
Question 2: Which of the following statements best describes the type of recordkeeping programme in your organisation? (n=41) From Government Recordkeeping Survey 2010: Public Offices – Final Report
The Act supports the creation of full and accurate records in order to support open and accountable government.
Figure 3 shows that since the passing of the Act there has been a steady increase in the number of public offices developing systems to support compliance with the Act. This increase demonstrates that there is an increasing realisation within public offices of the benefits that having full and accurate recordkeeping brings to both business performance and wider societal outcomes.
Question 1: Does your organisation have policies and procedures in place to assess whether it is complying with the Public Records Act 2005? (n= 176) From Government Recordkeeping Survey 2010:
Public Offices – Final Report
Note: ‘Currently developing processes to assess compliance’ was added as an option in 2008.
Under section 27 of the Act, the Chief Archivist has a mandate to issue standards in relation to the management of public records. The Public Records Act Standards Programme has been in place since 2006. Since the passing of the Act the Chief Archivist has issued four mandatory standards:
These standards present a consistent, whole of government approach to records and information management by setting clear expectations relating to a specific area of recordkeeping. Standardised practices for information management ensure that public records are being consistently and appropriately managed.
Since the first standard became mandatory in 2009, public offices have made good progress in performing risk assessments against individual mandatory standards. In 2010, 43 per cent of public offices reported that they had undertaken a risk assessment against at least one of the mandatory standards.5
Figures 4, 5 and 6 below provide a breakdown on the percentage of public offices that have undertaken risk assessments against a mandatory standard. While these numbers are encouraging I would like to see more public offices assessing their business practices against the mandatory standards. Assessing business practices against the mandatory standards will enable public offices to prioritise resources and effort to ensure areas that pose the biggest recordkeeping risks are addressed appropriately.
4 Standards available at http://archives.govt.nz/advice/continuum-resource-kit
5 Data was gathered prior to the release of the mandatory Disposal Standard.
Question 9a: Please indicate if your organisation has performed a risk assessment against the following mandatory standards issued by Archives New Zealand: Storage Standard (n= 176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Note: Total exceeds 100% because of rounding.
Question 9b: Please indicate if your organisation has performed a risk assessment against the following mandatory standards issued by Archives New Zealand: Create and Maintain Recordkeeping Standard (n= 176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Note: Total exceeds 100% because of rounding.
Question 9c: Please indicate if your organisation has performed a risk assessment against the following mandatory standards issued by Archives New Zealand: Electronic Recordkeeping Metadata Standard (n=176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
A key way to gain business efficiency is to only manage public records for as long as is required. Section 20 of the Act requires public offices to have the Chief Archivist’s authorisation before disposing of public records. Disposal covers a range of activities including destruction, transfer to Archives New Zealand or another public office, alteration, sale or discharge. Centralised disposal authorisation ensures a whole of government approach to the long-term management of the records of the New Zealand Government.
The General Disposal Authorities are issued by the Chief Archivist to authorise the disposal of routine administrative records common across government.6 The General Disposal Authorities apply to a significant number of the records created and maintained by government. In 2006, Archives New Zealand issued a revised version of the General Disposal Authorities. All public offices should be routinely using these all of government authorities to ensure they are managing and disposing of their routine administrative records appropriately.
In addition to the General Disposal Authorities, public offices must gain authorisation from the Chief Archivist to dispose of their core business records. Since 2008, Archives New Zealand has been collecting data on public offices’ intentions to develop disposal authorisation.
Figure 7 (below) shows that since the Act was passed, public offices have made good progress in developing disposal authorisation. In 2010, close to half of all public offices had disposal authorisation for core business records. Another 22 per cent of public offices were developing coverage. While the increase in disposal coverage since the passing of the Act is pleasing, 29 per cent of public offices have not yet made a clear commitment to fulfilling their obligations. Disposal authorisation and implementation is a key foundation of a successful recordkeeping programme.
6 General Disposal Authorities available at http://archives.govt.nz/advice/continuum-resource-kit
Question 11: Does your organisation have a disposal authority from the Chief Archivist that covers records of core functions? (n= 176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Analysis of information gathered by Archives New Zealand highlights five key themes in the government recordkeeping environment:
Government exists in a world of constant change with systems, products, priorities and ways of working continually evolving. Although constantly changing, government must remain accountable and transparent. Good information management helps ensure both continuity of government services and the continued accountability of government during times of change. Records from yesterday form the basis of decisions made today which will impact tomorrow.
In today’s environment, the way government operates must change in order to remain responsive. Government has signalled that it is focused on delivering better, smarter public services with limited resources due to the current economic situation. Both the structure and the technology used to deliver services are changing. Public offices must consider their recordkeeping requirements in this changing environment. In last year’s report I highlighted the risks to records associated with changes made to the functions of government including:
Over the past year a number of public offices and local authorities, including Archives New Zealand, have been implementing machinery of government changes such as the merging of agencies. We have worked with these agencies to manage the recordkeeping risks and to ensure the expected benefits can be realised.
Evidence gathered by Archives New Zealand in discussions with public offices involved in machinery of government changes during 2010 suggests that systems’ lack of interoperability was a barrier to the smooth transition of information. Standardisation of information creation is a key way to support interoperability between systems.
Archives New Zealand has issued several standards to support the creation of standardised information including the Electronic Recordkeeping Metadata Standard and the supporting Technical Specifications and the Digital Recordkeeping Standard. These standards support the creation of reliable records. When building or redesigning business systems, all public offices should be considering their recordkeeping requirements and using the Archives New Zealand standards to assist the creation of information assets that are authentic, reliable, have integrity and are easily reusable.
The way government delivers services is also changing. In today’s world online is frontline, with digital technology including the internet transforming the way we interact and do business. Digital technology offers significant benefits to how government does business and interacts with the public. As government increasingly operates in the digital environment it is important that robust recordkeeping frameworks are translated into the online world to ensure that accurate and reliable information is available to support digital processes. A recordkeeping framework should be considered as a strategy, not as an automated system or an off-the-shelf software package. A considered recordkeeping framework will assist public offices to effectively create, maintain, make available and dispose of records regardless of the format in which these records are generated.
Digital technologies have the potential to automate and streamline information management. In order to capitalise on this, public offices must understand and incorporate good information management and architecture practices when developing systems. Systems must be supported by appropriate resources, training for staff using the systems and with appropriate monitoring to ensure they remain up-to-date and effective.
Considered investment in the creation of reliable information assets will have long-term benefits to New Zealand. Public offices have a responsibility to ensure information assets are created and maintained in a manner that supports efficient business practice enabling the delivery of better, smarter public services. A good information management framework underpins the continuity of government services over time.
Public offices should use the Archives New Zealand Standards to ensure they are creating information assets that are authentic, reliable, have integrity and are easily reusable.
Public offices should ensure they have robust recordkeeping frameworks for both physical and digital records.
As government capitalises on the efficiencies that technology brings, there will be an increasing volume of digital information that must be actively and appropriately managed. The often complex nature of digital records means that there is a significant risk that the data held by public offices will become inaccessible. On average, digital data has a life cycle of just seven to 10 years.
In order to realise the significant benefits of digital information and systems, public offices must make sure they take active steps to manage long-term access to digital information from the moment of its creation. Digital information management is not a new phenomenon. We have enough information to know that poorly managed digital information is a significant burden to government.
Open and transparent government relies on the sustainability of government information. The inability to access reliable digital information disrupts the continuity of government. We must not repeat the mistakes of yesterday’s systems. Public offices must ensure that the management of digital information is integrated into all business systems and practices.
Figure 8 (below) demonstrates that government already faces the challenge of dealing with information contained within legacy digital systems. Twenty-two per cent of public offices are still not aware of whether they hold digital records that are older than 25 years. This is a significant number when compared to the one per cent of public offices that were uncertain if they held paper records over 25 years old. The lack of knowledge about the amount of legacy digital information highlights the complexities of managing digital information.
If we do not act now we will have a gap in the memory of government. Long-term security of and accessibility to digital information will be secured by considering the nature of the information that needs to be maintained by a system and ensuring an appropriate recordkeeping framework is in place at the point of system design.
Question 17: What quantity of electronic records does your organisation hold that were created before 1985? (n=120) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Note: Total exceeds 100% because of rounding.
Figure 9 (below) shows that investment in core records and information management principles for digital information management, such as recordkeeping metadata, is beginning to have a positive impact on long-term access to digital information.
It is encouraging to see that public offices have taken on board my 2009 recommendation that at-risk information is identified and action is taken to address risks. In 2009, 58 per cent of public offices reported an inability to access digital documents due to documents being saved or archived without appropriate titles or other metadata. In 2010 this figure had decreased to 40 per cent.
Question 7: Does your organisation hold any records it can no longer access for the following reasons?
(n=176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Archives New Zealand is actively committed to the long-term management of digital information of archival value. In 2010, the government announced $12.6 million in new Budget funding to develop a Government Digital Archive. The funding will enable the development of an industrial-strength digital archive to take in large-scale transfers of digital public records.
The existing infrastructure of the National Library’s National Digital Heritage Archive will be used as a base on which to build a full solution for digital public archives. Integration with existing and new software and a redesign of Archive New Zealand’s internal processes and interactions with agencies will also occur.
From mid 2012, public sector agencies will be able to transfer digital public archives to Archives New Zealand. Archives New Zealand has begun working with agencies to involve them in the development process to ensure the digital archive meets their needs.
The Government Digital Archive Programme aims to achieve a cost-effective, efficient and sustainable digital archive process and system. This will ensure high-value public sector digital information is secure and available for future generations.
As the way government conducts business becomes increasingly digital there is more urgency for public offices and Archives New Zealand to ensure digital information is created and managed appropriately. This will ensure today’s digital information is secured for and available to tomorrow’s users.
Public offices should ensure that technical systems are designed to create and maintain records for as long as they are required.
The earthquakes in the Canterbury region in 2010 and 2011 have highlighted the impact that natural disasters can have on New Zealand.
Following these earthquakes, Archives New Zealand staff contacted public offices that held records in the Canterbury region. Many public records were lost or damaged as a result of the earthquakes. Archives New Zealand is now working with public offices in the Canterbury region to manage the rescue and recovery of records and address priority areas.
Much has been learnt as a result of the earthquakes. Archives New Zealand will be working with public offices across New Zealand to increase awareness of the issues and support the development of plans and strategies to prepare for and manage the effects of natural disasters on information.
While natural disasters have significant and visible effects, disasters can occur in many forms. A disaster is any event that creates an inability for an organisation to provide critical business functions. Appropriate disaster recovery/business continuity planning for records and information management can reduce the likelihood of such disasters and minimise their impact.
Information from our annual government recordkeeping survey indicates that many public offices are not prepared to manage their information in the event of a disaster. As figure 10 (below) illustrates, only 34 per cent of public offices reported they have a disaster recovery plan covering their paper records and only 60 per cent reported they have a business continuity plan for electronic records, down from 67 per cent in 2009.
Question 5: What programmes and procedures are in place to ensure that your organisation is maintaining its records over time? (n= 176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Public offices should work to develop disaster recovery and business continuity plans based on a risk assessment undertaken by the organisation. A plan will allow the public office to develop and implement strategies to address disaster and business continuity risks. The plan should address both prevention and response.
In order for plans to be effective, they must be implemented and maintained. Plans should be continually updated to reflect changes to business processes and systems to ensure that they are comprehensive. The plans also need to be regularly tested. In the Digital Information at Risk survey run by Archives New Zealand in 2010, 91 per cent of respondents identified that untested disaster recovery plans posed a medium to high risk to digital information.
A key component of a successful business continuity and disaster recovery programme is the identification of vital records. A vital records programme is part of a comprehensive records management programme. In 2010, only 22 per cent of public offices were operating a vital records programme. Agencies should move to address this gap in their recordkeeping programmes. Identification of vital business information is an ongoing activity and will need to be updated to reflect changes to processes and systems.
Comprehensive and well tested business continuity programmes, vital records identification and disaster recovery planning will help ensure that public offices can react quickly in the event of a disaster. Access to vital business information is a key component of getting government services back up and running promptly following a disaster. Comprehensive planning can also reduce the risk of losing important information assets.
Public offices should work to develop and regularly test disaster recovery and business continuity plans for records and identify their vital records so they are ready for disasters.
Disposal is the action taken on a record when it is no longer required to be retained by a public office. Disposal covers a range of activities including destruction, transfer to Archives New Zealand or another public office, alteration, sale or discharge.
Under the Act, the disposal of public records must be authorised by the Chief Archivist. Centralised authorisation for the disposal of public records ensures a whole of government approach to the long-term management of New Zealand’s public sector information assets.
All records created as a result of the common administrative functions of government, such as Human Resources and Finance, are covered by the General Disposal Authorities issued by the Chief Archivist. Public offices must work with Archives New Zealand to develop disposal authorisation for their core business records.8
Gaining disposal authorisation from the Chief Archivist is a key step in establishing a successful records management programme. Disposal authorisation establishes the minimum time records must be retained by the public office before a disposal action can take place. Knowing the minimum retention periods for records helps ensure records are only managed for as long as necessary before disposal is implemented. This allows public offices to better focus limited resources and save on administrative and storage costs.
Early identification is particularly important in the digital world where vast quantities of public records are created on a daily basis. It is pleasing to see that there is a growing awareness of the need to develop digital disposal coverage and processes.
Last year, I recommended that public offices ensure they have comprehensive disposal coverage. Figure 11 illustrates the continual upward growth in the level of disposal authority coverage. In 2010, almost half (49%) of public offices reported that they had a disposal authority in place for records of their core business function. This is an increase from 43 per cent in 2009. The positive trend in disposal authorisation reflects an increasing engagement between public offices and Archives New Zealand to ensure disposal authorisation for public records is coordinated across government.
8 General Disposal Authorities available at http://archives.govt.nz/advice/continuum-resource-kit
Question 11: Does your organisation have disposal authority from the Chief Archivist that covers records of your core function? (n=176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Collaboration between organisations to gain disposal authorisation minimises duplication of effort and is an efficient way for public offices to realise the benefits of disposal authorisation and subsequent disposal implementation.
Several sectors have made significant gains in disposal authorisation by collaborating to develop sector specific disposal authorities. In 2010, the Tertiary Education Commission sponsored a project in which 17 of the 20 Institutes of Technology and Polytechnics collaborated to develop a sector wide general disposal schedule. As a result of this project 93 per cent of Tertiary Education Institutes now have comprehensive disposal coverage. This is an increase of 77 per cent, from 16 per cent in 2009 to 93 per cent in 2010. I encourage other sectors to look at collaborating to develop sector-wide disposal authorisation.
In 2010, Archives New Zealand published the mandatory Disposal Standard. The disposal standard sets guidelines for public offices on managing the disposal process. Public offices should be using the standard to assist with the implementation of disposal.
Question 12: Has your organisation disposed of any records in the past year? (n=176) From Government Recordkeeping Survey 2010: Public Offices - Final Report
Disposal authorisation allows public offices to legally undertake disposal actions on their records. Figure 12 (above) shows that since the passing of the Act in 2005, the percentage of public offices implementing disposal has remained fairly constant despite the significant increases in disposal authority coverage.
Regular and routine disposal of public records is a key way to improve business efficiency. Implementing disposal reduces storage costs and saves unnecessary time and expense managing records that are no longer required. All public offices should be regularly and routinely disposing of records using the General Disposal Authorities and disposal authorities covering their core business records.
Question 10: Does your organisation have a regular disposal programme in place for records? (n=176) From Government Recordkeeping Survey 2010: Public Offices – Final Report
Last year I recommended that public offices ensure they have robust processes for the disposal of records and that disposal is undertaken regularly. It is pleasing to see in the 2010 survey results, illustrated in figure 13, that a significant number (39%) of public offices are currently developing a regular disposal programme for their records. Developing processes and systems for disposal and integrating disposal into business processes helps ensure that the efficiencies of regular and routine disposal are realised. Failure to regularly implement disposal is a waste of government resources.
Public offices should ensure they have comprehensive disposal coverage under the Public Records Act 2005, including ongoing disposal schedules for their core business records.
Public offices need to ensure that disposal of public records is planned and regularly implemented in line with the requirements of the Disposal Standard.
Archives New Zealand has a responsibility to gather, store and protect the record of government. Valuable information about the history of our nation remains scattered across public offices where it largely remains inaccessible to government and members of the public. Information gathered during the government recordkeeping survey informs us that two in three public offices (68%) hold records over 25 years old.
Archives New Zealand’s engagement with public offices informs us that many of these records are well over 25 years of age. For instance, in 2010, Archives New Zealand received records that were created as early as 1855 in transfers from public offices.
In 2008, Archives New Zealand launched the Legacy Records Programme to help public offices manage records in their custody that are over 25 years old. The programme ensures the interests of public offices, Archives New Zealand and the New Zealand public are taken into account when decisions are made concerning legacy records.
Archives New Zealand continues to work actively with public offices helping them to better manage their legacy records and when required to transfer these records to the archives. Legacy records are given priority for transfer into our repositories.
As records age they require more intensive care to ensure they remain accessible. Archives New Zealand staff work with public offices to facilitate the safe transfer of public records to our repositories. These records are then given a contextual description so they can be easily located within the archival system and made available to the public through our reading rooms and online.
Public offices holding legacy records of archival value should contact Archives New Zealand to discuss transfer options. Transferring historical records of archival value to Archives New Zealand eases the burden on public offices to care for these records and lets them focus on their core business.
I encourage public offices to engage with the Legacy Records Programme run by Archives New Zealand. Incorporating the Legacy Records Programme into ongoing records management programmes enables public offices to maximise use of limited resources. As a result they can achieve significant improvements in their overall recordkeeping capability.
Records created by government have a fundamental role in helping New Zealand and New Zealanders to understand what and who we are as a nation. It is important for public offices to transfer these records to Archives New Zealand so they are preserved for future generations of New Zealanders.
Public offices should incorporate legacy records management into ongoing records management programmes to maximise limited resources and improve ongoing recordkeeping capacity.