Report on the State of Government Recordkeeping 2019/20

In 2014/15, the Chief Archivist expressed concern about the low level of maturity of recordkeeping in more than half of the public offices audited under the Public Records Act 2005 (PRA). Since then, a programme of work to optimise Archives New Zealand’s regulatory role has been undertaken, with a mandatory Information and records management standard (the Standard) issued, along with a Regulatory statement, the revision of core guidance material and the embedding of the Executive Sponsor role.

These have provided a strong foundation from which to develop our monitoring framework. The annual survey of public sector information management practice is now into its second year. Some of the findings from this are covered later in this report. In 2020, we are launching the second round of audits under the PRA and a newly devised maturity self-assessment. These will assist regulated parties in understanding where they are at, and where they need to be in order to reach acceptable levels of maturity in their information management practice.

Five years is long enough for changes to a system and we are beginning to see these come through in the responses to our annual survey, and in our work assessing compliance with the PRA. The redevelopment of our regulatory programme has highlighted that a considerable driver of low maturity levels across the public sector is inadequate resourcing of information management areas. This is reflected in the results of the annual survey, which show low volumes of IM staffing, a lack of resources for IM activities, unsuitable system set-up and low prioritisation of IM within organisations.

Key to improving the state of government recordkeeping is the inclusion of IM considerations at the executive level. We have seen a slight rise in regulated parties reporting that they have included IM components to their governance arrangements. This is a positive result and we strongly encourage all organisations to ensure that IM requirements are considered when developing organisational strategies and policies, systems and processes. This is a foundation for elevating the importance of IM in organisations and integrating it into business operations.

Through our work assessing compliance with the PRA, it is clear that the management of digital records, and legacy digital records in particular, remains an issue for public offices and local authorities. Survey results also indicate a decrease in the number of organisations incorporating IM requirements into new business systems. Again, this appears to be grounded in less than ideal resourcing, as we are noting various degrees of compliance with principle 1.4 of the Standard. This requires organisations to have IM staff or access to appropriate skills. The onus for this sits with the leadership of organisations (principle 1.2 of the Standard).

As digital technologies are critical to public service operations and in some cases rapidly adopted during COVID-19 lockdowns, it is critical that the transition to digital processes does not lose sight of the requirements of the PRA and the Standard, and the relevant privacy and security considerations. In March this year, we released guidance on the adoption of Microsoft 365 software, noting that out of the box, these commonly used tools are not compliant with the PRA. We discuss this in more detail further on in this report.

Information is at the core of government business and is a key strategic asset. IM is the discipline that allows information assets to be governed, protected and prioritised. Survey results for 2020 indicate a slight decrease in organisations who have identified their high-risk and/or high-value information. This is concerning. Knowing and understanding the value and risk related to information assets is good IM practice. It also helps protect an organisation from risks to its reputation, from potential financial or material loss, and breach of statutory obligations.

Lastly, we have noted a decrease in the volume of authorised disposals over the past year. Destruction of digital information remains very low compared to physical information. Several factors contribute to this including the top three challenges of inadequate system set-up, resourcing and low prioritisation. The ongoing suspension of transfer to our Wellington repository and the Moratorium on the disposal of records relevant to the Royal Commission of Inquiry into Abuse in Care will have slowed disposal. However, it is concerning to see a lack of progress on digital destruction given that risk management is indicated by most organisations as an extremely important driver for good IM practice and the increasing number of organisations transitioning from paper-based to digital business processes. We will continue to work with regulated parties to improve responses in this area and ensure that authorised disposal takes place where appropriate.

Information management in the time of COVID

The information and records created in response to COVID-19 are invaluable in ensuring that the government response is effective, and that government can be held accountable now and over the long term for its decision making. Many of these records will be of long-term archival value.

In response to COVID-19, we advised the public service that as they operated under the restrictions imposed by the various alert levels, the information that was being created, often in rapidly improvised systems and workflows, would still need to be properly managed in accordance with the Standard, and current disposal authorities. We reminded organisations that it is imperative that all reasonable measures are taken to ensure that this information is identified, preserved and classified to support appropriate disposal in the future. The demands of the COVID-19 response across government reinforces the need to be able to share information between organisations safely and legally, maintaining the integrity of the record.

Microsoft 365

In June 2020, we released our guidance on the use of Microsoft 365 services. This was timely as the shift to work from home has seen an increase in the uptake and popularity of many of the software services that are commonly part of the 365 suite. These include Outlook mail services, Skype for Business, SharePoint Online, the collaboration tool Teams, and the standard Office applications such as Word and Excel, among others.

Minimum compliance requirements of the Standard apply to the Microsoft 365 environment as they do with any system that creates and manages public or local authority records. In its standard roll-out form, Microsoft 365 is not compliant with the Standard or the PRA. The implication of this for information and records management will depend on how organisations configure their software, the type of licence held and whether Microsoft 365 is integrated into an electronic document records management system (EDRMS) or enterprise content management system (ECMS).

Our guidance sets out how organisations can move toward compliance with the Standard and the PRA by applying controls, recognising risks and taking the principles of the Standard and the PRA into consideration. This requires organisations to understand the administrative applications and tools used to manage information and records in Microsoft 365. It is necessary to understand where and how information is stored across the suite including whether off-shore storage is appropriate for the type of information being held. Gaps in capability may need to be addressed by using third-party add-ons for IM functionality.

We recommend that IM and Information Technology staff work closely together to monitor and manage their instance of Microsoft 365 to ensure that updates and enhancements do not undermine compliance measures that have been implemented.

Open Government Partnership – Commitment 10: Monitoring the effectiveness of public body information management practices

The Open Government Partnership is an international agreement by governments to create greater transparency, increase civic participation and use new technologies to make governments more open, effective and accountable. The Public Service Commission leads this work in New Zealand.

Under the Third National Action Plan, we are leading Commitment 10: Monitoring the effectiveness of public body information management practices. The objective of Commitment 10 is to make the creation and management of government information more visible and therefore transparent, by developing and implementing a monitoring framework that supports public reporting on the effectiveness of IM in central and local government organisations.

Throughout 2019/20, we continued to develop and roll out the monitoring framework which reflects the Standard. The framework consists of the annual survey of public sector IM launched in 2019, a new Information Management Maturity Assessment and published Public Records Act audit reports. The monitoring framework is part of a larger work programme to implement our long-term strategy, Archives 2057, and feeds directly into our strategic focus areas of upholding transparency and building systems together. These focus areas support open government principles and shape the processes and systems of public sector information and records management.

Public Records Act Audits

Monitoring public sector information and records management is a shared responsibility between Archives and each public sector organisation. Section 33 of the PRA confers the audit function on the Chief Archivist.

The PRA audit is a point-in-time view of core IM practices that recognises an organisation’s operating environment and IM challenges. It is also an opportunity to show an organisation’s IM strengths and where there might be opportunities for improvement. Only public offices are audited under the PRA; we are not mandated to audit the IM practices of local authorities. Some public offices are outside the current scope of the audit programme, such as school boards of trustees and Ministers of the Crown.

Audits help ensure that public offices are delivering high-quality information to decision-makers, other government organisations, customers and stakeholders. They enable continuous improvement within a rapidly changing environment.

During 2019/20, we developed our audit programme for public offices. The first-year cohort of audits began in December 2020 and will be completed by the end of June 2021.

PRA audits are based on the Standard and the PRA, as represented through the IM Maturity Assessment. The audit programme will consider eight key areas of IM practice outlined in the IM Maturity Assessment: governance, self-monitoring, capability, creation, management, storage, access and disposal. Our follow-up approach will track remedial and improvement actions undertaken by public offices as a result of audit findings. After each audit, there is an expectation that the public office will use the recommendations identified by the independent auditors and Archives to create an action plan for their organisation to improve IM maturity. We will work with the public office audited to monitor their progress through the activities identified within their action plan.

This will also assist us to target advice, guidance, and other activity to support public offices and local authorities based on insights gained from the surveys and audits. You can read more about our audit programme, and view the schedule for public office audits here.

How we identify issues

Information and records are vital for accountability and transparency. They form the basis for significant decision making. When issues arise with information management that frustrate accountability, the Chief Archivist’s regulatory role allows intervention.

The PRA does not provide Archives and the Chief Archivist with explicit investigative powers, but potential non-compliance is assessed against the Standard and the principles of the PRA. If we find through the assessment process that a breach of the PRA or the Standard has occurred, we take action and work with the parties concerned, including the responsible organisation, on remedies.

Our tools include the ability to direct a public office to report to us, to inspect records and to give directions about how estray records must be managed. The PRA also contains offences, notably damaging or improperly disposing of records. We have not actively considered prosecution this year.

We find out about potential non-compliance issues comes from various sources:

• our monitoring of regulated parties

• our daily interactions with regulated parties

• complaints from affected or concerned members of the public

• information reported in the media or received as part of journalists’ investigations

• complaints from concerned third-party organisations

• referrals from the Office of the Ombudsman.

Working with the Ombudsman

The PRA intersects with several other Acts. The main ones are the Official Information Act 1982 (OIA), the Local Government Official Information and Meetings Act 1987 (LGOIMA) and the Privacy Act 2020.

Throughout 2019/20, we continued our close working relationship with the Office of the Ombudsman, Tari o te Kaitiaki Mana Tangata.

Some complaints made to the Ombudsman include instances where organisations cannot supply information because:

• it cannot be located, despite extensive searching

• it would require significant collation and research to be made available, and/or

• it is determined not to be held when it could be reasonably expected that the information should exist.

Under section 28(6) of the OIA and section 27(6) of the LGOIMA, the Ombudsman may notify the Chief Archivist when an information request has been refused by an organisation for these reasons.

Understanding Public Records Act obligations

While Archives New Zealand receives requests for assessments from the Office of the Ombudsman and from the public, we also respond when the media or other sources publicise IM concerns. One of our key tools is the provision of advice and guidance. We work closely with agencies to understand their needs, concerns and areas for improvement in order to build better compliance with the PRA and improve our advice and guidance.

Records, rights, and royal commissions

Archives New Zealand has multiple roles in supporting inquiries and plays a pivotal role in supporting the Royal Commission of Inquiry into Historic Abuse in State Care and in the Care of Faith-Based Institutions (Abuse in Care Inquiry). We provide advice and instruction on government records management, the regulation of disposal of public records, and the preservation of and provision of access to public archives that document New Zealand’s history, heritage and identity.

Records created in the course of government activities can seem trivial or transitory at the time of creation. To a care-leaver or abuse survivor however, such records can contain information about their childhood and life which may not form part of their own memory or knowledge. The records created by government organisations, and their agents, can help care-leavers understand who they are and where they come from, telling them something about their parents, birthplace, childhood or other significant events.

Our work in supporting the Abuse in Care Inquiry has highlighted several areas of IM and PRA compliance that are concerning to us.

Recordkeeping must support inquiries

During 2019/20, district health boards (DHBs), among other organisations, began to receive Notices to Produce from the Abuse in Care Inquiry. These notices are issued under section 20 of the Inquiries Act 2013 and require the receiving agency (or individual) to produce documents and provide information necessary for the Abuse in Care Inquiry to identify, examine and report on the matters within scope of the inquiry. As the Abuse in Care Inquiry is expected to deliver its final report in 2023, we expect these requests for public records and archives to continue for some time.

DHBs will have records that are relevant to the Abuse in Care Inquiry. As the terms of reference for the Inquiry extend from 1950 through to the present day, relevant records could be active, inactive or archival records stored at the DHB, contracted storage, with Archives New Zealand or at an alternative approved repository.

As section 20 Notices to Produce continue to be issued, we expect that adequate recordkeeping controls are in place over the records relevant to any request. Section 17 of the PRA requires public offices to create and maintain in an accessible form, full and accurate records of their affairs. We strongly recommend that organisations that receive section 20 Notices confirm the mandate for research and ensure that they only provide access to records within scope of the Notice, ensuring that the IM staff know the contents of the records they are providing access to before giving researchers access. Keeping a register of all records that are accessed and copied, and by whom, will be important in ensuring the chain of custody for the records.

If organisations are meeting their recordkeeping requirements, then there is less likelihood of failing to produce documents for any inquiry. Costs will be incurred in identifying, locating, retrieving and copying relevant documents. These costs are not a convincing reason for failing to produce them, and, if high, are a likely indicator of fundamental information and records management deficiencies.

Ongoing disposal moratorium

The Chief Archivist has issued a moratorium on the disposal of records held by all public offices which may be relevant to the subject matter of the Abuse in Care Inquiry. This means that no public records that may be relevant to the Abuse in Care Inquiry can be destroyed, discharged, or transferred while the Royal Commission is in progress.

A number of DHBs have asked us whether they can destroy patient records in line with their disposal authorities where the records are deemed to not be of relevance to the Inquiry. A driver for this appears to be storage and cost pressures. The Abuse in Care Inquiry is responsible for determining what records are relevant, not Archives New Zealand or any of the Crown response agencies. Our advice to agencies that are not sure about whether records are relevant or not is: if in doubt, retain the records. This mirrors the advice received from the Abuse in Care Inquiry to DHBs that while they can continue with appraisal and sentencing of records, no records can be destroyed until after the moratorium is lifted at the end of the Commission’s mandate.

Outsourcing of functions

Engaging external parties to perform business functions has implications for information and records management. It is reasonably common for DHBs particularly to outsource some functions. Under the PRA, every public office must create and maintain full and accurate records of its affairs, in accordance with normal, prudent business practice. Outsourcing a business function or activity does not lessen an organisation’s responsibility to ensure that it is carried out properly and that all requirements for information and records management are met.

When outsourcing business, public offices are responsible for ensuring that the contracted party is actively creating and maintaining information and records of their affairs. We recommend that these provisions are clearly articulated in contracts with service providers, and that these are monitored to ensure obligations are being met. Organisations can learn more about their responsibilities here.

Similarly, while the moratorium on disposal of records relevant to the Abuse in Care Inquiry does not directly apply to non-public records, that is records created by non-government agencies, any records created by non-government organisations that relate to work carried out on behalf of the government agencies are covered by the moratorium.

Key findings from the Survey of public sector information management 2020

In 2019/20 Archives New Zealand conducted its second annual survey of information management (IM) practices in public offices and local authorities.

Objectives of the survey

Information is at the core of government business and is a key strategic asset. IM is the discipline that allows information assets to be governed, protected and prioritised.

In 2018/19 we reinstated an annual survey. As part of the survey design, we selected five key indicators to measure the overall state of public sector IM and provide a high-level perspective on whether IM within the public sector was improving, deteriorating, or remaining stable. With the completion of this year’s survey, we can begin to see whether progress is being made against the indicators and identify where we need to take more proactive steps to bring about improvements. This year, we made some refinements to the survey questions that affected our ability to directly compare some results with the first survey. The picture of change over time is still emerging and will improve over successive surveys.

The key indicators are not the sole measure of the state of public sector IM, but they have been selected because we consider them to be fundamental building blocks for effective IM. The full survey results provide more comprehensive data on the performance of public sector organisations. These results will be reported in a separate findings report, and we will publish the raw data on data.govt.nz. We will also use them to inform our interactions with individual organisations over the coming year.

Who was surveyed?

The survey was sent to 270 public sector organisations, including:

• 191 public offices, which were required to respond by direction to report (section 31 of the PRA)

• 79 local authorities, which were invited to respond.

The survey recorded an 80% response rate, slightly down on last year’s response rate of 89.7%. We received one unsolicited response, two late responses and five incomplete responses, all of which were excluded from the analysis. A total of 47 organisations did not respond, comprising 22 public offices and 25 local authorities. The responses from the Government Communications Security Bureau and New Zealand Security Intelligence Service are excluded from the analysis.

Indicator 1: An increasing number of organisations have implemented governance groups for information management

What we asked and why it is important

We asked survey participants if they have a formal governance group in place which is either dedicated to IM or has IM oversight as part of its mandate (Q.9).

The role of an active governance group is to ensure, at a strategic level, that IM requirements are considered when developing organisational strategies and policies and implementing systems and processes. It is a foundation for elevating the importance of IM in organisations and integrating it into business operations.

What we found and how it compares to last year

This year, just over half of respondents (52%) have a formal governance group in place. Last year, 30% of respondents had a formal governance group in place, while 24% had one in development. Although we did not provide respondents with an ‘in development’ answer option this year, the data suggests that there has been an increase in the proportion of organisations implementing governance groups for IM (Figure 1). This change may reflect work completed by respondents who previously had a governance group ‘in development.’ That said, it is concerning to see that a significant number of respondents (48%) are lacking a formal governance group and missing out on the potential benefits such a group can bring to the organisation’s IM.

Figure 1: Indicative change over time for Indicator 1

Looking at the response split by organisation size it is encouraging to see that for very large organisations (3000+ full-time equivalents) most organisations have a formal governance group in place (Figure 2). These organisations are more likely to operate in a complex information environment and hold high-value and/or high-risk information. However, as a whole there does not seem to be an obvious relationship between organisation size and the presence or absence of formal governance groups.

Recommendations

In 2018/19 we recommended that executive sponsors take a lead in their organisations to establish and/or sustain governance groups for IM. Reflecting on the insights from this year, we will seek to influence change in those organisations that do not have a formal governance group. This approach may include:

• making expectations more explicit in our guidance, communications and interactions with executive sponsors

• engaging with selected organisations that hold high-value and/or high-risk information, on the basis that an absence of formal IM governance heightens the risk of IM failure and consequent public harm

• engaging with individual organisations that lack formal IM governance, as a component of follow-up on their audit findings.

Indicator 2: An overall increasing number of IM staff employed by public sector organisations

What we asked and why it is important

We asked how many dedicated, full-time equivalent (FTE) IM staff organisations employed (Q.5). The question asked respondents to exclude staff in geospatial information systems, business intelligence, data management, medical records or staff whose main role is not IM.

IM impacts all areas of business, and IM specialists should be involved and included in a wide variety of business activities. These include system and process design, information and records sharing, risk management, and managing information, data and records for accountability and value.

As new technologies proliferate at speed, the opportunities and challenges for meeting IM requirements also multiply. IM specialists remain essential for the proper functioning of digital government, through their IM leadership and advocacy, and by harnessing the abilities of technology to make IM easier for their organisations.

What we found and how it compares to last year

This year, 79% of respondents have some dedicated, specialised IM resources. Last year, the figure was also 79%. This suggests that from a sector-wide perspective the level of resourcing has been essentially static over the past 12 months (Figure 3).

Figure 3: Change over time for Indicator 2

For the 2019/20 survey we refined the response options for the question so that organisations could tell us the exact number of IM staff they employed, rather than selecting a range. This means that after next years’ survey we will be able establish whether there has been an overall increase in IM staff numbers. Table 1 shows this year’s data, configured against the range we used last year, alongside last year’s data.

Table 1: Breakdown of change in IM staffing levels between surveys

Recommendations

In 2018/19 we recommended that executive sponsors take steps to consider whether their staffing levels or lack of dedicated IM staff created risks for the organisation and take actions to manage those risk. While this recommendation still stands, the findings suggest that more work is required on our part to:

• engage with selected organisations that have no dedicated IM staff, as a component of follow-up on their audit findings

• establish recommended staff metrics, which can assist executive sponsors with assessing appropriate staffing levels.

Indicator 3: An overall increase in organisations that have identified their high-value and/or high-risk information

What we asked and why it is important

We asked survey participants if they have identified their high-value and/or high-risk information (Q.32).

For an organisation, high-value information is information that is critical to performing its core, legislated functions. High-risk information is information that, if mismanaged, could expose the organisation to major operational failure, financial or material loss, breach of statutory obligations, or loss of public or Ministerial confidence.

For New Zealanders, high-value information is information that supports their individual or collective rights, entitlements, identity and aspirations. High-risk information is information that, if mismanaged, could result in public harm. Actions such as improper release of information or barriers to access can have real-world impacts on the lives of New Zealanders. Those impacts can include physical, emotional and psychological harm. We have seen this through the work of the Abuse in Care Inquiry.

Identifying high-value/high-risk information is a foundation for other IM activities. It is a critical first step towards mitigating associated risks and extracting maximum value from information assets.

What we found and how it compares to last year

This year, 36% percent of respondents have identified their high-value and/or high-risk information, while 43% say that work is ‘in progress’. Last year, 64% of respondents had identified their high-value and/or high-risk information and we did not provide an ‘in progress’ answer option. Although refinements were made to the question this year with the addition of the ‘in progress’ answer option, on the face of it the data suggests that there has been a decrease in the proportion of organisations that have identified their high-value and/or high-risk information (Figure 4). This downward trend is concerning but, factoring in the modifications made to the question this year, we will look to next year’s survey for confirmation of that trend.

Figure 4: Indicative change over time for Indicator 3

We also asked organisations about whether they had an information asset register (IAR) or similar tool in place (Q.23), as a helpful means of documenting high-risk and/or high-value information assets. An IAR-type tool puts an organisation in a strong position to manage accessibility and usability, mitigate risks that might affect the assets and manage their relevance, currency, retention and disposal.

Only 38% of respondents said that they do not have an IAR, while 48% responded ‘yes’ or ‘in development’, and 14% responded ‘work started but deferred’ (Figure 5). This suggests that just under half of respondents are moving beyond the discovery stage and ensuring that their high-value and/or high-risk information assets are documented.

Although for many organisations we have indications of promising progress with the discovery and documentation of high-value and/or high-risk information, it is an activity we consider important for all public sector organisations, regardless of their size, function or complexity.

Recommendations

In 2018/19 we recommended that public sector organisations take actions to help them manage their high-value and/or high-risk information, such as using an IAR and including IM requirements in system specifications for business activities that create this type of information. The findings show that there is still work to be done with those organisations that have not made a start on discovery and documentation, so in the coming year we will focus on:

• engaging with organisations that have not identified their high-value and/or high-risk information as a component of follow-up on audit findings.

Indicator 4: An overall increase in the number of organisations building IM requirements into new business systems

What we asked and why it is important

We asked survey participants whether they have built a process for managing information through its lifecycle into new business information systems (i.e. systems implemented in the last 12 months) (Q.35).

Building IM requirements into a business system from the very beginning is a key enabler for proper management of the information created and stored in that system. This means that the system is optimised to support the creation and maintenance of complete, accurate and accessible information, as well as its eventual, authorised disposal.

We recognise that it can be extremely challenging to retroactively add or plug-in IM requirements to existing systems, particularly when they have already been in operation for an extended period and are bespoke, no longer supported or at end of life. For new systems, we expect these requirements to be built in from the start.

Business information systems are not limited to electronic documents and records management systems or enterprise content management systems. Information that has to be managed in accordance with our requirements is created and stored across a wide variety of business systems, including:

• finance and human resources

• line-of-business systems that support the organisation’s unique functions

• systems that support collaboration between government organisations and/or external parties

• email and email archiving systems

• network drives.

What we found and how it compares to last year

This year, of the 147 organisations that implemented a new business information system in the last 12 months, half (50%) have built in a process for managing information through its lifecycle, while 17% of respondents replied, ‘don’t know.’ Last year, we asked a similar question without the 12-month timeframe and found that 23% of respondents had built IM requirements into new systems, while 62% had partially built in requirements, for a combined figure of 85%.

Refinements were made to both the question and answer options in this year’s survey, with the addition of the 12-month timeframe and the removal of the ‘partially’ answer option. If we combine the ‘partially’ and ‘fully’ responses from last year’s survey as a ‘yes’ response, then the data suggests that there has been a decrease in the proportion of organisations that have built IM requirements into new business systems (Figure 6). This downward trend is concerning but, factoring in the modifications made to the question this year, we will look to next year’s survey for confirmation of that trend. In any case, we consider that the proportion of organisations building IM requirements into new systems remains too low.

Figure 6: Indicative change over time for Indicator 4

Figure 7 plots this year’s responses against the presence or absence of a formal IM governance group. This shows that when a formal governance group is present there appears to be a greater likelihood that the organisation has also built a process for managing information through its lifecycle into new business information systems. One of the purposes of a formal governance group is to ensure, at a strategic level, that IM requirements are considered when developing systems.

This year we repeated a question about the challenges affecting organisations’ ability to build in IM requirements when developing new systems. The top three challenges remain the same as last year: lack of awareness amongst internal staff, the number of systems in use and lack of consultation with IM staff.

Recommendations

In 2018/19 we made several recommendations about how executive sponsors could address the challenges their organisations face with building IM requirements into new business information systems. This year’s findings suggest that we need to take further steps to help organisations overcome those challenges. Those steps may include:

• using our guidance, communications and interactions with executive sponsors to emphasise the importance of involving IM staff in new business system projects and of raising awareness amongst staff responsible for the system build

• engaging with the ICT community and industry on involving IM in new business system projects.

Indicator 5: An overall increase in the number of organisations actively doing authorised destruction of information

What we asked and why it is important

We asked survey participants if they have carried out any authorised destruction of information in the past 12 months (Q.55 on physical information and Q.56 on digital information).

Our general disposal authorities (GDAs) (GDA 6 and GDA 7) have been developed for the public sector to enable the lawful destruction of common corporate records without requiring organisation-specific authorisation from the Chief Archivist. GDAs are designed to make it easy to destroy information that has no long-term value.

This indicator focuses on destruction as one of the approved methods of disposal because it is an activity that all public sector organisations can be doing. Even if they do not have an organisation-specific disposal authority in place, organisations can still apply and action the GDAs.

Although destroying information may seem daunting or risky, it is an important component of effective IM. Typically, a large proportion of the information an organisation creates does not have long-term value for the organisation or New Zealanders, and a time will come when it is no longer required and can be safely destroyed.

The benefits of active, authorised destruction include:

• mitigating the risks associated with retaining information for longer than required, such as privacy or security breaches and unauthorised access

• minimising the quantity of digital information an organisation has to manage, thereby increasing the efficiency of business systems (e.g. fewer irrelevant search results to wade through) and making the organisation’s high value information easier to discover and manage

• decreased storage costs, for both physical and digital. The cost of storing digital information over the long-term should not be underestimated. The price per gigabyte combined with the cost of storing back-ups, versioning and vendor costs, such as retrieval charges, may be high. On 28 March 2019, a moratorium was put in place on the disposal of any records relevant to the Royal Commission of Inquiry into Historical Abuse in State Care and in Faith-Based Institutions. This is likely to have had an impact on authorised destruction by some public offices during the timeframes of the survey. However, the impact on destruction practices was not measured as an explicit component of the survey.

What we found and how it compares to last year

This year, we found that 58% of respondents had done some form of destruction (i.e. either physical or digital), compared to last year’s figure of 63%. This is a small but notable decrease in the number of organisations doing authorised destruction (Figure 8).

Figure 8: Change over time for Indicator 5

We repeated a question about the challenges affecting organisations’ ability to undertake regular, authorised destruction (Q.57). Figure 9 shows that the top three challenges remain the same: system set-up, lack of resources and lack of prioritisation by staff responsible for electronic deletion.

Authorised destruction of digital information remains low (26%) compared to physical information (52%) possibly as a reflection of the top three challenges. It is concerning to see the lack of progress on digital destruction given that:

• a high proportion of respondents (77%) told us that risk management is an extremely important driver for good IM practices and processes in their organisation (Q.7) and yet regular destruction is a component of managing risk

• a high proportion of respondents (92%) told us that they are transitioning from paper-based to digital business processes (Q.21) which will increase the volume of digital information that is ultimately going to need to flow through a digital destruction process

• as discussed for Indicator 4, only half of the respondents (50%) that have implemented new business systems in the past 12 months have built in IM requirements (Q.35) which are fundamental for enabling destruction.

Recommendations

In 2018/19 we recommended that organisations establish the groundwork for applying the general disposal authorities (GDAs) issued by the Chief Archivist and, where applicable, start applying their organisation-specific disposal authorities. The findings suggest that we need to take further steps to help IM staff members overcome the challenges identified. This approach may include:

• making the importance of regular, authorised destruction more explicit in our guidance, communications and interactions with executive sponsors. This could include addressing those challenges that a formal governance group has influence over, such as resourcing, prioritisation, and design of new business systems

• engaging with the ICT community on system functionality that supports disposal.

In additions to these actions, we will also engage with organisations that are subject to the disposal moratorium and reported undertaking authorised destruction in the past 12 months. We will seek to confirm that the information destroyed is outside the scope of the moratorium.

The Public Records Act 2005

Records of public sector activities have always been kept. However, until the introduction of the Public Records Act 2005 (PRA), there was no general legislative requirements for what information and records needed to be created and managed.

The PRA sets out the regulatory framework for information management across the public sector. The primary purpose of the PRA is to enable the accountability and transparency of government decision making by ensuring organisations create and maintain full and accurate records of their activities.

The PRA also establishes the statutory role and duties of the Chief Archivist.

These include:

• exercising a leadership role of information management across public offices

• setting standards for public sector information management

• authorising the disposal of records when they are no longer required for business purposes

• providing advice and support for organisations so they can comply with the requirements of the PRA.

Public offices and local authorities are covered by the PRA, although with different compliance requirements. A wide range of organisations are public offices, including government departments, district health boards, Crown entities, state owned enterprises, school boards of trustees and government ministers.

Regional councils and territorial authorities are local authorities under the PRA. Council-controlled organisations, council-controlled trading organisations and local government organisations are also considered local authorities under the PRA.

Archives New Zealand is a regulator

The PRA establishes the Chief Archivist as an independent information regulator within government. In delivering this role, the Chief Archivist and Archives New Zealand have responsibility for supporting, monitoring and directing the public sector to facilitate compliance with information management requirements. These requirements are set out in the Standard.

We regulate approximately 3,000 public offices and local authorities, which includes 2,500 school boards of trustees. These organisations vary widely in their size, complexity, access to funding, staffing levels, and the number of functions they carry out.

These factors all affect the level of information management maturity in organisations, as well as the level of risk associated with not being able to find or access information that has been created.

Determining the status of public offices

Archives New Zealand monitors change in the structure of the public sector and relevant legislation to ensure that both Archives and regulated parties correctly understand the extent of our regulatory responsibilities.

There are two instances where an organisation can be defined as a public office by legislation:

• if an organisation is defined as a public office in section 4 of the PRA

• if an organisation is otherwise defined as such in other legislation.

The types of organisations are that defined as public offices under the PRA include:

• all departments as defined in section 5 of the Public Service Act 2020 including departmental agencies and interdepartmental executive boards

• interdepartmental ventures as defined in section 5 of the Public Service Act 2020

• all Offices of Parliament as defined in section 2(1) of the Public Finance Act 1989

• all state enterprises as defined in section 2 of the State-Owned Enterprises Act 1986

• all Crown entities as listed in Schedule 1 and Schedule 2 of the Crown Entities Act 2004 including:

  • Crown agents

  • autonomous Crown entities

  • independent Crown entities

  • Crown entity companies

• All Crown entities as defined in section 7(1) of the Crown Entities Act 2004 including for example:

• boards of trustees for state and state integrated schools

• tertiary education institutions including colleges of education, polytechnics, specialist colleges, universities, and wānanga

• Crown entity subsidiaries (dependent on control factors).

We will be clarifying the way entities now defined by the Public Service Act 2020 need to comply with the PRA.

Glossary of key terms