ARCHIVED - Internal Audit and Evaluation Bureau - Audit of Electronic Record Keeping

This page has been archived.

Internal Audit and Evaluation Bureau

Approved: September 23, 2011

Table of Contents

• Assurance Statement
• Executive Summary
• 1.0 Introduction
  • 1.1 Operating Environment
  • 1.2 Secretariat Operations
• 2.0 Audit Details
  • 2.1 Objectives and Scope
  • 2.2 Criteria
  • 2.3 Approach and Methodology
• 3.0 Audit Results
  • 3.1 Policy and Governance
  • 3.2 People and Capacity
  • 3.3 Enterprise and Information Architecture
  • 3.4 Information Management Tools and Applications
  • 3.5 Information Management and Service Delivery
  • 3.6 Overall Conclusion
• Appendix A: Information Life Cycle
• Appendix B: Audit Criteria
• Appendix C: List of Leading Information Management Practices
• Appendix D: Management Response and Action Plan

Assurance Statement

The Internal Audit and Evaluation Bureau (IAEB) has completed an audit of electronic record keeping for the Treasury Board of Canada Secretariat (the Secretariat) as part of a broader horizontal audit initiated by the Office of the Comptroller General (OCG). The OCG audit encompasses the responsibilities of three central agencies as well as a sample of 17 large departments and agencies, and will result in a separate audit report focusing on government-wide results.

This report relates specifically to the Secretariat as a large department. While OCG developed the audit program, IAEB conducted both the detailed examination phase and the supplementary audit procedures in order to produce a stand-alone audit report for the Secretariat.

The objective of the audit was to provide assurance that the management control framework over electronic record keeping is in place and provides relevant, timely and accessible information to support decision making at the departmental level. Work was carried out by IAEB from February 2011 to July 2011 and covered activity during the 2010–11 and 2011–12 fiscal years. Specifically, the audit covered the following five lines of enquiry:

• Policy and Governance;
• People and Capacity;
• Enterprise and Information Architecture;
• Information Management Tools and Applications; and
• Information Management and Service Delivery.

We conclude with a high level of assurance that although key aspects of a management control framework over unstructured electronic record keeping are in place within the Secretariat, a number of significant improvements are necessary relative to Enterprise and Information Architecture, Information Management Tools and Applications, and Information Management and Service Delivery, in order to fully ensure the provision of relevant, timely and accessible electronic information to support decision makingand general IM practices.

A management response and action plan has been developed by the Secretariat and is presented in Appendix D.   

The audit consisted of interviews with Secretariat staff, document review and detailed testing and analysis of administrative data. We interviewed a sample of staff members to gain an understanding of information management practices within the Secretariat and reviewed documents to validate our findings. OCG provided an audit work plan and program for the first phase of the examination. This audit work plan was supplemented by additional audit work carried out by IAEB staff during the second phase to provide further assurance in selected areas. Additional interviews and a document review were also conducted. Where merited, audit testing beyond OCG requirements was conducted to support the development of a stand-alone audit report for the Secretariat.

The audit approach and methodology followed the Internal Auditing Standards for the Government of Canada and The Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

In the professional judgment of the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence has been gathered to support the accuracy of the opinion provided in this report. The opinion is based on a comparison of the conditions, as they existed at the time of the audit, against pre-established audit criteria. The opinion is applicable only to the entities examined and for the time period specified.

Executive Summary

Background

The Internal Audit and Evaluation Bureau (IAEB) has completed an audit of electronic record keeping for the Treasury Board of Canada Secretariat (the Secretariat) as part of a broader horizontal audit initiated by the Office of the Comptroller General (OCG). 

This report relates specifically to the Secretariat as a large department. While OCG developed the audit program, IAEB conducted both the detailed examination phase and the supplementary audit procedures in order to produce a stand-alone audit report for the Secretariat.

Objective and Scope

The objective of the audit was to provide assurance that the management control framework over electronic record keeping is in place and provides relevant, timely and accessible information to support decision making at the departmental level.

The scope of the audit was limited to unstructured electronic data (i.e., data produced outside enterprise systems, such as SAP), and was divided into the following five lines of enquiry that were identified by OCG during the planning phase:

• Policy and Governance;
• People and Capacity;
• Enterprise and Information Architecture;
• Information Management Tools and Applications; and
• Information Management and Service Delivery.

Key Findings

The main audit findings are presented below:

• Policy and Governance: A governance framework is in place within the Secretariat that defines information management (IM) roles and responsibilities to support unstructured electronic record keeping at each level. Although key monitoring and reporting processes are in place, improvements to planning, performance measurement and compliance monitoring would further strengthen the governance framework.
• People and Capacity: Some processes to support the development of highly skilled workforces are in place, but there is room for improvement. Specifically, available learning resources are not being universally leveraged by staff, and opportunities exist to improve planning for these activities.
• Enterprise and Information Architecture: The Secretariat has not consistently been developing information architecture and processes that respect IM risks, controls, and operational requirements. Practices vary by sector and user group.
• Information Management Tools and Applications: The extent to which key methodologies, mechanisms, and tools have been established and implemented to support departmental record keeping throughout the Secretariat varies by sector, with few consistent practices department-wide.
• Information Management and Service Delivery: Record-keeping practices have not been fully implemented to ensure that information is timely, accurate, and accessible. 

Conclusion

We conclude with a high level of assurance that although key aspects of a management control framework over unstructured electronic record keeping are in place within the Secretariat, a number of significant improvements are necessary to fully ensure the provision of relevant, timely and accessible electronic information to support decision making and general IM practices.

Recommendations

The following recommendations are directed to the Secretariat's departmental Chief Information Officer (CIO), in relation to the management of unstructured electronic information. While the focus of our audit was electronic record keeping in general, these recommendations could be applied to IM as a whole.

1. The CIO should define performance expectations and performance measures for IM strategies and operational activities and should ensure periodic reviews and reports on performance results (including compliance) against these expectations;
2. The CIO should define monitoring and reporting roles and responsibilities for IM in order to meet the needs of the Secretariat and to ensure that IM strategies and goals are met. This should be done by leveraging the knowledge of sectors and defining their responsibilities for IM, while respecting the holistic IM stewardship responsibilities of the Enterprise Information Management Services (EIMS) group within the Secretariat. Once defined, these roles and responsibilities should be approved by the Secretariat's governance committees to ensure acceptance;
3. The CIO should develop an inventory of existing IM practices and should identify key practices that may be transferable or applicable to the Secretariat as a whole. EIMS should develop department-wide IM practices and tools based on these key practices, as appropriate, and should ensure that existing sector capabilities are leveraged to support their development and implementation;
4. The CIO, in conjunction with sectors, should perform a gap analysis of the IM life cycle of electronic records to ensure that consistent IM life-cycle practices are in place across the Secretariat and information repositories, ensuring that:
   • Information needs and processes are defined, documented, and periodically reviewed for all of the Secretariat's user groups; and
   • Policies and training are updated to support the consistent application of these practices and to meet the needs of the Secretariat's users.

A management response and action plan has been developed by the Secretariat and is presented in Appendix D.

1.0 Introduction

The Internal Audit and Evaluation Bureau (IAEB) has completed an audit of electronic record keeping for the Treasury Board of Canada Secretariat (the Secretariat) as part of a broader horizontal audit initiated by the Office of the Comptroller General (OCG). 

This report relates specifically to the Secretariat as a large department. While OCG developed the audit program, IAEB conducted both the detailed examination phase and the supplementary audit procedures in order to produce a stand-alone audit report for the Secretariat.

1.1 Operating Environment

Information is an essential component of effective management in business. The availability of high-quality, authoritative information to decision makers supports the delivery of programs and services, thus enabling management and staff to be more responsive and accountable.

The Government of Canada is no exception to this theme. Across government, the management of information impacts all lines of business and is a key component in achieving its objectives. Information is being created throughout government at a rapidly increasing rate. Without the capability to effectively manage this information, departments may be at risk of losing their capability to identify and retrieve information in an organized and timely manner.

Within the context of the Government of Canada, the information life cycle [1] (see Appendix A) has been defined as follows:

• Plan: Users must determine their information needs in order to accomplish their objectives and plan accordingly.
• Create and collect:As users create and collect information, they need to identify its value to the organization and manage it accordingly, ensuring that the information is accessible to those who need it. 
• Organize: Users should organize their information logically and systematically in order to facilitate search and retrieval.
• Reuse and share: Once information is organized, users will be able to find and reuse information and leverage its usefulness by sharing it with others, reducing duplication of effort and improving service delivery.
• Maintain and protect: Protecting information involves not only guarding against unauthorized access, disclosure or destruction, but also preserving its integrity and authenticity.
• Transfer or destroy: While some information needs to be kept long-term to support an institution's operational needs or to preserve information of enduring value, other information can be disposed of when it has outlived its usefulness.

This life cycle is applicable to all types of information, including paper and electronic records, and should be consistently applied irrespective of type.

1.2  Secretariat Operations

As the administrative arm of the Treasury Board, the Secretariat has a dual mandate: to support the Treasury Board as a committee of ministers and to fulfill the statutory responsibilities of a central government agency. The Secretariat is tasked with providing advice and support to Treasury Board ministers in their role of ensuring value for money as well as providing oversight of the financial management functions in departments and agencies. To this end, the Secretariat makes recommendations and provides advice to the Treasury Board on policies, directives, regulations, and program expenditure proposals with respect to the management of the government's resources. Its responsibilities for the general management of the government affect initiatives, issues, and activities that cut across all policy sectors managed by federal departments and organizational entities and result in the production, review, and sharing of large quantities of government information having varying degrees of information sensitivity to support these functions.

While the ultimate responsibility for corporate information and records rests with the creator of that information, the departmental responsibility to oversee IM activities in line with Government of Canada policies resides in the Corporate Services Sector (CSS) of the Information Management and Technology Directorate (IMTD). The responsible group in this directorate is Enterprise Information Management Services (EIMS) of the Client Service Delivery Division. At the time of our audit, forecast resources for this group relative to the Secretariat's overall resources were as follows:

Fiscal Year	2011–12		2012–13		2013–14
Sector/Department	EIMS	Secretariat	EIMS	Secretariat	EIMS	Secretariat
($ thousands)
FTEs	23.39	2,216	23.76	1,633	23.7	1,570
Total	1,709	2,701,592	1,728	2,589,375	1,738	2,572,750

2.0 Audit Details      

2.1 Objectives and Scope

The objective of the audit was to provide assurance that the management control framework over electronic recording keeping is in place and provides relevant, timely and accessible information to support decision making at the departmental level.

It was acknowledged that most information created by the Government of Canada is electronic and can be categorized into the following two types:

• Structured data are generated from enterprise systems, such as SAP or PeopleSoft. Numerous controls exist to manage the risks associated with this type of information.
• Unstructured data include information in working documents, such as project plans, spreadsheets, emails, and records of decision. The management of this information faces similar risks, yet the controls are frequently much less structured, often ad hoc in nature.

For the purposes this audit, the scope was limited to unstructured data, with an emphasis on electronic record keeping. In addition, because management of electronic data is not considered to be a discrete IM activity, observations and findings that refer to IM also apply to electronic record keeping unless otherwise noted.

The scope of IAEB's work was limited to the Secretariat as a large department. Work was carried out by IAEB from February 2011 to July 2011 and covered activity during the 2010–11 and 2011–12 fiscal years. 

Scope Exclusion

Although the Secretariat has a central agency role relative to IM, this role is covered by the OCG's horizontal audit and was excluded from IAEB's examination. 

It should be noted that on August 8, 2011, the Government of Canada announced the creation of Shared Services Canada, to streamline information technology (IT) within the federal government.Its role will be to consolidate email, data centre, and network services across the public service. While this new initiative does not impact the findings and observations of this audit, it is recognized that it will impact the implementation of actions related to the electronic infrastructure and architecture upon which electronic record keeping rests.

2.2 Criteria

The audit of electronic record keeping included the following five lines of enquiry:

• Policy and Governance;
• People and Capacity;
• Enterprise and Information Architecture;
• Information Management Tools and Applications; and
• Information Management and Service Delivery.

The criteria for the audit were drawn from the Treasury Board Policy on Information Management and its supporting directives. Detailed audit criteria for each of these lines of enquiry are presented in Appendix B.

2.3 Approach and Methodology

The audit approach and methodology followed the Internal Auditing Standards for the Government of Canada and The Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that audit objectives are achieved. For this audit, planning was initiated by OCG in consultation with large departments' and agencies' internal audit functions.

The audit consisted of interviews with Secretariat staff, documentation reviews, and detailed testing and analysis of administrative data. We interviewed a sample of staff members to gain an understanding of IM practices within the Secretariat, focusing on electronic record keeping. We also reviewed documents to validate our findings. OCG provided an audit work plan and program for the first phase of the examination. This audit work plan was supplemented by additional audit work carried out by IAEB staff during the second phase, to provide further assurance in selected areas. Additional interviews and a documentation review were also conducted. Where merited, audit testing beyond OCG requirements was conducted to support the development of a stand-alone audit report for the Secretariat.

In conducting this audit, we also noted some strong IM and electronic record-keeping practices within the Secretariat. These are presented for information purposes in Appendix C.

3.0 Audit Results     

Each area of focus was assessed against audit objectives and related audit criteria. The audit results are presented below by line of enquiry.

3.1 Policy and Governance

As part of the audit, we examined the Secretariat's policy and governance capacity relative to electronic record keeping. We expected to find that governance structures were in place to effectively support an IM strategy and IM outcomes, in particular electronic record keeping. Specifically, we expected that:

• Governance structures, mechanisms, and resources were in place to ensure continuous and effective management of information; and
• Monitoring and reporting processes were in place.

Overall, we found that a governance framework was in place that defined IM roles and responsibilities. While monitoring and reporting of IM activities were occurring, improvements could be made to ensure effective management. 

We found that an IM governance structure was in place to support electronic record keeping within the Secretariat. The Secretariat's Management and Infrastructure Committee, comprising senior management at the Assistant Secretary level, has a subcommittee dedicated to IM and IT. Responsibility for overseeing IM within the Secretariat is assigned to the EIMS group within IMTD.

Roles and responsibilities for IM were defined and communicated as follows:

• Organization and committee charts which demonstrate reporting relationships;
• Committee terms of reference which defined the scope of committee activities and included oversight of IM activities;  and
• Departmental guidelines and policies which reflect these roles and responsibilities. and are updated as necessary.

In line with the periodic review requirements of the Policy on Information Management, the Secretariat was in the process of updating its IM strategy at the time of our audit. High-level IM strategies were presented and approved through the governance structure by the Secretariat's  Management and Infrastructure Committee. This update included a review of the operating environment and consultation with senior internal stakeholders. In addition, a plan to support the implementation of these strategies was under development at the time of our audit. We noted that electronic record keeping was treated not as a discrete activity but as part of the broader need for managing information. The current strategy and plan were articulated when IMTD was part of the Department of Finance Canada in 2008–09. We noted that the timing and the frequency of review complies with the Policy on Information Management.

In reviewing the IM planning process, we were told that planning is largely based on historical activities rather than on forecasted needs. In addition, many IM services, including activities related to electronic record keeping, were delivered in response to ad hoc requests from sectors rather than planned activities arising from a formal mechanism that identifies IM needs from sectors in advance.     

A related issue is the absence of defined measures of performance. While Departmental Performance Reports report on IM activities, there are no defined performance expectations and measures for IM strategies or activities. For example, EIMS carries out IM training activities, but the intended outcomes have not been defined nor have performance measures been identified. With defined desired outcomes and related performance measures, the Secretariat would be better positioned to determine whether intended outcomes were being achieved.

In setting performance expectations, the department should distinguish between performance measures for ongoing activities and those for project-driven activities with end dates (such as those associated with activities undertaken to implement a strategy). Ongoing activities require measures that indicate success and are largely aimed at defining satisfactory performance on an ongoing basis. Examples may include numerical targets and quotas, such as records created, user statistics, and error rates. While this is also true for project-driven activities, clear, phased deliverables should also be defined, with anticipated completion dates that reflect the sequential nature of project life cycles. This would affect many of the activities that involve the creation, development, and implementation of processes.

We found that monitoring of IM activities was done as part of regular departmental reporting mechanisms, but performance information largely pertained to activities undertaken during the reporting period and focused on outputs without reporting on outcomes. Also, we were informed that there was no formal monitoring of compliance with IM practices and therefore no reporting on monitoring. 

Without clear performance expectations and performance measures, the Secretariat may find it difficult to measure either its progress toward or its success in achieving intended objectives.

In addition, without some form of monitoring, the Secretariat cannot determine whether its practices comply with its policies or whether these policies are sufficient. 

Recommendation 1

The CIO should define outcome performance expectations and performance measures for IM strategies and operational activities and should ensure periodic reviews and reports on performance results (including compliance) against these expectations. 

Recommendation 2

The CIO should define monitoring and reporting roles and responsibilities for IM in order to meet the needs of the Secretariat and to ensure that IM strategies and goals are met. This should be done by leveraging the knowledge of sectors and defining their responsibilities for IM, while respecting the holistic IM stewardship responsibilities of the EIMS group within the Secretariat. Once defined, these roles and responsibilities should be approved by the Secretariat's governance committees to ensure acceptance.

3.2 People and Capacity

We examined how the Secretariat developed people and capacity to support its electronic records management activities. We expected to find that the Secretariat was developing highly skilled workforces to ensure that capacity exists to deliver IM outcomes. Specifically, we expected that the Secretariat:

• Had a common body of knowledge, learning and assessment tools; and
• Had a common understanding of common policy instruments and assessment tools.

We found that the Secretariat used various processes to support the development of highly skilled workforces that supported sound electronic IM practices. However, available learning resources have not been universally leveraged throughout the Secretariat, and opportunities exist to improve planning for these activities.

As previously stated, EIMS is the group within IMTD that is responsible for overseeing IM activities, including those pertaining to electronic records management, within the Secretariat. Its role in IM service delivery includes internal development of policy instruments, training and development, and promotion of IM in order to build awareness of the importance of IM. EIMS informed us that they are reorganizing, and transitioning from an organization of specialists in paper-based records management to one that is better able to support electronic record keeping and overall IM stewardship functions. 

In reviewing EIMS activities, we assessed the delivery of its services by interviewing staff and reviewing IMTD performance reports, training statistics, and training schedules that were available at the time of our review. We noted that IM practitioners regularly take IM courses to maintain and expand their knowledge of IM practices as required under the Policy on Information Management.

There was a uniform understanding of the importance of IM across the Secretariat. We found that standardized training formats and general IM policy/guidance documents were in place to support Secretariat staff's understanding of IM. Furthermore, the Secretariat's orientation courses include brief discussions of IM concepts, and all staff members receive a security briefing upon arrival that includes elements of IM. Although training was available, there was no mandatory IM and electronic record-keeping training. As a result, IM understanding varies across the Secretariat, since staff members are not receiving consistent and mandatory training. 

The issue concerning the development of skilled workforces is discussed further in section 3.5 of this report and in the associated recommendation.

3.3    Enterprise and Information Architecture

As part of the audit, we examined the manner in which the Secretariat has been developing enterprise and information architecture. We expected to find that the Secretariat was developing information architecture and processes that respected their IM risks and controls, and operational requirements. Specifically, we expected that:

• Information and records would be identified and managed as valuable assets to support the outcomes of programs and services, as well as operational needs and accountabilities; and
• The Secretariat's programs and services would provide convenient access to relevant, reliable, comprehensive and timely information.

For enterprise and information architecture, we found that the Secretariat has not been consistently developing information architecture and processes aligned with their IM risks and controls, and operational requirements. Practices vary by sector and user group.

At the corporate level, we found that the Secretariat has periodically performed corporate risk assessments and has established a Corporate Risk Profile. The risk areas identified included IM. The Secretariat' risk methodology included an assessment of impact and likelihood, as well as the development of mitigation strategies to ensure that risk areas are addressed. However, our review found that mitigation strategies to address identified IM risk areas tended to be high-level and long-term, which might expose the Secretariat to identified risks in the short-term.

We were informed that sectors did not regularly analyze business processes with a goal of identifying IM needs. Instead, sectors employ tools (e.g., electronic systems) available within the Secretariat to manage information and build their processes to these systems. Staff indicated that only when a need for a new electronic system was identified would they analyze their processes and consult with EIMS staff.

EIMS staff works with client sector staff to develop tools and IM processes that support the sector's business activities. In particular, we were told that this primarily results in the development of file plans. Although we confirmed that all sectors had established file plans, sector staff expressed differing views on the accuracy of these plans.

There is no set schedule for the review of sector file plans, but it is incumbent on client groups to identify issues and bring them to the attention of EIMS. A regular review process would help ensure that file plans remain accurate and support users' needs. 

Although we found that general guidelines are available to support the development of naming conventions and that sharing of practices and consultations across sectors are occurring, sectors are at different stages of implementing independent IM practices. 

We found that various generic documents exist within the Secretariat to help with the identification of information of business value and choice of repository. However, few documents had been developed to provide details on specific record-keeping practices at the Secretariat level. Rather, they were at the sector or division level. In many cases, this often resulted from sectors identifying a need for expanded guidance on IM practices and the assignment of IM roles and responsibilities for this task within their sector through sector IM working groups or IM champions.

Nonetheless, the Secretariat staff expressed differing views about the consistency as well as the clarity of IM practices across the Secretariat. Department-wide tools and applications are further discussed in section 3.4 of this report. 

3.4    Information Management Tools and Applications

We examined the extent to which the Secretariat developed and implemented IM tools and applications to support its electronic record-keeping practices. We expected to find that IM tools were developed and implemented that respect appropriate control requirements of the Department and of the business users. Specifically, we expected that the Secretariat had developed and implemented common and enterprise-wide tools and applications.

Overall, we found that the extent to which key methodologies, mechanisms, and tools were established and implemented to support departmental record keeping throughout the Secretariat varied by sector, with few consistent practices across the department.

Within the Secretariat, the main repository for unstructured corporate electronic information is the Records, Document and Information Management System (RDIMS), which was the government-wide solution for electronic record keeping, and implemented in the Secretariat in 2000. However, Secretariat staff has other options for information storage. In addition to RDIMS, repositories include shared drives, personal drives, Microsoft Outlook, paper files and the Corporate Information Centre (CIC), which is a centralized records office in the Secretariat.

Guidance documents related to IM and records management indicated a clear preference within the Secretariat for storage in RDIMS. In addition, staff members are aware of this preference. However, RDIMS use is not mandatory department-wide. 

EIMS estimates that a significant portion of unstructured information is saved in repositories other than RDIMS, which was confirmed through our audit testing. 

We identified several possibilities for the low adoption rate of RDIMS, including:

• User acceptance: Proficiency and comfort with the system were identified by staff as possible barriers to its use because staff were not comfortable searching the repository for information and/or were unfamiliar with the system to accurately save the corporate information;
• Reliability: It was expressed by multiple staff members that RDIMS was not user friendly, was antiquated, and did not meet their needs, or a combination of these;
• Understanding of the aspects of IM: General policies and guidelines define business value as information records that have enduring value. However, in practice, staff members are confused about the difference between transitional and corporate information;
• Alternative options: Because of the availability of other repositories, users may have deliberately chosen to use these other preferred repositories;
• System limitations: Incompatibility of RDIMS with some software in use within the Secretariat means that users are unable to store information in the repository;
• Network limitations: The security certification of the network does not support storage of information above a certain security classification.

Although anecdotal evidence suggests duplication of information across repositories is occurring, the extent to which these information resources (outside RDIMS) have a corporate value is unclear. EIMS has indicated that other than conducting a manual review, it cannot estimate the extent of duplication based on the existing tools. The risks arising from the potential duplication may need to be considered and managed. These risks include:

• Accessibility of information:  If information is duplicated across repositories, there is a risk that information of business value may go missing and/or that incomplete or inaccurate information may be used in decision making. 
• Increased costs: If information is duplicated across repositories, there are costs associated with acquiring additional storage and maintaining this additional storage, as well as costs associated with increased workloads to support expanded search, retrieval, and review of information for activities such as access to information requests.

As previously stated, staff expressed differing views about the accuracy of their file plans and the extent to which they support their business needs. The RDIMS credibility issues identified above may influence this view.  

While the establishment of file plans and naming conventions is a positive step, the absence of department-wide practices in this area may create barriers to information sharing across the Secretariat, as staff in different sectors may be unfamiliar with how to store or locate information in another sector and therefore may misfile or be unable to locate required information. Furthermore, it may also encourage additional information duplication, as users store additional copies or versions of information already in the corporate repository or elsewhere for their individual use or group use.

In terms of responsibilities for the various repositories:

• EIMS is responsible for the overall maintenance and support of all repositories and for the IM life-cycle management of RDIMS records and records retained centrally by CIC;
• Sectors, directorates and individuals, or a combination of these, are responsible for maintaining their records retained in repositories outside RDIMS, such as shared drives, personal drives, Outlook and paper files. However, EIMS retains overall responsibility for these repositories.

Limited guidance was found on the differences in use of the various repositories, and we found that formal monitoring or reporting on compliance with existing guidance was not occurring. 

Although we did not find common enterprise-wide IM practices in place within the Secretariat during our review, we noted some examples of strong or leading IM practices. These are presented in Appendix C.

Recommendation 3

The CIO should develop an inventory of existing IM practices and should identify key practices that may be transferable or applicable to the Secretariat as a whole. EIMS should develop department-wide IM practices and tools based on these key practices, as appropriate, and should ensure that existing sector capabilities are leveraged to support their development and implementation.

3.5 Information Management and Service Delivery

Finally, we examined the manner in which the Secretariat delivered IM and services in support of its operations. We expected that record-keeping practices would ensure the provision of timely, accurate, and accessible information, in support of the delivery of the Secretariat's programs and services. Specifically, we expected that:

• All information would be managed to ensure the relevance, authenticity, quality, and cost-effectiveness of the information for as long as it is required to meet operational needs and accountabilities; and
• The Secretariat programs and services would integrate IM requirements into development, implementation and reporting activities.

Overall, we found that record-keeping practices (especially electronic) were not consistently in place to ensure timely, accurate and accessible information. However, EIMS is aware of IM practice weaknesses and is working on implementing its revised IM strategy to improve electronic records management.

As previously discussed, we were told that EIMS staff work with sectors to develop a file plan (including electronic) to support sector activities. However, our testing found that instead of being based on sector activities, file plans were based on organizational structures. From an organizational perspective, activities have more permanence than organizational structures; this may be a factor in why staff members do not believe that file plans support their work. EIMS indicated that retention periods have been set for all Secretariat information resources. Our audit testing found that all sectors included in this audit had retention schedules, most of which had been set, with the remaining under review. EIMS staff also told us that the Secretariat applies the Retention Disposition Authorities (RDA) or the Multi-institutional Disposition Authorities (MIDA) or the Institutional Specific Disposition Authorities (ISDA), and that retention periods for most information resources are established. 

We found that a disposition process has been developed and implemented for records retained centrally by CIC (i.e., paper records). The extent to which a designed and implemented disposition process was in place depends on the repository in which the information is held, as well as the sector, directorate, division, and user. RDIMS is a central repository; however, it was found that a disposition process has not yet been defined and implemented owing to system limitations. Within the Secretariat, users also have the ability to store information in a variety of repositories (mainly electronic). Our audit interviews found that while some sectors had a defined process to transfer or dispose of some types of information in these other repositories, the overall consensus was that a universally defined disposition process was not in place within the Secretariat. However, at the time of our audit, work was underway by EIMS to develop a risk-based disposition process for unstructured electronic records.

We reviewed RDIMS documentation to determine the extent to which electronic information was being retained longer than necessary in sectors included in the scope of our audit. Our testing found that the majority of information in RDIMS has not yet exceeded its retention periods. However, it is important to note that RDIMS was implemented in the Secretariat in the year 2000 and that a large majority of retention periods set for the various folders of sectors included in the scope of our audit have been set for a duration longer than the time which has elapsed since RDIMS' implementation. Therefore, if no disposition process is implemented for unstructured electronic data, this finding will likely significantly change and pose risks in the short-term. 

Inconsistent disposition processes, coupled with the potential duplication previously mentioned, may lead the Secretariat to dispose of one version of a record, but other copies may be retained. This risk will increase if formal disposition processes for the repositories are not consistently rolled out, since the repositories often work in coordination. This could further complicate information storage and retrieval.

From a departmental standpoint, we found that IM requirements are addressed during departmental strategic planning. In setting the revised IM strategic plan, we found that the Secretariat carried out a review of the operating environment, including consultation at senior levels. This was done to guide the development of an IM strategy for the Secretariat. We note that this revised strategy recognizes some of the issues raised in this report and that some work has begun to strengthen IM practices in the Secretariat.

At the operational level, we found that IM requirements are considered by program staff at the time of system design, but not consistently in program process design/review. We also found that few sectors had retained documentation on their information requirements or were aware whether these were ever documented. As program information needs evolve, staff should consider IM impacts and required changes within existing systems and processes on an ongoing basis to meet business needs. Instead, through our interviews with staff, we found that IM considerations are only considered when building or acquiring a system. 

Much of the work of the Government of Canada is dependant on managing information, and the Secretariat is no exception to this concept. Therefore, it is crucial that information needs drive the development of business processes. While technology frequently enables innovation and processes, IM activities should be guided by current business needs. Also, if business processes are not documented, it is difficult to periodically review them for required adjustments and improvements.

Recommendation 4

The CIO, in conjunction with sectors, should perform a gap analysis of the IM life cycle of electronic records to ensure that consistent IM life-cycle practices are in place across the Secretariat and information repositories, ensuring that:

• Information needs and processes are defined, documented, and periodically reviewed for all of the Secretariat's user groups; and
• Policies and training are updated to support the consistent application of these practices and to meet the needs of the Secretariat's users

3.6 Overall Conclusion

We conclude with a high level of assurance that although key aspects of a management control framework over unstructured electronic record keeping are in place within the Secretariat, a number of significant improvements are necessary in order to fully ensure the provision of relevant, timely and accessible electronic information to support decision making and general IM practices. The Secretariat was in the process of implementing its revised IM strategy to improve electronic records management at the time of our audit. However, some further improvements are necessary. Specifically:

• Policy and Governance:  A governance framework is in place within the Secretariat that defines IM roles and responsibilities to support unstructured electronic record keeping at each level. Although key monitoring and reporting processes are in place, improvements to planning, performance measurement and compliance monitoring would further strengthen the governance framework.
• People and Capacity:  Some processes to support the development of highly skilled workforces are in place, but there is room for improvement. Specifically, available learning resources are not being universally leveraged by staff, and opportunities exist to improve planning for these activities.
• Enter prise and Information Architecture:  The Secretariat has not consistently been developing information architecture and processes that respect IM risks, controls, and operational requirements. Practices vary by sector and user group.
• Information Management Tools and Applications: The extent to which key methodologies, mechanisms, and tools have been established and implemented to support departmental record-keeping throughout the Secretariat varies by sector, with few consistent practices department-wide.
• Information Management and Service Delivery: Record-keeping practices have not been fully implemented to ensure that information is timely, accurate, and accessible.

Appendix A: Information Life Cycle

Information Life Cycle – Text Version

Note: This graphic, developed by the Internal Audit and Evaluation Bureau, is based on the Guideline for Employees of the Government of Canada: Information Management Basics, [2] under the mandate of the Chief Information Officer Branch of the Treasury Board of Canada Secretariat.

Appendix B: Audit Criteria

The objective of this audit was to provide assurance that the management control framework over information management is in place and provides relevant, timely and accessible information to support decision making. The following table provides the detailed criteria and subcriteria by line of enquiry, as identified by the Office of the Comptroller General and adopted for this report accordingly. 

• Policy and Governance -Departments have governance structures in place to effectively support an information management strategy and information management outcomes.
  • 1.1  Governance structures, mechanisms and resources are in place to ensure the continuous and effective management of information.
  • 1.2  Monitoring and reporting processes are in place for information management.
• Line of Enquiry 2:People and Capacity - Departments are developing highly skilled workforces to ensure that capacity exists to deliver information management outcomes.
  • 2.1 Departments have a common body of knowledge, learning and assessment tools. Monitoring and reporting processes are in place for information management.
  • 2.2 Departments have a common understanding of common policy instruments and assessment tools.
• Line of Enquiry 3:Enterprise and Information Architecture – Departments are developing information architecture and processes that respect their information management risks and controls, and operational requirements
  • 3.1 Information and records are identified and managed as valuable assets to support the outcomes of programs and services, as well as operational needs and accountabilities.
  • 3.2 Government programs and services provide convenient access to relevant, reliable, comprehensive and timely information.
• Line of Enquiry 4:Information Management Tools and Applications – Information management tools are developed and implemented that respect appropriate control requirements of the department and of the business users, and are compliant with the information architecture within and across departments.
  • 4.1  Departments develop and implement common and enterprise-wide tools and applications.
• Line of Enquiry 5:Information Management and Service delivery – Record-keeping practices ensure that information is timely, accurate, and accessible for departments in the delivery of Government of Canada programs and services.
  • 5.1 All information is managed to ensure the relevance, authenticity, quality, and cost-effectiveness of the information for as long as it is required to meet operational needs and accoutabilities.
  • 5.2 Departmental programs and services integrate information management requirements into development, implementation, evaluation, and reporting activities.

Appendix C: List of Leading Information Management Practices

In conducting this audit, several leading or strong practices in support of information management (IM) activities were noted. These were identified through a combination of interviews and documentation reviews and were then validated with the Enterprise Information Management Services group of the Information Management and Technology Directorate. 

This is not a comprehensive list of practices within the Secretariat, and we acknowledge that other leading or strong practices may exist in the sectors included in this audit, as well as other parts of the Secretariat.

Summary	MAF Element	Description and Analysis
1. Champion, Lead IM Representative, or IM Working Group	Accountabilities Learning, Innovation and Change Management	Several sectors/divisions have begun to define/delegate specific IM responsibilities within their organizations. In so doing, these organizations have tended to develop stronger practices. The delegated individuals were frequently asked to lead, develop and/or implement new IM tools or processes, with the goal of improving IM practices in the sector. IM may be able to leverage these tools and practices to encourage further dissemination and/or development of IM practices as well as to support its monitoring and reporting needs.
2. Case Management System	Stewardship	We found that several groups within the Secretariat whose main activity was to provide review, advice and comments did so by developing an electronic system to track this, which they called a case management system. These systems were used to control the information sharing process and helped ensure that activities undertaken were consistent and documented. Organizations that had implemented case management systems frequently obtained senior management support and sought user engagement to develop a solution to facilitate their IM needs in order to better support their IM life-cycle practices. They: planned for their needs; identified a solution; implemented the solution; and developed and defined supporting tools (manuals, quick reference guides, etc.) to facilitate the use of IM practices. While our review found several types in use, and given that this is a common activity for the Secretariat's sectors, there may be some opportunities to expand the scope of these systems in other parts of the Secretariat.
3. Mandatory Training	Learning, Innovation and Change Management	Sectors and divisions that were more mature in their IM practices, relative to others in the Secretariat, had developed and implemented mandatory learning activities to ensure that all staff responsible for IM were aware of their responsibilities, tools and practices as well as how to use them. Furthermore, learning frequently accompanied an impending change process and therefore was employed to support change management activities.
4. IM Reference Materials	Stewardship	Within the Secretariat there are general reference materials that support good IM practices. These tools provide general guidance for all the Secretariat staff on IM-related topics, including expectations, roles and responsibilities, and guidelines on how to establish more specific IM practices. Sector/division-specific reference materials have been developed by several sectors and divisions and build on other IM guidance documents. In some cases, they serve to amalgamate and integrate multiple policy documents to provide a single point for users to locate all relevant information. They also provide more specific guidance on: Information storage to facilitate search and retrieval within the sector; and Documentation of the information workflows of the division.
5. Standardized naming convention	Stewardship	Several sectors have implemented or are developing standardized naming conventions. Use of a standardized naming convention allows and facilitates information storage, search and retrieval among users who follow these practices. A departmental standard would support greater information storage, search and retrieval across the department. Individual and independent processes may create artificial barriers to information storage, search and retrieval across user groups.
6. Policies and procedures for retirement and disposition	Stewardship Accountabilities	Different parts of the organization have instituted practices to dispose and transfer information resources from their sector. Disposition helps ensure costs for storage do not continue to increase and are in line with requirements. However, without departmental disposition activities being in place, the amount of information in the Secretariat's possession will continue to grow.

Appendix D: Management Response and Action Plan

Management Response

Corporate Services Sector / Information Management and Technology Directorate (CSS/IMTD) has developed an information management (IM) road map that will provide the necessary strategy, guidance and high-level planning to ensure that the IM role is carried out beneficially within the Treasury Board of Canada Secretariat (the Secretariat). This road map will encompass and embed the Management Action Plan activities of the audit of electronic record keeping, ensuring a comprehensive holistic approach to IM activities and stewardship at the Secretariat. The target completion for this project is three years.

Key Activities:

• IM Practices: Update IM practices to focus on electronic IM and to fully leverage the records management tools and software available to the Secretariat. A component of this key activity will be the conclusion of IM agreements with each Sector/Branch. These agreements will outline roles and responsibilities and identify both the information assets of business value and the required methods of managing these assets to enable the business. Discussions on concluding these agreements will determine needs and gaps and will help determine ongoing IM priorities.
• Enterprise Taxonomy: Consolidate the existing taxonomies and put an emphasis on functions (derived from the Secretariat's Program Activity Architecture) over subject to create a more evergreen product.
• Records Management Tools: Upgrade IM products by providing increased functionality and social media and Web 2.0 applications to IM users.
• Collaboration: Develop the Secretariat's corporate approach to software, business processes and IM.
• Electronic Information Flow: Integrate IM practices into business processes and workflows to lessen the burden on users and to increase the accuracy of records metadata while encouraging the use of electronic records. Limit the collection and storage of paper records to those deemed necessary to collect and manage in that format. 

Recommendation 1:

The Chief Information Officer (CIO) should define performance expectations and performance measures for IM strategies and operational activities, and should ensure periodic reviews and reports on performance results (including compliance) against these expectations.

Priority Ranking: High

Management Action

1. Recommendation is accepted and agreed to.
2. Performance metrics are now part of the overall IMTD governance structure.
   • IM expectations will be reviewed and defined; (Completion Date: Q4 2011–12)
   • Metrics will be established; and (Completion Date: Q4 2011–12)
   • Metrics will be measured and reported. (Completion Date: Q1 2012–13)
3. Determine objectives and targets for areas of improvement to be included in the TBS IM Road Map project. (Completion Date: Q1 2012–13)

Office of Primary Interest (OPI)

IMTD leads; Other TBS organizational units to participate

Recommendation 2:

The CIO should define monitoring and reporting roles and responsibilities for IM in order to meet the needs of the Secretariat and to ensure that IM strategies and goals are met. This should be done by leveraging the knowledge of sectors and defining their responsibilities for IM, while respecting the holistic IM stewardship responsibilities of the Enterprise Information Management Services (EIMS) group within the Secretariat. Once defined, these roles and responsibilities should be approved by the Secretariat's governance committees to ensure acceptance.

Priority Ranking: High

Management Action

1. Recommendation is accepted and agreed to.
2. As part of the Secretariat's IM Road Map Project, agreements on roles and responsibilities will be concluded with all Sectors/Branches. Additionally, the IM Road Map Project will define a life-cycle management plan for identified information assets and will include a time frame for cyclical reviews.
   • Obtain senior management approval to begin. (Completion Date: Q4 2011–12)
   • Work with Sectors and Branches to draft agreements. (Completion Date: Q3 2012–13)
   • Accept and sign off agreements. (Completion Date: Q4 2012–13)

Office of Primary Interest (OPI)

IMTD leads; Other TBS organizational units to participate

Recommendation 3:

The CIO should develop an inventory of existing IM practices and should identify key practices that may be transferable or applicable to the Secretariat as a whole. It is also recommended that EIMS should develop department-wide IM practices and tools based on these key practices, as appropriate, and should ensure that existing sector capabilities are leveraged to support their development and implementation.

Priority Ranking: Medium

Management Action

1. Recommendation is accepted and agreed to.
2. Document known processes, publish them on InfoSite, and communicate them to the department. (Completion Date: Posted by Q4 2011–12)
3. The IM Road Map Project will facilitate an information/inventory gathering exercise to determine whether other best practices exist and how to incorporate them into departmental strategy (Completion Date: Q4 2012–13)

Office of Primary Interest (OPI)

IMTD with some input from business managers

Recommendation 4:

The CIO, in conjunction with sectors, should perform a gap analysis of the IM life cycle of electronic records to ensure that consistent IM life-cycle practices are in place across the Secretariat and information repositories, ensuring that:

• Information needs and processes are defined, documented, and periodically reviewed for all of the Secretariat's user groups; and
• Policies and training are updated to support the consistent application of these practices and to meet the needs of the Secretariat's user.

Priority Ranking:High

Management Action

1. Recommendation is accepted and agreed to.
2. The IM Road Map Project will define the needs and processes to be captured and documented in the IM agreements. (Completion Date: Q4 2012–13)
3. As part of the IM Road Map Project, an Enterprise taxonomy will be developed with a cyclical, built-in review process for business line/functional activities. (Completion Date: Q2 2012–13)
4. IM training will include:
   • email management; (Completion Date: Q3 2011–12)
   • an RDIMS on-line training tool; and (Completion Date: Q4 2011–12)
   • general IM/record-keeping training. (Completion Date: Q3 2011–12)
5. Disposition strategy to eliminate legacy electronic repositories, e.g., shared drives, will be included as part of the IM Road Map Project. (Completion Date: Q4 2011–12)

Office of Primary Interest (OPI)

IMTD leads, with participation as required from the Secretariat

---

Footnotes

[1] Guideline for Employees of the Government of Canada: Information Management (IM) Basics

[2] Guideline for Employees of the Government of Canada: Information Management (IM) Basics