Rescinded [2016-07-01] - Policy on Evaluation

This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Archived

This page has been archived on the Web.

1. Effective Date

1.1 The policy takes effect on April 1, 2009. Departments have until March 31, 2013 to fully implement section 6.1.8 (a) of the policy.

1.2 This version of the policy incorporates updates effective April 1, 2012.

1.3 Transitional considerations:

1.3.1 Pending the full implementation of section 6.1.8 (a), which can be no later than March 31, 2013, deputy heads will ensure that:

  1. approved departmental evaluation plans demonstrate progress towards achieving coverage of all departmental direct program spending (excluding ongoing programs of grants and contributions) over five years; and
  2. departmental evaluation plans that they approve for submission to the Treasury Board of Canada Secretariat as per section 6.1.7 of this policy, and that do not demonstrate evaluation coverage of all direct program spending over the ensuing five-year period, use a risk-based approach to planning coverage of direct program spending (excluding ongoing programs of grants and contributions.) Departmental evaluation plans that use a risk-based approach to planning coverage of direct program spending will include a written, risk-based rationale to explain the department's evaluation coverage and non-coverage choices.

2. Application

2.1 This policy applies to departments as defined in section 2 of the Financial Administration Act, with the exception of the Office of the Governor-General's Secretary, and the staffs of the Senate, House of Commons, Library of Parliament, Office of the Senate Ethics Officer and Office of the Conflict of Interest and Ethics Commissioner.

2.2 Application of this policy to small departments and agencies as defined in Annex A is deferred until otherwise directed by the President of the Treasury Board, with the exception of section 6.2, which shall apply as of the effective date of the policy.

2.3 Sections 3.6, 7.3 and 8.2, as well as the provisions in paragraphs 6.1.8(d) and (e) and 6.2.5(b) that provide for the Secretary of the Treasury Board to request specific evaluation coverage, do not apply with respect to the Office of the Auditor General, the Office of the Privacy Commissioner, the Office of the Information Commissioner, the Office of the Chief Electoral Officer, the Office of the Commissioner of Lobbying, the Office of the Commissioner of Official Languages and the Office of the Public Sector Integrity Commissioner. The deputy heads of these organizations are solely responsible for monitoring and ensuring compliance with this policy within their organizations, as well as for responding to cases of non-compliance in accordance with any Treasury Board instruments that address the management of compliance.

3. Context

3.1 In the Government of Canada, evaluation is the systematic collection and analysis of evidence on the outcomes of programs to make judgments about their relevance, performance and alternative ways to deliver them or to achieve the same results.

3.2 Evaluation provides Canadians, Parliamentarians, Ministers, central agencies and deputy heads an evidence-based, neutral assessment of the value for money, i.e. relevance and performance, of federal government programs. Evaluation:

  1. supports accountability to Parliament and Canadians by helping the government to credibly report on the results achieved with resources invested in programs;
  2. informs government decisions on resource allocation and reallocation by:
    1. supporting strategic reviews of existing program spending, to help Ministers understand the ongoing relevance and performance of existing programs;
    2. providing objective information to help Ministers understand how new spending proposals fit with existing programs, identify synergies and avoid wasteful duplication;
  3. supports deputy heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to produce, at an affordable cost; and,
  4. supports policy and program improvements by helping to identify lessons learned and best practices.

3.3 This policy is issued pursuant to section 7 and section 42.1 of the Financial Administration Act.

3.4 The deputy head is responsible for the department's evaluation function.

3.5 The Treasury Board has delegated to the President of the Treasury Board the authority to issue, amend and rescind the Directive on the Evaluation Function and the Standard on Evaluation for the Government of Canada related to this policy; to approve any exception to the directive or standard; as well as to determine when the policy will apply to small departments and agencies.

3.6 The Treasury Board has assigned responsibility to the Secretary of the Treasury Board to require departments to undertake specific evaluations, as needed, following consultation with the affected deputy head.

3.7 This policy is to be read in conjunction with the Directive on the Evaluation Function, Standard on Evaluation for the Government of Canada, and the Financial Administration Act.

3.8 An evaluation conducted pursuant to this policy constitutes a review of relevance and effectiveness for the purposes of section 42.1 of the Financial Administration Act.

4. Definitions

Definitions used in the interpretation of this policy can be found in Annex A.

5. Policy Statement

5.1 Objective

The objective of this policy is to create a comprehensive and reliable base of evaluation evidence that is used to support policy and program improvement, expenditure management, Cabinet decision making, and public reporting.

5.2 Expected Results

Credible, timely and neutral information on the ongoing relevance and performance of direct program spending is:

  1. available to Ministers, central agencies and deputy heads and used to support evidence-based decision making on policy, expenditure management and program improvements; and,
  2. available to Parliament and Canadians to support government accountability for results achieved by policies and programs.

6. Policy Requirements

6.1 Deputy heads are responsible for establishing a robust, neutral evaluation function in their department and for ensuring that their department adheres to this policy and to its supporting directive and standard. Deputy heads:

6.1.1 designate a head of evaluation at an appropriate level as the lead for the evaluation function in the department;

6.1.2 ensure that the head of evaluation has direct and unencumbered access to the deputy head, as required;

6.1.3 ensure that a committee of senior departmental officials (referred to as the Departmental Evaluation Committee) is assigned the responsibility for advising the deputy head on all evaluation and evaluation-related activities of the department and is:

  1. chaired by the deputy head or a senior level designate;
  2. supported by the head of evaluation on evaluation matters; and,
  3. structured with specific roles and responsibilities, as per Annex B.

6.1.4 approve evaluation reports, management responses and action plans in a timely manner;

6.1.5 use evaluation findings to inform program, policy, resource allocation and reallocation decisions;

6.1.6 ensure that complete, approved evaluation reports along with management responses and action plans are made easily available to Canadians in a timely manner while ensuring that the sharing of reports respects the Access to Information Act, Privacy Act, and the Government Security Policy;

6.1.7 approve for annual submission to the Treasury Board of Canada Secretariat, a rolling five-year departmental evaluation plan (hereafter referred to as the departmental evaluation plan) and confirm that the departmental evaluation plan:

  1. aligns with and supports the departmental Management, Resources and Results Structure;
  2. supports the requirements of the Expenditure Management System, including strategic reviews; and,
  3. includes all ongoing programs of grants and contributions as required by section 42.1 of the Financial Administration Act.

6.1.8 ensure that the following evaluation coverage requirements are met and reflected in the departmental evaluation plan:

  1. all direct program spending, excluding grants and contributions, is evaluated every five years;
  2. all ongoing programs of grants and contributions are evaluated every five years, as required by section 42.1 of the Financial Administration Act;
  3. the administrative aspect of major statutory spending is evaluated every five years;
  4. programs that are set to terminate automatically over a specified period of time, if requested by the Secretary of the Treasury Board following consultation with the affected deputy head;
  5. specific evaluations, if requested by the Secretary of the Treasury Board following consultation with the affected deputy head.

6.1.9 ensure that the departmental evaluation committee and the head of evaluation have full access to information and documentation needed or requested to fulfill their responsibilities;

6.1.10 ensure that ongoing performance measurement is implemented throughout the department so that sufficient performance information is available to effectively support the evaluation of programs; and,

6.1.11 ensure that the head of evaluation is consulted on the performance measurement strategies embedded in the department's Management, Resources and Results Structure.

6.2 Deputy heads of small departments and agencies are responsible for:

6.2.1 designating a head of evaluation at an appropriate level as the lead for the evaluation function in the department;

6.2.2 ensuring that the head of evaluation has direct and unencumbered access to the deputy head, as required;

6.2.3 approving evaluation reports, management responses and action plans in a timely manner;

6.2.4 ensuring that complete, approved evaluation reports along with management responses and action plans are made easily available to Canadians in a timely manner while ensuring that the sharing of reports respects the Access to Information Act, Privacy Act, and the Government Security Policy;

6.2.5 ensuring that the following evaluation coverage requirements are met:

  1. all ongoing programs of grants and contributions are evaluated every five years, as required by section 42.1 of the Financial Administration Act; and,
  2. direct program spending, excluding grants and contributions, and the administrative aspect of major statutory spending are evaluated as appropriate to the needs of the department or agency, and specific evaluations are undertaken if requested by the Secretary of the Treasury Board following consultation with the affected deputy head.

6.3 Responsibilities of the Secretary of the Treasury Board of Canada

6.3.1 The Secretary of the Treasury Board is responsible for providing functional leadership for evaluation across government, including monitoring and reporting annually to the Treasury Board on government-wide evaluation priorities and the health of the evaluation function.

7. Monitoring and Reporting Requirements

7.1 Deputy heads are responsible for monitoring compliance with this policy in their departments to ensure its effective implementation.  They are responsible for ensuring that a neutral assessment of their departmental evaluation function is conducted at a minimum of once every five years.

7.2 Deputy heads are responsible for addressing issues that arise regarding compliance with this policy, and with its associated directive and standard, and ensuring that appropriate remedial actions are taken to address these issues.

7.3 The Secretary of the Treasury Board will monitor compliance with this policy through ongoing monitoring of evaluations and departmental evaluation plans, including evaluation coverage and quality of evaluations, and may require deputy heads to provide information on the following: 

7.3.1 the use of evaluation in support of the requirements of the Expenditure Management System, including the extent to which evaluation information is used within the department:

  1. to manage for results and inform resource reallocation decisions;
  2. to underpin proposals to Cabinet, including Treasury Board submissions and memoranda to Cabinet;
  3. to provide evidence for use in strategic reviews of program spending;
  4. to support accountability to Parliament, including through public reporting.

7.3.2 the quality and timeliness of evaluations;

7.3.3 actual evaluation coverage of direct program spending and the administrative aspect of major statutory spending;

7.3.4 actual evaluation coverage of ongoing programs of grants and contributions; and,

7.3.5 the capacity and competence of evaluation units. 

7.4 The Secretary of the Treasury Board will ensure that an evaluation of this policy is conducted every five years.

8. Consequences

8.1 Consequences of non-compliance with this policy can include any measure allowed by the Financial Administration Act that the Treasury Board would determine as appropriate in the circumstances.

8.2 If the Secretary of the Treasury Board determines that a department has not complied with the requirements of this policy or its supporting directive or standard, the Secretary of the Treasury Board may request that the deputy head take corrective actions and report back on the results achieved.

9. Roles and responsibilities of government organizations

Note:  This section identifies other significant departments that have a role in the Policy on Evaluation. In and of itself, it does not confer an authority.

9.1 The Treasury Board of Canada Secretariat is the government's main source of information on evaluation and its use in informing policy and expenditure management decision-making, including program improvement.  It is responsible for providing leadership, advice and guidance in the conduct, use and advancement of evaluation practices across the federal government.

10. References

10.1 Relevant Legislation

10.2 Related Policies or Publications

11. Enquiries

Please send any questions about this policy to:

Assistant Secretary, Expenditure Management Sector
Treasury Board of Canada Secretariat
222 Nepean Street, 3rd Floor
Ottawa, Ontario K1A 0R5

Telephone: 613-946-3061
Fax:  613-946-3718

Annex A: Policy Definitions

For the purposes of this policy, the following definitions are provided:

administrative aspect of major statutory spending:
The resources (both human and financial) used to deliver and administer major statutory spending
Departmental Evaluation Plan:
is a clear and concise framework that establishes the department's evaluations to be undertaken over a five-year period, in accordance with the Policy on Evaluation and supporting directive and standard. 
direct program spending:
refers to the portion of total budgetary spending that excludes public debt charges and major transfers to persons and to other levels of government.  Direct program spending includes operating and capital spending and grants and contributions (as specified in the Public Accounts).
economy:
minimizing the use of resources.  Economy is achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes.
effectiveness:
the extent to which a program is achieving expected outcomes.
efficiency:
the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output. The level of input and output could be increases or decreases in quantity, quality, or both.
evaluation products:
any output of the departmental evaluation function, which may include, but is not limited to, the departmental evaluation plan, terms of reference for individual evaluations, evaluability assessments, evaluation frameworks, evaluation reports, and advice.
Expenditure Management System (EMS):
is the framework that guides decisions on spending. It supports responsible and effective spending by the government through the identification and implementation of the Government's spending plans to achieve its priorities within the fiscal limits established by the Budget. The EMS governs how money is spent, how much, on what and for what purpose. The overarching objectives of the government's approach to expenditure management is to ensure that: aggregate fiscal discipline is maintained (fiscal credibility); all government resources are allocated to the Government's priorities (management excellence in the allocation of resources); and programs are operationally efficient and effective (management excellence in the delivery of programs and services).
head of evaluation:
is the official who is responsible for leading the evaluation function in a department.
major statutory spending:
refers to the portion of total budgetary spending that includes public debt charges, major transfers to individuals (e.g. elderly benefits, employment insurance benefits, children's benefits), and major transfers to other levels of government (e.g. Federal transfers in support of health and other programs, fiscal arrangements, alternative payments for standing programs).
neutral:
is an attribute required of the evaluation function and evaluators that is characterized by impartiality in behaviour and process. In carrying out their evaluation responsibilities evaluators do not allow official, professional, personal or financial relationships or interests to influence or limit the scope of the evaluation or evaluation questions and the rigour of their methodology, limit disclosure, or weaken or bias findings.  In addition, they will not allow preconceived ideas, prejudices or social/political biases to affect their analysis; the development of evaluation findings, conclusions, recommendations; and the tone and content of an evaluation report.
performance:
the extent to which effectiveness, efficiency and economy are achieved by a program.
performance measurement strategy:
the selection, development and ongoing use of performance measures for program management or decision-making.
policy:
official guidelines or operating principles that influence behaviour towards a stated outcome.
program:
a group of related activities that are designed and managed to meet a specific public need and are often treated as a budgetary unit.
Program of Grants and Contributions:
refers to the definition of "program" in section 42.1 of the Financial Administration Act, which defines program as a program of grants or contributions made to one or more recipients that are administered so as to achieve a common objective and for which spending authority is provided in an appropriation Act.
relevance:
the extent to which a program addresses a demonstrable need, is appropriate to the federal government, and is responsive to the needs of Canadians.
review of relevance and effectiveness:
Section 42.1 of the Financial Administration Act requires that every department shall conduct, every five years, a review of the relevance and effectiveness of each ongoing program of grants and contributions. This policy defines such a review as a form of evaluation.
risk-based approach to determining evaluation approach and level of effort:
is a method for consideration of risk for the purposes of determining the evaluation approach for individual evaluations. Departments should determine, as required, the specific risk criteria relevant to their context. Specific risk criteria may include the size of the population that could be affected by non-performance of the program, the probability of non-performance, the severity of the consequences that could result, the materiality of the program and its importance to Canadians. Additional criteria could include the recentness and quality of the last evaluation and/or other studies, their findings, the extent of change experienced in the program's environment, or other criteria.
risk-based approach to planning coverage of direct program spending:
is a method for considering risk when planning the extent of evaluation coverage of direct program spending (excluding ongoing programs of grants and contributions) pending the full implementation of section 6.1.8 (a) of the Policy on Evaluation. Departments using a risk-based approach to planning coverage of direct program spending should determine the specific risk criteria relevant to their context that will inform their evaluation coverage and non-coverage choices. Specific risk criteria may include the size of the population(s) that could be affected by non-performance of their individual programs, the probability of non-performance, the severity of the consequences that could result, the materiality of their individual programs and their importance to Canadians. Additional criteria could include the recentness and quality of previous evaluations and/or other studies, their findings, the extent of change experienced in the environment of their individual programs, or other criteria.
small departments and agencies:
are defined as having less than $300M in Annual Reference Level and revenues credited to the Vote.
value for money:
the extent to which a program demonstrates relevance and performance.

Annex B: Roles and Responsibilities of the Departmental Evaluation Committee

The Departmental Evaluation Committee is a senior executive body chaired by the deputy head or senior level designate.  This committee serves as an advisory body to the deputy head related to the departmental evaluation plan, resourcing, and final evaluation reports and may also serve as the decision-making body on other evaluation and evaluation-related activities of the department.  This committee could also support other functions.  The Departmental Evaluation Committee:

  1. reviews the adequacy of evaluation coverage, as expressed in a departmental evaluation plan developed by the head of evaluation, as well as the risk-based approach used for determining the evaluation approach and level of effort to be applied to the individual evaluations comprised in the plan, and recommends the plan for approval by the deputy head;
  2. if requested by the head of evaluation:
    1. reviews and recommends approval of key elements of an evaluation, such as the terms of reference; and
    2. considers and responds to key evaluation-related issues;
  3. reviews final evaluation reports, including management responses and action plans, and recommends approval to the deputy head;
  4. ensures follow-up to action plans approved by the deputy head;
  5. reviews the adequacy of resources allocated to the evaluation function and recommends to the deputy head an adequate level of resources consistent with the departmental evaluation plan;
  6. reviews the adequacy of resources allocated to performance measurement activities as they relate to evaluation, and recommends to the deputy head an adequate level of resources for these activities; and,
  7. reviews the performance of the evaluation function, and recommends action to address any weaknesses.