ARCHIVED - Evaluation of the Treasury Board Submission Process

1. Source: A Guide to Preparing Treasury Board Submissions

2. Refers to all Treasury Board submissions except orders in council requiring Treasury Board recommendation.

3. The study did not assess Treasury Board submissions and associated précis documents for quality. Findings are based on qualitative data gathered from a survey and stakeholder interviews.

4. These issues are derived from the definitions of "relevance," "effectiveness," and "economy" found in Appendix A of the Treasury Board Policy on Evaluation.

5. A Guide to Preparing Treasury Board Submissions.

6. A limited review of submission process models used in other jurisdictions (see Methodology section) suggests that the role played by the Secretariat's assistant secretaries, whereby they present a federal organization's submission to Treasury Board, may be unique.

7. The Secretariat's program sectors are responsible for reviewing and providing advice on the management of federal organizations' financial resources.

8. A Guide to Preparing Treasury Board Submissions.

9. The term "Secretariat analysts" refers to both program analysts and COE analysts.

10. Other than Treasury Board, generally involves a policy committee.

11. It should be noted that when the Secretariat undertakes a Treasury Board submission on its own behalf, it is considered a federal organization like any other.

12. There is no formal definition of "complexity." The evaluation therefore relied on respondents' own criteria of "complex" and "standard." Note that strategic reviews (which can be "standard" or "complex") are not included in the table.

13. The Policy on the Management, Resources and Results Structures supports the development of a common, government-wide approach to planning and managing the relationship between resource expenditures and results while serving as a consistent and enduring foundation for collecting, managing, and reporting financial and non-financial information to Parliament. (http://www.tbs-sct.gc.ca/rma/mrrs-sgrr_e.asp)

14. A Guide to Preparing Treasury Board Submissions, p. 5

15. The Treasury Board submission and précis are then presented to Treasury Board ministers for consideration and approval.

16. While this question was part of the evaluation matrix, it was not covered in this report because (as discussed) it was determined that this was not an outcome of the Treasury Board submission process.

17. This question is not addressed in the findings of the report because the evaluation did not find any unexpected outcomes.

18. Note that when the report refers to "Secretariat analysts," it includes both program analysts and COE analysts if the context in which it is used is the Secretariat but only program analysts if the context is federal organizations (as program analysts are the primary contact for federal organizations). When this distinction is relevant, it is noted in the report.

19. The survey questionnaire was available on the Internet for a period of three weeks. Reminder emails were sent to respondents twice during the three-week period.

20. Responses and comments indicated that these respondents filled out the wrong survey; consequently, their responses were removed to ensure that the data was not contaminated.

21. Models in use in provinces were not reviewed.

22. Co-ordination at the Centre of Government: The Functions and Organization of the Government Office Comparative Analysis of OECD Countries, CEECs and Western Balkan Countries. Michael Ben-Gera. 2004.

23. Ibid. The ten countries are Australia, Austria, France, Germany, Iceland, Ireland, Netherlands, Norway, Spain, and Switzerland.

24. "The audit found that … a more consistent and inclusive approach to overtime costs and overtime forecasts, improvement to management and authorization of leave and overtime, … and development of formal and approved guidelines in the administration of leave and overtime would be beneficial."

25. The Program Activity Architecture of the Treasury Board of Canada Secretariat

26. A Guide to Preparing Treasury Board Submissions, p. 1

27. Ibid., p. 5

28. Feedback reviewed from the following program sector boot camps: February 2009, January 2008, August 2007, September 2004, March 2001, and September 2000.

29. "Appropriateness" refers to whether a submission put forward by a federal organization is necessary (i.e. required).

30. Interviewees also commented on the drawbacks of MCs not being required to undergo an early stage approval process from the Secretariat. One interviewee noted that the absence of Secretariat involvement can lead to problems when MCs include elements that may be contrary to Treasury Board policies.

31. Some federal organizations, however, indicated that they might choose to proceed with a submission even when the program analyst believes that it is not necessary.

32. Numbers may not add up to totals due to rounding.

33. This same question was not asked of federal organizations; therefore, the opinion of whether submissions were necessary is attributable only to Secretariat analysts.

34. From the time the program analyst receives the federal organization's first draft of the Treasury Board submission.

35. It is important to note that the time requirement may be significantly less in some cases, e.g. when the submission is a "top-up" to an existing program or when the submission is deemed to be of low risk and low complexity.

36. Some may argue, however, that flexibility in the Treasury Board submission process is an advantage in that it allows certain submissions, such as those related to Canada's Economic Action Plan, to be given priority handling when special circumstances warrant it.

37. According to guidelines in A Guide to Preparing Treasury Board Submissions

38. RDIMS is a Canadian federal government initiative to manage the full life cycle of any type of electronic document (email, correspondence, reports, etc.).

39. Budget Office Systems Renewal (BOSR) Project, Treasury Board of Canada Secretariat, 2007

40. http://www.tbs-sct.gc.ca/maf-crg/indicators-indicateurs/2008/policies-politiques/policies-politiques-eng.asp

41. Furthermore, scores between MAF Rounds IV and V are not fully comparable. The average MAF scores analyzed for the evaluation are for larger federal organizations.

42. Federal organizations' internal consultations

43. It should be noted that this percentage may not be representative of the level of effort of all COE analysts since only some COE analysts' duties include the review of Treasury Board submissions. Those COE analysts with significant duties relating to Treasury Board submissions may have been more likely to respond to the survey.

44. Numbers may not add to totals due to rounding.

45. Numbers may not add to totals due to rounding.

46. Data on the timing of Treasury Board submissions over the last 10 years show definite peaks in June, October, December, and March.

47. Factors contributing to the quality of Treasury Board submissions (according to survey): submissions appear not to have completed internal consultations (program analysts: 42.7%, COE analysts: 44.9%); submissions not allowing enough time for Secretariat analysts' review (program analysts: 42.5, COE analysts: 49.2%); submissions did not follow the Guide (program analysts: 43.3%, COE analysts: 37.3%); submissions lacking risk and mitigation strategies (program analysts: 40.5%, COE analysts: 47.5%); submissions requesting wrong authorities (program analysts: 39.8%, COE analysts: 38.1%); submissions without appropriate level of justification (program analysts: 45.3%, COE analysts: 44.1%); submissions containing inaccurate information (program analysts: 45.8%, COE analysts: 45.7%); submissions written in an unclear manner (program analysts: 48.6%, COE analysts: 39.4%).

48. While the evaluation team recognizes that turnover rates in federal organizations may also contribute to the quality of Treasury Board submissions and the efficiency of the submission process, this matter is outside the scope of the evaluation and therefore not addressed in the recommendation.

1. The data from this interview was not used, as Corporate Services was not identified as a submitting organization to be interviewed.

2. Treasury Board of Canada Secretariat, Roadmap for Enabling Change: New Risk-Based Approach to Departmental Business and Treasury Board Submissions, 2007.

3. Treasury Board of Canada Secretariat, Increased Delegated Authorities for Contributions, 2007.

4. Treasury Board of Canada Secretariat, Roadmap for Enabling Change: New Risk-Based Approach to Departmental Business and Treasury Board Submissions, 2007.

5. Ibid.

6. Treasury Board of Canada Secretariat, Increased Delegated Authorities for Contributions, 2007.

7. Treasury Board of Canada Secretariat, Roadmap for Enabling Change: New Risk-Based Approach to Departmental Business and Treasury Board Submissions, 2007.