ARCHIVED - An Enhanced Framework for the Management of Information Technology Projects
This page has been archived.
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
An Enhanced Framework for the Management of Information Technology Projects
An Enhanced Framework for the Management of Information Technology Projects
May 28, 1996
Chief Information Officer Branch
Treasury Board of Canada Secretariat
1. Introduction
This paper describes an proposed enhanced framework for the management of information technology projects in the federal government. This enhanced framework, the detailed components of which will be developed and implemented over the next eighteen months, is designed to ensure that government information technology projects fully meet the needs of the business functions they are intended to support, deliver all expected benefits and are completed within their approved time, cost and functionality.
The enhanced framework comprises best practices, principles, methodologies, tools and standards that were identified in an inter-departmental initiative and which provide solutions to project management concerns experienced in the federal government. The framework will continue to evolve as new ideas and tools become available. Opportunities to incorporate new best practices and community knowledge and experience will be sought continuously in the future.
The enhanced framework is so-called because it is based on the existing project management framework defined in Treasury Board policies. Most of the proposed enhancements can be implemented without changes to these policies while the remainder, if adopted, will require only minor changes that will not affect the overall thrust of these policies.
Departments are expected to adopt the enhanced framework as it evolves. The Treasury Board has directed departments to apply this framework to existing information technology projects as applicable, as well as all future information technology projects and to so attest when seeking Treasury Board project approvals. Departments are expected to apply the elements of the framework in an intelligent fashion appropriate to the nature, size and risk of each project. Some elements of the enhanced framework will require some effort and financial investment and may take three to five years to be fully implemented in large departments.
2. Background and Problem Definition
The government is committed to deliver its programs and services more efficiently and effectively through the use of information technology. However, recent cancellations of major information technology projects together with indications that other projects may be experiencing difficulty have raised concerns for the achievement of this goal.
In recognition of these concerns and as part of its ongoing responsibility for advising Ministers on major projects, the Treasury Board Secretariat carried out a review of a sample of 25 of the government's information technology projects, with a total estimated cost of two billion dollars, to identify business, project management, risk management, and human resource issues influencing their outcome.
The Auditor General also reviewed four large government information technology systems under development as described in the Report of the Auditor General, Chapter 12 - Systems Under Development - Managing the Risks, released on October 5, 1995. The Auditor General's findings concurred with those of the Treasury Board Secretariat's review.
In addition, in 1992 Public Works and Government Services Canada initiated a series of Common Purpose Procurement pilot projects in an attempt to improve project success through procurement reform. In January 1995, the department published a summary of findings(1) from audit reviews of this method of procurement and its relation to the success of information technology projects. In addition to procurement issues, these reviews also identified a number of deficiencies in departmental project planning, e.g., risk analysis and business-case.
While problems with information technology projects are not new, their impact seems more pronounced. Their underlying cause seems to be related to significant changes in the project environment that have occurred in the last few years. These changes include: the decentralization of the management of information technology; the transformation of the way the government does its business (business process re-engineering); and rapid technological evolution from mainframe-based systems to more complex distributed client-server systems.
The Canadian government is not alone in this regard. A recent study(2) of 8,380 projects in government jurisdictions and the private sector in the United States indicated that a staggering 31% of all information technology projects are cancelled before they are completed; 53% of those that are completed cost an average of 189% of their original estimates and average only 42% of the originally-proposed features and functions. Only 9% of projects are completed on time and on budget. The lost opportunity costs from project delays and cancellations could not be measured but could easily be in the trillions of dollars, e.g., the delay of the baggage-handling software for the Denver airport cost the city $1.1 million per day.
Applying these Standish findings to the sample of twenty government information technology projects reviewed by the Treasury Board Secretariat would predict that seven would be cancelled and project overruns would total over one billion dollars. It is clear that the government must change to ensure that this prediction does not come true.
The challenge for the government, therefore, is to enhance its framework for managing information technology projects to resolve this situation and ensure greater success. A Treasury Board Secretariat Project Management Office has conducted research in government and private sector organizations that are successfully implementing information technology projects and, with departments, has identified enhancements to the current framework based on the successful practices discovered.
3. The Enhanced Framework
3.1. Direction
The enhanced framework for the management of information technology projects is founded on the following broad directions:
- all components of project management will be addressed;
- the correct balance will be struck between corporate objectives and departmental empowerment and accountability;
- departments will be involved in its development and be committed to its implementation;
- the enhanced framework will be flexible and dynamic to allow optimum application in different situations and to permit evolution of solutions;
- tried and true solutions from other organizations will be adopted as much as possible; and
- departments will deploy ideas as soon as appropriate (rather than waiting until the development of enhanced framework is complete).
3.2. Findings of the Treasury Board Secretariat Review
The enhanced framework will reflect the following conclusions of the Treasury Board Secretariat's review.
- initial project definitions are inadequate;
- senior management's understanding, involvement and support are lacking;
- client focus and involvement are insufficient; and
- Treasury Board Secretariat's submission review process is incomplete.
- project management discipline is inconsistent;
- the project manager's experience often does not reflect the project's size and risk; and
- changes are not rigorously managed.
- regular "sanity" checks are not performed; and
- project performance and progress is poorly measured and reported.
- the current procurement approach is too inflexible and not conducive to co-operation; and
- private sector consultants lack knowledge of government organizations, responsibilities and policies.
- the present culture discourages open discussion of problems and their resolution; and
- while project failures are well advertised, successful projects are generally not.
3.3. Government Response
The government has responded to the above findings as follows:
- the Treasury Board Secretariat has established a Project Management Office to develop and implement an enhanced framework for the management of information technology projects in government;
- an Implementation Team comprising information technology project managers meets regularly to provide advice and review proposals and documents from the Project Management Office;
- an Interdepartmental Committee consisting of assistant deputy ministers responsible for information technology projects meets regularly to provide general direction to the Project Management Office; and
- the Project Management Office has consulted with departments, other governments and the private sector to identify successful practices that could be adapted for use in the federal government.
3.4. The Framework
The enhanced framework for the management of information technology projects is described in seven key areas or "pillars" (see Figure 1).
- Governance: an appropriate management framework that balances the government's corporate requirements with departmental empowerment needs.
- Review: the establishment of effective mechanisms for monitoring the status and assessing the performance of projects.
- Facilitation: the creation and maintenance of a set of best practices, processes and guides that will support and assist departments.
- Professional development: building a cadre of information technology project managers within government who are trained in the application of the enhanced framework and designated common methodologies and tools.
- Pathfinder projects: exploiting opportunities to use and assess new ideas and solutions in an operational situation.
- Communications: initiating and supporting a communication program on the enhanced framework across government.
- Management of Change: introducing and acceptance of changes in attitudes and cultures.
4. Implementing the Enhanced Framework
4.1. Principles
Four overall principles have been defined for the enhanced framework to set the broad parameters within which information technology projects are managed. Specific practices will be developed to support the achievement of these principles.
4.1.1. Support for the Business
Principle: Projects are aligned with, and support, the business directions and priorities
Information technology projects are undertaken to achieve successful and economical support of a business function. The project sponsor, project leader and project manager, together with the management and major users of the business function, must ensure that the system achieves these goals and delivers the expected benefits . The support of business needs must be especially foremost when the system is first conceived and then re-evaluated throughout the project.
There is a danger of this principle becoming mere rhetoric unless there is definite commitment by management and the project team to take it seriously.
Principle: Clear accountabilities are established
Information technology projects can be a huge, complex undertakings. Unless the responsibilities of all parties are clearly defined and their delegated authority clearly specified, the project's accountabilities will be blurred and it will be less likely that problems are solved in a timely manner to prevent them from threatening the success of the project and the achievement of the benefits expected from it. Departments must establish appropriate accountability frameworks to ensure that if problems cannot be resolved promptly they are elevated to the appropriate higher level for resolution.
4.1.3. Corporate Project Manager Discipline
Principle: Project managers are developed in, and work within, a corporate discipline
The project manager is the key for the successful completion of the project and the achievement of the expected benefits. The project manager, therefore, should have the appropriate training, skills and experience required to manage the project's scope and risks. The department must be able to rely on this person to know the project's status, i.e., its progress, cost, and problems, at all times.
Only a few departments have enough projects for them to implement a comprehensive project manager development program. Most have little choice but to appoint an internal manager who may lack some of the desired training and experience or to purchase project management expertise from the private sector. Both of these choices increase the risk that the department will not be able to exercise adequate planning and control over the project. A solution to this problem is a corporate approach to the development or acquisition of project managers based on government-wide practices and common tools and methodologies.
Principle: Project management decisions are based on risk management
The objective of a system development project is to develop an information technology system that successfully supports the business function and complete it within its planned cost and time parameters. Typically the government has emphasized meeting the target date with the result departments have pressed ahead to meet commitments even when there are indications the project is in difficulty. More attention must be given to the project's risk at the outset and at periodic reviews during the course of the project.
The project management framework is expected to change this. Risk management, the determination of and resolution of all threats to the successful achievement of project objectives including the benefits identified in the business case, is a primary concern of the enhanced project management framework. Projects must be planned, organized and structured to ensure success from their initial organization and planning through design, development and implementation. Project management training should place more focus on risk management.
4.2. Practices
This section describes practices to support each of the above four principles for managing projects. Where considered appropriate, common methodologies and tools are proposed that will ensure a consistent, quality project management discipline government-wide.
4.2.1.1. Projects Compatible with Business Plans and Information Management Plans
Projects must be fully consistent with their departmental context. The department's annual Business Plan describes the department's mission, goals and priorities and the changes to be made to implement these. Departments are currently required to have an Information Management Strategy and an Information Management Plan that describe the department's information and technology strategies, architectures, infrastructures, and projects. Projects must be consistent with these strategies and plans and, for those departments that have one, their Long Term Capital Plan. Projects must also be consistent and compatible with ongoing or planned re-engineering initiatives designed to change fundamental business processes.
4.2.1.2. Imps Aligned with Blueprint and Common Infrastructure
Departments' strategies and plans in their Information Management Plans must be consistent with government-wide strategies for the delivery of services, e.g., the TBS Blueprint for Renewing Government Services Using Information Technology. Departments must also conform with approved common information technology architectures or infrastructures, e.g., the public key infrastructure and infrastructure elements provided by the Government Technology and Information Service, unless there are over-riding operational requirements.
4.2.1.3. Full Business-case Analysis Performed
Departments should identify and select the most beneficial investments using an institution-wide strategic planning process based on the business-case approach. Project approval must be based on a business-case analysis that relates each investment directly to the business function and demonstrates the benefits of the investment to the department or government as a whole.
The business case should be based on the full cost of the system from initiation through development, implementation and estimated annual cost of operation. It should be reviewed and revalidated at each scheduled gate and whenever there is a significant change to the project or the business function. If the business-case of a project changes it should be re-approved through the departmental planning and approval process.
4.2.1.4. Clients are Integral to all Project Phases
Clients of systems should be fully involved in all phases of projects to ensure that systems meet their business requirements in the best manner possible and the expected benefits are obtained. The project manager must ensure that all types of clients are represented in the project so that each client group can influence the system's design and implementation. They should be given a clear picture of how they will interact with the system and what it will do for them while there is opportunity to recommend improvements.
Clients should also be involved in the reviews at the project's gates and subsequent decisions on the project's future.
4.2.1.5. Client Responsibilities Defined and Committed To
The project leader must ensure that representatives of client groups commit to the level of effort required to meet their project responsibilities. Their names and explicit responsibilities should be included in the project charter. They should not be seen merely as people to bounce ideas off or to "approve" decisions of the project team but as full team members who take part in all design and implementation decisions of the project. They should be involved in the reviews at project gates and have a major say in the release of funds to continue the project.
4.2.2. Principle: Clear accountabilities are established
4.2.2.1. Departments are Accountable for Projects
Departments are accountable for the successful completion of projects. This accountability should be clearly re-affirmed to Parliament, the Auditor General and the media. This will encourage the senior management of departments to take a more active role over their projects and the department's policies and practices for their management.
4.2.2.2. TBS Responsible for the Project Framework
The Treasury Board Secretariat's central policy role over projects is to establish and maintain the government's framework for the management of projects and to monitor the departmental implementation of this framework. The Treasury Board Secretariat will verify that departments have implemented the framework in order to advise Ministers on project submissions.
4.2.2.3. PWGSC Role in Contracting
Public Works and Government Services is the government's contracting agent for goods and services. Their role with respect to information technology projects is to devise a contracting plan and contract terms and conditions that best meet the requirements of the project and the application of the enhanced framework. In order for PWGSC to achieve this without incurring delay, departments must involve PWGSC at the beginning of the projects. This will allow PWGSC to understand the projects needs and the project manager to take government contracting regulations, trade agreements and economic benefit requirements in consideration when planning the project.
4.2.2.4. Accountabilities of Multiple Stakeholders are Defined
Some projects are not the sole responsibility of one department or a department's project may have requirements attached to it by another department (e.g., economic benefits).
All departments involved in a multi-departmental project should understand and agree with all of the other departments' objectives, roles and levels of participation. These should be documented in memoranda of understanding (MOU), or a project charter signed by a senior official of each department. All of the departments should establish a project management structure for their part of the project. The designated lead department should establish an overall project office, separate from its project team, with an overall project sponsor, project leader and project manager (see 4.2.2.8.) staffed at a sufficiently senior level to be able to intercede at the appropriate level in all participating departments.
For projects to which another department has attached requirements such as regional economic benefits, there is an onus on both departments to ensure that these requirements are achievable without compromising the successful completion of the project. Where the department setting the requirements has operational responsibilities in the area of the requirements, that department should commit through a memorandum of understanding to help meet them. There should be a process whereby a department can obtain relief from the imposed obligations on presentation of evidence that the obligations cannot be met without compromising the success of the project.
4.2.2.5. Organizational Readiness Determined
Departments have different experiences with regard to managing projects. Some departments have many projects underway simultaneously while others may only occasionally have a major project. While the former departments can be expected to maintain project support functions that develop departmental policy and practices for project management, the latter may not. Furthermore, departments may vary in the degree to which they have implemented the management framework for projects. Thus, departmental "readiness(3) to commission a major project, which is directly related to project risk, varies widely and so a department's authority to commence projects should take the department's readiness into consideration.
4.2.2.6. Higher Project Authority Levels Based on Readiness
Departments that have fully implemented the enhanced management framework as indicated by their certification at a level of readiness can request higher project authority levels commensurate with the level of readiness they have achieved. In the interim, departmental project submissions will include an assessment of the extent to which they have implemented the enhanced framework.
4.2.2.7. TBS Reporting Based on Readiness
Departments that have fully implemented the enhanced management framework as indicated by their certification at a level of readiness will have a commensurate reduction in the central verification of their projects (i.e., projects above their authority level). These departments will be allowed to submit abbreviated reports to the Treasury Board Secretariat for projects for which they have been directed to provide progress reports, e.g., major crown projects or projects with high risk.
4.2.2.8. Project Sponsor, Leader and Manager Roles Defined
Three key officials should be identified for each project (for small projects these roles could be assumed by two or even one person; however, all of the responsibilities for the three role are to be specifically assigned). The Project Leader's and Project Manager's roles and responsibilities are defined in, and required by, the Treasury Board policy, Project Management, while the Project Sponsor role has been recommended by the Auditor General. In summary, these responsibilities are:
- project sponsor: is responsible for ensuring that the department understands the value and importance of the project and, ultimately, for realizing the benefits predicted for the project. The project sponsor is typically a senior official in the organization responsible for the business function that the project will support. The project sponsor should have a major say in the release of funds after a review gate.
- project leader: has overall responsibility for the project and is accountable for all external and internal aspects of it. The project leader is typically a senior departmental official.
- project manager: has specific accountability for achieving all of the defined project objectives within the time and resources allocated. The project manager performs the day to day management of the project. Without diluting the responsibility of the overall project manager, he or she may be supported by one or more assistant project managers who have the same responsibilities over specific portions (sub-projects) of the overall project. Project managers should have demonstrated knowledge, skills and experience commensurate with the size, complexity and risk of the project.
The specific responsibilities and obligations of these key project officials should be documented in a formal project charter.
4.2.2.9. Project Team has the Necessary Competencies
The project manager must ensure that the project team has all the necessary competencies required by the project. This requires careful determination of all the different factors in the project that demand particular skills or expertise and then staffing the team to include them.
4.2.2.10. Core Responsibilities and Functions not Outsourced
Organizations that are successful in implementing information technology projects report that they do not outsource the core project management responsibilities and functions. They consider that these are so vital to the project's success that they want to ensure that these people have the requisite training, skills and experience, and that they are under the complete control of the organization's management. They also question the objectivity and conflict of interest of a project manager who is an employee of the company responsible for the project's implementation.
Currently, the government does not have enough employees with the necessary project management training, skills and experience to make it a policy not to outsource core project management responsibilities and functions; however, this is a goal to work towards as such a cadre is developed. In the meantime, departments that must contract for core project management responsibilities should consider acquiring them from a different supplier than that awarded the primary development contract. When these core project management responsibilities are outsourced, the department must structure the contract and its internal project management to ensure full departmental accountability is preserved.
4.2.3.1. Project Managers Selected Against Project Parameters
The project manager is key to the successful completion of projects. The project manager must, therefore, have the knowledge, skills and experience required to manage the project's scope, complexity and risk profile. Project management should be recognized as requiring specialized knowledge, skills and experience and not considered a function that any manager can perform without first acquiring the requisite knowledge, skills and experience. Departments should perform a preliminary assessment of the project's scope, complexity and risk before finding and assigning a project manager who is capable to manage it.
4.2.3.2. Consistent Project Management Discipline
Only a few departments are large enough to have a critical mass of information technology project managers to have their own development program, and even they could benefit from cross-fertilisation of experiences from other departments. To ensure that all departments employ similar standards for information technology project management and to facilitate the movement of project managers among departments, the government is considering a information technology project management discipline that is consistent across all departments.
4.2.3.3. Government-wide Project Manager Development
A single information technology project management discipline will allow the development of government-wide training courses, workshops and forums that should have better quality and higher standards than an individual department could implement. This common program will be implemented through existing government training organizations and shared departmental programs.
In addition to formal development initiatives, an apprenticeship program will be considered whereby beginner project managers and project managers ready to move up to a higher level could obtain a few months experience on existing projects before being appointed as project managers of new projects.
4.2.3.4. Continuous Learning Culture
An individual project manager can never experience all of the things that can happen to projects. A project management culture will be encouraged whereby information technology project managers share their experiences and pass on lessons learned. Forums, workshops and information dissemination vehicles will be organized to support the exchange of practical project management knowledge.
4.2.3.5. Project Manager Support Network
In addition to the general acquisition of knowledge, an information technology project manager support network will be established whereby project managers encountering specific problems could get advice from other knowledgeable, skilled and experienced project managers. This network will take the form of peer group forums specifically set up to discuss actual problems and propose potential solutions, and a mentor system whereby a project manager with more experience will act as a mentor to a project manager with less.
4.2.4. Principle: Project management decisions are based on risk management
4.2.4.1. Existing Solutions Adopted
Adopting a proven solution that can meet the needs of the business function is less risky than developing a custom solution. An off-the-shelf product, or a solution that performs similar functions and services, has been implemented in an organization, and can be adapted to the department's needs, offers lower cost and risk than a custom system. A custom solution should be selected only as a last resort.
4.2.4.2. Projects Structured for Minimum Risk
The size and duration of a project are directly related to risk. Large projects are inherently more complex and risky, and long duration increases the risk of the business needs and technology changing. Successful organizations implement project structures in which large projects are reconstituted into a set of more manageable and less risky sub-projects, each of which is typically less than 12 months in duration and costs less than $1 million.
For this solution to work, a management approach is required that can effectively co-ordinate all of the individual sub-projects, ensure communication among the different sub-project teams, and address shared or horizontal issues. Typically departments must:
- re-organise a large project into a number of smaller sub-projects with sub-project teams of 10 people or fewer - probably more suitable for a major business (re)engineering initiative; or
- establish continuous improvement to the business process through smaller sub-projects providing frequent upgrades to the system - likely more suitable for re-engineering an existing business process that does not require immediate fundamental change.
4.2.4.3. Project Off-ramps Linked to a Gating Process
Projects, particularly long ones, must have scheduled checkpoints or gates when the project is reviewed and management can decide on the future of the project and any appropriate corrective action. Note that a project "gate" is not synonymous with the project milestones, e.g., PPA, EPA, or initial planning. definition, implementation, and close-out. Rather gates should be chosen to coincide with logical project review points, e.g. completion of an activities or sub-projects, that also correspond to relatively constant cash expenditure increments.
Gating also allows the department to control the cost of projects and minimize the financial loss on problem projects. In this approach, a designated senior departmental official, e.g., the project sponsor, manages the funds allocated to the project and releases only the funds needed to reach the next gate to the project leader. The performance of the project is reviewed at each gate, or when the released funds run out (to avoid delays, the designated official could release sufficient funds to permit work to continue on the project for a short time while this review and decision took place). After the review, departmental management can decide to proceed with the project as planned, modify the project and/or its funding, or even terminate the project limiting the loss to the amount previously allocated.
Projects and contracts will have to be structured to avoid incurring major penalties from the application of gating. By requiring the contractor to provide complete information on project performance and progress and also specifying in the contract when the scheduled reviews are to take place, the reviews could be conducted in a reasonable time without the need to stop work. By specifying the option to cancel the contract at the scheduled gates including the criteria on which such a decision would be made, in the contract, gating can be implemented without incurring major penalties.
After performing a risk assessment to identify the risks involved and establishing the project's scheduled gates and decision points, departments can develop contingency plans for potential problems at these points. By developing contingency plans in advance, they can design the project to reduce the impact of switching to another contingency. These contingency plans can be built into the contract to avoid major penalties or having to re-tender the contract.
4.2.4.4. Methodologies and Tools
The starting point for managing risk is the determination of threats to project success and their level of risk. With this knowledge, departments can develop strategies and plans to deal with the identified risks and make informed decisions on the project. Some potential solutions are as follows.
Risk Assessment: Successfully managing risk requires a thorough understanding of all of the risks in the project, including: scope, complexity, extent of change in the business function, skills and experience of the project team, the employment of new technology, and the number of organizations involved. A project risk assessment, such as the non-proprietary methodology developed by the Software Engineering Institute,(4) can identify and quantify the risks of the project so that strategies can be developed to control them. Since risk factors can change over the course of the project, the risk assessment should be performed whenever there is significant change to the project and at scheduled times during a long project.
Determination of project complexity: project complexity derives from the number of business rules, the technology employed and the project's size, and is a major component of project risk . Complexity should be determined at the start of all large projects, and when changes occur so that appropriate action to minimize its risk can be taken. One way to quantify project size is function point analysis. Experience has shown that risk increases rapidly with increasing numbers of function points. In addition, this method also provides an estimate of project cost early in the life-cycle. Projects should be structured so that each sub-project is less than 1500 function points.
Early product: when the project has a long duration, changes may occur to the project environment, e.g., changes in the organization of the department, changes to the business processes, technological change, changes in project team members, and changes affecting the contractor. These risks can be mitigated with a project implementation strategy that produces results in successive implementable pieces - each piece designed to be completed in a relatively short time to provide immediate benefit to the business process. Any environmental change would affect only the parts already completed or in progress. There are a number of methodologies for early product delivery, e. g., prototyping and rapid (architected) applications delivery (RAD or RAAD).
Change Management: unforeseen changes to projects are inevitable; therefore, every project needs a process to manage functional changes. This process should ensure that changes are analyzed quickly to determine their impact (risk, cost and time) and that this information is brought to the attention of the appropriate level of management as soon as possible. Contracts should bind contractors to the department's change management process and provide for third party intervention to resolve any disputes over the cost to implement changes.
Performance measurement: the risk of exceeding time and cost targets is inversely related to the amount and accuracy of the information received by the project manager on the performance of the project team. To know the current project status, the project manager needs data on the time and money expended and on the work completed at frequent intervals. Information on time and money expended is easily obtained for projects done in-house and should be required to be provided as part of the contract terms and conditions. To get complete, up-to-date data on work completed, the work needs to be broken into distinct modules that can be completed in short time periods. A work breakdown structure based on large items forces the project manager to assume that work completed equals time expended and he or she will not know, for some time, if the work on a work item is exceeding the time forecast. With complete information for all three parameters, the project manager knows the status of the project against the project schedule. A suggested performance measurement tool is one based on the national standard, C/SPMS.
Adoption of common methodologies and tools: the methodologies for performing risk assessment and control are so crucial that common methodologies and tools should be considered for use across government. Project managers trained in these methodologies and tools could then use their skills, knowledge and experience in any assignment in any department. In addition, central agency staff and managers new to a particular department will be familiar with the methodologies and tools used and will have more confidence in the risk assessment.
4.2.4.5. Procurement Managers Involved at an Early Stage
Government procurement is complex because it is governed by regulations designed to give equal access to government contracts to all eligible companies, regional development policies designed to ensure that all regions get their fair share, and international agreements such as GATT and NAFTA. When PWGSC procurement officers are involved early in the project planning, they can develop a procurement process that reduces delays, and they can design a procurement plan that best aligns the contracting plan with the project plan while meeting legislative, policy and treaty requirements, and which aligns the contract with the stated benefits of the project. The project's gating plan can be incorporated in the procurement plan and thus in the contracts in such a way that the gating process and subsequent decisions taken will not incur major additional costs or penalties, or result in unnecessary contract splitting. Departments find that if they delay involving PWGSC until the time they want to contract the requirement, they may face long delays to resolve issues and may not be able to structure the contracting as they would like.
To contain the cost and time wasted because of problems, departments should implement activities that will help to identify and resolve problems as early as possible. The project manager can then investigate the problem and develop a strategy to resolve it before it goes too far. The additional cost of these reviews should be offset by the saving that will accrue from early warning of project problems allowing more timely resolution. The following types of reviews are available.
Independent reviews: These are conducted by an independent party at scheduled gates to identify environmental changes, overrun of time and cost targets, or other problems. Funding should also be set aside for unscheduled independent reviews to be undertaken whenever there are significant changes in the project environment or serious concerns about the project.
Internal Peer reviews: Departments engaged in several projects simultaneously will have several project managers and other managers in the systems development, maintenance and operations groups. This body of expertise can be drawn on to conduct periodic peer reviews of projects. These semi-formal reviews allow the project manager to present performance and progress data, to discuss upcoming challenges and to identify any horizontal issues. The object of the peer review is for the group to verify that the project is still on course and to provide expert advice, council and assistance to the project manager. Thus the combined skills and experience of all of these managers is applied to the project.
External Peer Reviews: Departments may also draw upon similar people in other departments or organizations to provide a different perspective and to bring a wide range of expertise to bear on project strategies, plans and issues.
Project team sanity checks: Another source of early warning for project problems is the project team members. These people are the most intimately aware of difficulties or planned activities that may pose particular challenge. The project manager should plan regular sessions where team members can review the continued relevance of the project, project performance and concerns about actual or potential problems in a non-incriminating way.
Oversight reviews: Oversight reviews under a senior steering committee should be planned to take place at each gate to reconfirm that the project is aligned with departmental priorities and directions and to advise senior management on project progress.
Internal audit: The departmental internal audit group can also review the performance of projects. As part of this enhanced framework for managing projects, kits will be developed to assist auditors to check that the framework is being applied to the department's projects and thereby measure the health of projects, e.g., by determining that an appropriate performance measurement tool is being used and from the current status of the project as shown by that tool.
4.3. Implementation Plan
Following consultation with departments, the Management of Information Technology and Project Management policies will be reviewed and amended, as necessary, to include recommended changes to support the requirements of the enhanced framework. Potential new policy directives could include:
- new projects are to be subject to a rigorous risk assessment and appropriate measures taken to reduce and manage risk;
- a gating process is to be implemented for each project to ensure projects are reviewed at prescheduled points with funding for subsequent phases of projects subject to this review;
- methodologies and tools approved for common use are used to manage projects;
- all types of users of a system are to be represented in its planning, design and development;
- departments are to participate in a corporate approach for project manager development;
- project managers will have the requisite skills and experience to match the project's size, complexity and risk; and
- departments that can attest they have fully implemented the enhanced framework and have been certified at a high level of readiness can request a higher project authority level. In the interim, departmental project submissions will include an assessment of the extent to which they have implemented the enhanced framework.
4.3.2.1. Information Technology Procurement Review
Purpose: Building on the review of the Common Purpose Procurement approach,(5) this review of information technology procurement will identify and resolve procurement policy and process problems impeding the timely and cost-effective execution of large information technology acquisition projects.
Scope: The review will identify and resolve issues that can be dealt with within current legislation governing federal acquisition and the constraints imposed by international agreements; however, issues that require legislative changes to resolve will be identified.
Topics: The findings of the Treasury Board Secretariat's review of a sample of projects identified the following procurement-related issues:
- requirements imposed by the Procurement Review Committee, Treasury Board Secretariat and other government departments may add significantly to project risk. Means by which departments responsible for these requirements can be held accountable should be examined;
- in order to reduce risk, the evaluation of bid responses should emphasize the best overall solution;
- the approval, contracting and, in particular, contract amendment processes can absorb excessive amounts of the project team members' time. These processes should be reviewed and streamlined;
- although flaws were identified in the Common Purpose Procurement approach and methodology, its principles are recognized internationally and it provides valuable improvement over traditional acquisition approaches. The optimization of this approach, now referred to as Benefits-Driven Procurement including the examination and incorporation of enhanced partnering arrangements with industry, will be pursued; and
- competitive project definition for requirements specifications is a valid option but requires certain refinements to reduce potential risks.
More recent research indicates that:
- other innovative approaches to acquisition, including Evolutionary Acquisition, New Wave (or Best Value) Procurement should be examined;
- project managers need instruction in current procurement practices and policy, and the implications of international agreements (NAFTA, GATT) to help them to plan for dealing with them; and
- procurement personnel need to acquire more knowledge and exposure to project activities in order to better understand the special issues of systems development contracts.
As stated earlier, the key to successfully managing risk is a thorough understanding of all of the risks in the project, including: scope, complexity, the extent of change in the business function, skills and experience of the project team, employment of new technology, the number of organizations involved, etc. In addition, the methodologies for performing risk assessment are sufficiently crucial that common methodologies and tools should be adopted across government.
It is proposed that the government phase in standard approaches to risk assessment and risk management in the government and for government contractors. Starting with the implementation of the Applied Software Engineering Centre's (6) (ASEC) comprehensive S:PRIME approach as a base, departments would migrate, as appropriate, to the Software Engineering Institute's(7) (SEI) Team Risk Management process followed, also as appropriate, by SEI's Capability Maturity Model (CMM) and Project Risk Assessment Model. It should be noted that the ASEC and SEI approaches are fully compatible since ASEC's approach is a streamlined version of SEI's and is based directly on the SEI models.
The Applied Software Engineering Centre's approach is basic, concise and cheaper and is well suited to organizations with between 10 and 50 professionals directly or indirectly involved in software development and maintenance. Due to its easier implementation, it can be used to "jump start" the adoption of a proactive risk management regime throughout government. The SEI's robust and complex models are well suited to very large systems organizations, and offer a natural, straightforward evolutionary path from ASEC's S:PRIME base as and when appropriate.
Industry representatives will be consulted with regard to their implementation of these assessment tools. They will be advised that, as the government implements them, they will be expected to evolve towards them as well so that both use the same assessment standards. Eventually the achievement of a specified maturity level could be mandatory for both government departments and potential contractors.
4.3.2.3. Professional Development of Information Technology
Project Managers
Recognising that the experience and skills of the project manager are key to the successful completion of projects, a government-wide approach to the training and development of information technology project managers is proposed with the following objectives:
- to build and maintain a cadre of project managers trained in the application of the enhanced framework for the management of information technology projects;
- to encourage a continuous learning culture;
- to facilitate an exchange and sharing of information with industry; and
- to provide a forum to discuss project management issues with peers.
Content: the proposed approach will be based on common government practices, methodologies and tools. It will consist of two interrelated components:
- a training program consisting of workshops, forums and accredited training courses divided into five competency areas: project, technology, procurement, interpersonal skills and business. This training will cover: aligning projects with business needs, directions and priorities; assessing and managing project risks; selecting and awarding contracts; resolving project management issues and problems; employing best project management practices; sharing information on government and industry strategic directions and trends; and enhancing interpersonal skills; and
- a career development program providing development opportunities through on-the-job training, public and private sector rotational assignments and secondments.
Participants: program participants will be divided into four groups in order to address their specific training and development needs:
- (Intern) project managers who are at the start of their project management career path;
- (Professional) project managers with some experience managing medium-sized information technology projects and are responsible for project deliverables;
- (Master) project managers with more extensive experience who have had responsibility for the management of larger, more complex information technology projects; and
- project leaders at all experience levels.
Pathfinder projects provide opportunities to verify and extend the conceptual management framework for information technology projects. Two existing departmental initiatives have been selected as initial pathfinder projects:
- the Immigration Modernization Program in Citizenship & Immigration is using a number of innovative procurement approaches for the acquisition of development and integration services, for infrastructure consolidation, and for key human resources. The evaluation of these approaches will provide information on the possible need for process change, improved training, and further application in government. The department also intends to use a peer review approach at key milestones to ensure the necessary independent perspective is brought to bear on the progress of the project. This aspect of the project will provide a vehicle for determining the peer review process and assessing the relevance of such an approach in a complex project; and
- the implementation of the PeopleSoft human resources software package involves some sixteen departments and consequently will present a major challenge to the government to orchestrate its implementation and on-going maintenance. This initiative will be used to assess the issues involved in the conduct of independent reviews, e.g., positioning of a review, scoping of the work and the methodology used, and to address problems associated with the accountabilities of stakeholders, e.g., departmental users, application managers and service providers.
The enhanced framework for the management of information technology projects is expected to include a number of best practices, guidelines and common methodologies and tools. Initially, departments will be asked to provide examples that they use in the management of their projects that can be adapted for government-wide use. The remaining tools will have to be developed.
4.3.3. The Application of the Framework to Projects
The enhanced framework for the management of information technology projects is being applied as it is being developed. The Treasury Board has directed departments to apply this framework to their existing information technology projects as applicable, and to all future information technology projects and to so attest when seeking Treasury Board project approvals. Departments are at present applying the concepts espoused in this document; this application will continue to improve as supporting products and services are developed.
The Treasury Board Secretariat Project Management Office is facilitating this application of the evolving framework in two ways:
Communication and Consulting: the Treasury Board Secretariat Project Management Office meets regularly with members of the Implementation Team and when needed with the Interdepartmental Committee to advise on its findings from meetings with officials of other government jurisdictions and the private sector. Often members of these organizations are brought to Ottawa to make presentations to federal officials.
Project Review and Advice: Treasury Board Secretariat Project Management Office members are also working with departments with large information technology projects and are advising them on the concepts of the proposed framework as applicable. These departments are starting to perform risk assessments, to structure their projects into smaller more manageable sub-projects and to implement gating processes.
4.4 Detailed Work Plan and Cost Breakdown
1996/97 Development: In 1996/97 the Treasury Board Secretariat will lead the development of policy changes and government-wide practices making up the enhanced framework. The following priority work items will be carried out:
- necessary policy changes will be formulated;
- a review of information technology procurement will be performed;
- a common methodology for project risk assessment will be adopted;
- best practices (methodologies and techniques) will be developed, or adopted (project manager's tool kit);
- the development of a government-wide approach to training and developing project leaders and project managers will begin;
- a management of change program will be developed to ensure the effective implementation and use of the enhanced framework; and
- an inventory of project managers and large information technology projects will be established.
1996/97 Implementation: During 1996/97 the new framework will be implemented in departments as components are completed. The following schedule is anticipated:
- a common methodology for assessing project risk will be selected and two departmental persons will be trained in its use;
- a common tool for assessing project size (function point analysis) will be selected and departments will have staff trained in this technique and will begin to implement it in their departments;
- a common tool for project performance and progress tracking (based on the national standard, C/SPMS) has been adopted, departments will train their staff on its use and continue to implement it; and
- departments are expected to conduct three independent project reviews.
1996/97 Costs: The estimated costs to the Treasury Board Secretariat in 1996/97 to perform this work are:
TOTAL |
$100K $360K
$755K |
In addition, departments will incur costs in 1996/97 which will be absorbed within existing project budgets:
TOTAL |
$25K $150K |
1997/98 Development The Treasury Board Secretariat will continue the development (begun in 1996/97) of a government-wide approach to training and developing project leaders and project managers and will conduct 2 pilot courses.
1997/98 Implementation: Departments are expected to undertake the following implementation activities in 1997/98:
- twenty-four departmental project managers are expected to attend the above projects leader and project manager pilot courses; and
- departments are expected to conduct ten risk assessments and five independent reviews.
1997/98 Costs: The estimated costs to the Treasury Board Secretariat in 1997/98 to perform this work are:
TOTAL |
$125K
$360K |
Departments will incur the following costs in 1997/98 which will be absorbed within existing project budgets:
TOTAL |
$100K $250K $250K $600K |
4.4.3. Ongoing Annual Costs - 1998/99 Onward
The Treasury Board Secretariat is not expected to incur any additional costs to monitor and maintain the framework.
The ongoing annual costs to departments from 1998/99 on that will be absorbed within project budgets are estimated to be:
TOTAL |
$625K $250K $175K
|
5. Funding Requirements
| Stage | 1996/1997 | 1997/1998 | 1998/1999 on |
|---|---|---|---|
| Develop | TBS: Professional Development-125K Procurement Review - $100K Risk Assessment - $20K Project Manager's Tool Kit -$50K Management of Change - $50K Depts: - na |
TBS: Professional Development - $125K
Depts: - na |
TBS: - na
Depts: - na |
| Implement |
TBS: 3 Additional People - $216K Risk Assessment Training - $50K Depts: |
TBS: 2 Additional People - $144K
|
TBS: - na
Depts: - na |
| Operation |
TBS: 2 Additional People - $144K
|
TBS: 3 Additional People - $216K Depts: |
TBS: - na
|
| IRs - independent reviews RAs - risk assessments PD - project manager professional development (courses x participants) |
|||
6. Business Case
The findings of the Standish Group International Inc. were presented at the beginning of this report. If these findings were to apply to the sample of 25 government information technology projects estimated to cost two billion dollars that were reviewed by the Treasury Board Secretariat, as many as seven will be cancelled and project overruns will exceed one billion dollars. Given that, in total, there are about 80 large information technology projects in government then, based on the Standish prediction, the potential for loss could be over three billion dollars.
Although the government's record so far does not support the prediction of loss of this magnitude, it is clear that significant funds are at risk. Even if the enhanced framework only succeeds in saving a fraction of the funds at risk, the potential return on its investment is hundreds of millions of dollars! The expectation, of course is that, while nothing can totally eliminate risk from large, complex undertakings, this enhanced framework will significantly reduce project overruns, ensure project cancellations are rare and, when they do occur, they do so much sooner in the project life-cycle when less money has been spent on them.
Footnotes
- Procurement Reform and the Success of IT/SI Projects, Audit and Evaluation Branch, Public Works and Government Services Canada, January 1995. [Return]
- Charting the Seas of Information Technology, Chaos, The Standish Group International, Inc., 1994.[Return]
- As determined, for example, by the Capability Maturity Model, Software Engineering Institute, Carnegie Mellon University. Pittsburgh, Pennsylvania 15213.[Return]
- Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213. [Return]
- Procurement Reform and the Success of IT/SI Projects, Audit and Evaluation Branch, Public Works and Government Services Canada, January 1995.[Return]
- Applied Software Engineering Centre, Centre de Recherche Informatique de Montréal, 1801 McGill College Avenue, Suite 800, Montréal, Québec H3A 2N4[Return]
- Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvanis 15213 [Return]