Words: 2975
Technology in Government and valuable assistance from several staff members. Sharon Dates, Theresa Pardon, Donna Concentrators, and Meghan Cook made important contributions to early versions. Natalie Heeling, Daybreak Jaguar provided excellent suggestions and edits on the final version. Mark Leaving helped with editing as well, and also worked with Paula Hauser on the design of the printed document. The research on costs and returns of investment in Web site conversion, reported here as a case study, is based on the technical work of Derek Worthwhile and James Costello, and the assistance of Carrie Schneider and VI-Jung Www.
Table of Contents Executive Summary 1 Chapter One 5 ROI and the Need for Smart IT Investment Decisions 6 An approach to understanding and using ROI analysis 7 Understanding Return on Investment in Information Technology: a Guide for Managers By Quadrilateral 1 resources and at the same time respond to ever-increasing demands for improved performance and new technology. These competing demands generate close scrutiny of proposals for new information technology (IT) investments. What’s more, high profile IT system failures have raised concerns about why these investments so often fail to live up to expectations.
As a result, many IT investment planning processes now require some analysis of the costs and returns expected from that proposed investment. Unfortunately, public sector managers often lack models that can guide them through such analyses. This Guide is offered to help fill that gap. The Guide provides that help by presenting a practical approach to understanding what ROI analysis can and cannot do. A meaningful return on investment (ROI) analysis in information technology is a little like saying you want to live a healthier lifestyle.
Like lifestyle changes, ROI analysis is not Just a single component. Instead, it is a collection of methods, skills, tools, activities, and ideas. They can be combined and used in many different ways to assess the relative value of an investment over time. Applying this collection in a particular situation requires making many choices among the ideas and methods available and conducting an analysis appropriate to the decision at hand. Different choices will produce different results. Therefore the Guide presents a framework of the key questions that should lead to an appropriate ROI analysis.
It then presents a review of the methods and resources needed along with examples of different approaches in detailed case studies. How extensive should your ROI analysis be? Once the decision has been made to conduct an ROI analysis, what should it look like? The choice of how to conduct the analysis should be based on four critical principles pertaining to: I the strategic objective(s) of the ROI analysis, the place (and importance) of the IT investment in the overall enterprise architecture, the type of analysis that should be conducted (I. . , what data and methods of analysis are best suited to those objectives), and how the ROI analysis fits in the overall decision context for IT investments. Understanding the strategic objectives of your ROI analysis Understanding the strategic objectives of an ROI analysis will determine how the analysis is ultimately done and used. A handful of questions like these can help managers decide what the objectives of the analysis should be. I Is the proposed project critical to the business objectives of your agencies or government?
What are the risk factors associated with the investment? Who will be impacted-?positively or negatively-?by the proposed project? Is an ROI necessary for approval and support of the proposed project? Is the proposed project worth the investment of an ROI analysis? And if so, how detailed should it be? Answering these questions will help identify the resources needed to conduct an ROI analysis, which in some cases can itself be a substantial investment. Extending its level of detail beyond what is needed for effective decision making is a waste of resources.
Focusing on strategic objectives keeps attention on the full range of benefits to be expected from the investment, and how to measure them. Choosing the right type of analysis Choosing and using the various methods of ROI analysis requires sound knowledge ND Judgment: knowledge about the methods and Judgment about how best to apply them. The methods chosen should fit the particular questions asked of an ROI analysis. Different questions require different measurement approaches to fit them. In general, there are four types of questions that prompt or drive an ROI analysis: financial, effectiveness, efficiency, and impact.
Financial: Can we afford this? Will it pay for itself? An ROI analysis that answers these questions is based on expected savings and revenue increases compared to the dollar cost of all expenditures on the new system. The measures are set by generally accepted or legally mandated accounting standards and practices that apply to the particular government organization. The costs and savings or revenue might be projected over a multi-year time span to show a payback period or to estimate the present value of future returns. Effectiveness: How much “bang for the buck” will we get out of this project?
The ROI analysis that will answer these types of questions considers how much the investment contributes to achieving program goals and producing the desired results. It considers direct, indirect, and opportunity costs. The indirect costs include such things as training and administration over time. An opportunity cost could be the loss of return or revenue you would have received had you chosen a different alternative. The measurement of returns will be expanded beyond cost savings to include levels of performance relative to program or project goals.
Understanding the context of your IT investment Any IT investment project is embedded in an organization’s technology infrastructure (enterprise architecture), relevant business processes, organizational environment, and external relationships. I Technology infrastructure. There are direct costs associated with the technology and services in which you invest, and there will also be costs in terms of its impact on other technology systems already in place. The benefits range from more efficient automation and workflow to improved collection, storage, and access to information.
Business processes. An ROI analysis must not only account for the improvements to relevant business processes, but also for the costs associated with training staff involved in using the proposed technology system. Organizational environment. Other costs and returns will be linked to the organization, for example through letter resource flows, performance changes, changes in work flows and internal relationships. External relationships. Linkages with the external environment may be significant as well.
Resources may be committed from this environment to support the project and additional costs may be imposed on external persons or organizations by changes in the way services are delivered or other business is conducted. 2 Efficiency: Is this the most we can get for this much investment? The ROI that tackles this question requires information about the greatest possible value relative to its costs. Efficiency cannot be separated from effectiveness. It is usually expressed in terms of optimizing the value of a return for a given cost or input.
Establishing that some particular result is the best of all possible results requires either examining many alternatives or simulating performance in some way that provides a valid picture of what is possible. This can be done for some (but not all) kinds of systems with sufficient resources and data. However to do so can substantially increase the cost and complexity of the analysis. Impact: Will the benefits to society (our state, our city, etc. ) Justify the overall investment in this project? The analysis that answers impact questions will be concerned with the larger social and economic benefits and costs of a project.
To define and measure variables that represent social costs or benefits requires more than the typical economic or accounting frameworks. These measures are based on either the specific program results desired by an agency or on general social benefits and improved quality of life. Though not impossible, the breadth and complexity of this kind of ROI analysis is rarely found in IT investment planning. When deciding how to prepare and present an ROI analysis, therefore, it is best to aka into account all the potential risks that influence the decision process.
Undertaking an ROI analysis should include attention to the risk factors identified below. Risk factors that can impact investment decision process Politics and policy factors Public exposure to failure Divided authority over decisions Multiple stakeholders Year-to-year budget cycles Highly regulated procurement processes Organizational factors Complex program networks Misalignment of (or conflicting) internal goals Lack of leadership support Business process factors Impact on existing process Fear of changing work assignments Technology factors
Rapid changes in technology Interacting with parallel systems Scale and complexity How does the ROI analysis fit into the overall decision context for IT investments? Investment decisions in the public sector, whether they involve IT or not, necessarily take place in a context of political and policy influences. No matter how solid or technically sophisticated an ROI analysis may be, it will not likely be the sole determinant of an investment decision. Most of the risk assessment issues listed here involve problems related to thinking beyond the boundaries of the project, measuring factors, or determining rehabilitates.
Simply recognizing where uncertainty and potential damage may lie is half the battle. Careful risk analysis, based on the best available data and estimates, will surely assist in ROI analysis and improve planning, even if the amount or quality of data is less than ideal. Considering these various risk factors can help shape the style, emphasis or presentation strategies employed to introduce the analysis into the decision 3 making process. Such considerations as those listed here may also help in recruiting support for the conclusions of your ROI analysis and guiding how the analysis recess is positioned when seeking that support.
Conclusion There is no single “right” way to conduct a return on investment analysis. Nor is there a Consumer Reports for ROI products and services. In determining how to conduct your analysis, the best advice is to focus on the strategic objectives of the analysis along with the goals and business processes of the proposed project. This focus will help guide decisions about the resources and methods to use to conduct a sound and valuable ROI analysis. Chapter One: ROI and the Need for Smart IT Investment Decisions “It [ROI] will let us gin to make assessments and decisions about funding a project or developing a new service based upon some true data. That moves you from having emotional debates about projects to having factual discussions. ” Gerry Wetting, Missouri CIO overspent decision makers must make the most of scarce resources and at the same time respond to ever-increasing demands for improved performance and new technology. Thus the need for wise investment in information technology continues to grow.
Growing demand in the face of scarce resources generates hard questions and close scrutiny of proposals for new investments. What’s more, the dismal failure cord of many government IT investments raises legitimate concerns about the value of these investments and why they so often fail to live up to expectations, or even to work at all. As a result, IT planning processes often include, or even require, a rigorous business case to Justify new IT investments. These include ways of assessing the costs and returns to be expected from that investment, that is, return on investment (ROI) analysis.
This Guide is designed to help government executives who need to design, direct, conduct, or work with the results of such an analysis to make the most of their investments in information technology. Growing interest in assessing returns on IT investments has spawned wide interest in methods of return on investment analysis. However, there is little agreement about best practices or specific methods for assessing ROI. Professional publications and consultant white papers present quite a variety of possible approaches.
As a result, government executives and decision makers have difficulty choosing or designing a return on investment analysis that is both feasible and appropriate to their needs. To help administrators and decision makers with these choices this guide presents an overview of the purposes and concepts of ROI along with an introduction to basic methods and example cases. It also includes links to other resources for those who wish to explore some subjects in greater depth. This guide treats ROI analysis as part of the overall decision making process for IT investment.
The planners and designers of an IT project can use ROI analysis to help persuade decision makers to support the project. Decision makers can use an ROI analysis, indeed may even require one, as part of an IT investment proposal to aid them in evaluating it. In either case, an ROI analysis will be shaped by the tuition in which it is designed and carried out. Decisions about what sort of return on investment analysis to do, or whether to do one at all, will usually depend on a variety of factors. ROI analysis may or may not fit the larger context of investment decision making.
Decision situations driven by very short deadlines or highly specific policy directives may rule out extensive analysis. Scale matters as well. An elaborate ROI analysis would hardly be Justified for a small-scale, low-risk project that requires a fast decision. By contrast, large complex projects are typically highest propositions or which the added time and cost of an extensive ROI analysis would be fully justified. Even though Justified, in some environments ROI analysis is not used at all in favor of best practice reviews or benchmarking to evaluate investment possibilities.
Current practices vary “Running the Numbers,” Government Technology, June 2002, p. 23. 5 considerably. In Iowa, for example, the state government has a standard framework for all agencies submitting IT project proposals, including business case and ROI analysis requirements. 2 The US Office of Management and Budget has imposed similar requirements on federal agencies. Agencies and local governments in other states may develop their own internal business case and ROI requirements. Given the diversity of practice in the IT investment world, this guide takes an eclectic, non- prescriptive approach.
It treats the subject of ROI analysis from the point of view of a curious but uncertain decision maker. The key issues facing this decision maker are whether to do an ROI analysis and, if so, how. The guide does not advocate engaging in ROI analysis under all circumstances, nor does it favor any particular technique. Instead it presents an approach to understanding a range of purposes and methods or ROI analysis that can assist that decision maker to move forward with wise and effective IT investment choices. In general is a rather diverse collection of methods, skills, tools, activities, and ideas.
They all may be useful for assessing the relative value over time of some investment. These methods are not, however, a single formula or predetermined calculation that will yield a simple yes-or-no answer to the question of how to invest. ROI is not a silver bullet. Actually designing and carrying out any kind of ROI analysis requires making many choices among the ideas and methods available and conducting an analysis appropriate to the decision situation. Different choices will produce different results. Consequently, a meaningful analysis of returns on investment in information technology is far easier said than done.
Choices about how to conduct an ROI analysis should be based on critical understandings about: I the strategic objective(s) of the analysis, the place of the proposed IT investment in the overall enterprise 3, exactly how the analysis should be done (I. E. , what data and methods of analysis are best suited to those objectives), and how the ROI analysis fits in the overall decision context for IT investments. An approach to understanding and using ROI analysis Decisions about how to use ROI analysis depend on understanding the nature of the methods themselves and how they relate to the business setting.
ROI analysis This guide will introduce you to these four basic understandings and provide resources for deeper investigation of each. The Iowa ROI approach can be found at http://www. Info. State. IA. Us/ROI/index. HTML. The Federal business case requirements are found in MOB Circular No. A-11. 3 The enterprise can be a single agency or unit, or something as broad as the education or justice enterprise. The U. S. Chief Information Officers Council defines an enterprise in terms of enterprise architectures, or “blueprints” for systematically and completely defining organizations’ current (baseline) or desired (target) environments.
See A Practical Guide to Federal Enterprise Architecture, Version 1 . 1, Chief Information Officers Council, February 2001, p. 2. More specifically, the National Association of State Chief Information Officers defines enterprise architecture as an overall plan for designing, implementing and maintaining the infrastructure to support the enterprise’s business functions and underlying networks and systems. See Enterprise Architecture Development Tool-Kit, Version 2. 0, National Association of State Chief Information Officers, July 2002, p. 243. Understanding strategic objectives in ROI design Your understanding of the strategic objectives of an ROI analysis will determine how the analysis is ultimately done and used. The matter of strategic objectives has two related parts. One deals with the objectives and context of the proposed investment. The second deals with the objectives and context of the ROI work itself. An adequate understanding of the first part of the strategic objective must include answers to Hess key questions detailed below. What are the programmatic and business goals of the proposed investment?