|
Information Systems Development
Published: October 1, 2007 This article presents a 9-step approach for information systems development.
The ProblemA government agency needed a project management system: not Microsoft Project, but a business information system to manage their functional projects. The overall scope included projects, staff, deliverables, assignments, status reports, travel and expenses, client organizations and their staff, and the like. So the agency hired a contractor to come in and do a requirements analysis and build a business information system. The requirements analysis was standard fare: meet with management, meet with functional experts, and meet with technical staff. Everybody was interviewed and the requirements document filled several volumes. Everyone was happy. The contractor took the requirements document away and implemented the business information system. After about a year of detailed design, coding, unit testing, and system testing, the business information system was delivered to the government agency. The government agency shrieked in horror. What had the contractor done? That is not what the agency had said. Not what they had wanted. How could this happen? The contractor was promptly fired. Since this clearly had to be an aberration, a new contractor was hired. The new contractor met with management, met with functional experts, and met with technical staff. Everybody was interviewed and the requirements document filled several volumes. Everyone was happy. The contractor took the requirements document away and implemented the business information system. After about a year of detailed design, coding, unit testing, and system testing, the business information system was delivered to the government agency. The government agency shrieked in horror. What had the contractor done? That is not what the agency had said. Not what they had wanted. How could this happen? The contractor was promptly fired. Since this clearly had to be an aberration, a new contractor was hired. Now to save the readers time, just go to back to the start of this cycle and read it again: two more times. An epiphany then happened to the government agency. Since all the contractors were using the same data processing and implementation technology infrastructure to build the business information system, it had to be the fault of the underlying technology infrastructure. The next step was to try to terminate and/or replace the underlying technology infrastructure. At that point, the technology vendor brought in a consultant to completely evaluate the situation. The cycles of requirements through failure were examined to determine what went wrong. A classic methodology was used, and so it was presumed that there would be a classic result. Actually, there was. After the study, a meeting was called by the consultant with the government agency’s heads. The agency was eager to know not only what went wrong but who to blame. The answer was a shock. It was the agency that was at fault, not the contractors. Of course, the agency angrily protested the findings. The consultant carefully went through the requirements-failure cycles and showed that during every cycle the perception of what the problem was and, therefore, what the solution should be had changed. It was not a case of whether the contractor got the requirements right. Rather, it was a case of the government agency not knowing what requirements it actually wanted. When the contractor talked to different government staff, the answers were different not just from one staff member to another, but from previous answers provided by the same government staff member over the different cycles. The objective, a successfully developed business information system, was impossible to achieve. What to do? It was suggested to the government agency that they create their own detailed specification of what needed to be accomplished. The agency countered, “That’s the contractor’s responsibility!” The consultant indicated that if four different contractors couldn’t create the “just right” specification, it was not the contractor’s fault. The agency asked what it should do. The consultant suggested that a week-long workshop be conducted with agency staff to derive the requirements. The agency agreed. The consultant asked the agency head for the names of the staff that absolutely were too important to be on the team. A list was immediately produced. The consultant said, “These are the names of the individuals that must participate.” The agency head’s response was not “printable.” The final agreement was this. A workshop would be conducted that would cause the creation not only of the specifications of the business information system to be produced, but also a prototype as the specification’s proof. Further, the specifications were to be stored in an early version of the Whitemarsh Metabase so that the repository could be used in the subsequent full implementation contract as the key reference point for the specification of what needed to be implemented. Finally, the process was required to iterate the requirements via the prototype demonstrations until complete. The workshop was conducted over five full days. There were four agency teams of four individuals each along with one information technology person per team. Each team lead had to be a functional expert. Additionally, the team lead was instructed that the information technology person was to act solely as the team’s scribe in creating the metadata. If the information technology person got out of hand, it was suggested to the team lead that they could to use duct-tape to silence the information technology person. A stick was also provided. At the end of the week, the specification, the metadata and a high-level prototype were complete. The agency’s information technology department took the result and increased it with one or two more levels of detail, including evolving the prototype. The full implementation contract was let and the system was developed, tested, and accepted by the agency. The Problem from Requirements ChangesThe United States Government’s General Accountability Office (GAO) has been studying information technology projects for a number of years. A review of United States General Accountability Office studies of why business information systems fail shows that new requirements so commonly crop up during the business information systems development phase that this must be considered intrinsic to the software development life cycle as currently practiced. This phenomenon is the root cause of a preponderance of these GAO-identified reasons for failure. The new problems that arise when new requirements are uncovered during development fall into two categories: database design changes and software changes. Significant database design changes often result from an insufficiently data-driven methodology through which the database is initially designed. Experience has shown that very high quality database designs created through a data-driven methodology commonly return many times their design cost in reduced software development and evolution costs. Software changes result from database design changes and also from process logic changes. Database design changes can be largely eliminated through the use of a quality database design methodology. The onerous effects of process logic changes can be dramatically affected through the use of object-oriented analysis, design and programming techniques employed within the environment of business information system generators. A key strategy to minimize the negative impact of software changes is “code-generation,” that is, through a business information system generator. It is a software system that takes in metadata specifications that have been created through the Metabase or other tools such as data modeling tools, and outputs the actual production-ready business information system. Business information system generators have evolved greatly in the last 20 years and should be used in almost all situations. Clarion is a business information system generator. A review of eight GAO studies mentioned earlier in this article clearly shows that the main reasons why business information systems fail has nothing to do with information technology. Rather, the failures reside outside the sphere of information technology control. Nevertheless, information technology must work with the enterprise in the creation of a knowledge worker environment that enables information technology to be successful. Such an environment would be win-win all the way around. In support of that goal, this article presents the following:
Before presenting this 9-step approach in any detail, a fundamental objection to this entire approach must be addressed. It has become painfully clear that some organizations try to avoid the problem of underdeveloped or immature requirements by buying commercial off-the-shelf software (COTS). COTS, if improperly procured, may exacerbate this problem, not solve it. COTS Is Not the SolutionCommercial off-the-shelf software (COTS) is not the solution. When COTS is purchased, what is actually being procured is software based on somebody else’s requirements analysis. Was that analysis sufficient? Was it comprehensive? Did it match the organization’s real needs? If any of the answers is no, then buying COTS could produce a bad result for four reasons.
If an organization wishes to purchase COTS, it is because of the four reasons cited above that the 9-step approach, albeit modified, is even more essential. During Step 8, instead of actually “building” the software, the organization takes the five critical elements that result from this approach (Missions, Organizations, Functions, Database Design, and Validated Prototype) and wraps all five into a request for proposal and issues it to a potential set of vendors. What will be given to the potential vendors is a highly refined and validated set of requirements, and what will come back from the vendors will be a COTS proposal and software system that matches the “real” requirements. IT Environmental Prerequisites for SuccessThere are essentials for business information system development success that return many times the cost. While business information systems can be implemented without these essentials, they cannot be accomplished in an integrated, non-redundant, and cost-effective manner without them.
The 9-Step ApproachThe nine-steps that address many of the Standish and GAO information technology findings are:
The 9-step approach represents a significant change in responsibilities. From the preface example, there are cost overruns, late deliveries, and diminished capabilities. In contrast, this 9-step approach moves the responsibility for the critical-to-success steps to the requirements development organization where it always rightly belonged. The first seven steps are the responsibility of the requirements development organization. The business information systems development organization implements the business information system within Step 8. Step 9, Conformance Testing, is the responsibility of the requirements development organization. The division of labor is based on subject matter expertise. The PayoffBusiness information system generators play a critical role in prototyping and in design iterations. If a first-cut business information system can be created in an hour and made ready to demonstrate in just a day or two, the resources required for prototyping can be dramatically reduced. Consider the following example for business information system life cycle costs. In general, if the costs associated with requirements and design are $1, the activities associated with detailed design through initial business information system implementation cost another $4. That’s $5 in total for a first implementation cycle. Figure 1 illustrates the main phases associated with the traditional first implementation life cycle of an enterprise-wide business information system such as human resources, or accounting and finance. The scale on the bottom represents the quantity of months spent in each phase. The vertical scale that is not shown would be staff hours. The purpose of the chart is to show the relative quantities of staff hours expended across time. The curve clearly shows that the bulk of the effort (commonly about 70%) is expended before any demonstration is possible. Figure 1 shows that:
The “problem” may have also significantly changed before the final solution arrives. This is the “hazard” that GAO concludes is intrinsic to the very process. This can result in perpetual recycling of requirements without ever getting to System Testing. Beyond the first-cycle implementation cost, the total life cycle expenditure for business information system revision cycles (not shown in Figure 1) commonly costs five times more. The total life cycle cost is thus 30 times the design cost. The problem, however, is not that requirements change. Rather, the real problem is that the effects of requirement changes that occur once System Testing et al are complete are too costly to reflect in the implemented business information system. It must assumed as a given at the very start that requirements will change, even though this assumption is seldom folded into methodologies of today. Requirements changing can be especially fatal to procured software packages if these packages have not been designed for change from the very beginning. Because $1 in requirements’ changes causes $29 in additional life cycle costs, the exhortation is simple: Get the requirements right the first time because the cost of change is prohibitive. The reason the costs are so high and the time is so long is that:
Only by adopting the suggested techniques and tools can a reduction of 60% or more be realized. For example, if a data-driven methodology is adopted and is supported by a quality business information system generator, the steps, shown in Figure 1, “Detail Design...,” and “System Testing...” can be largely eliminated. If a quality Metabase environment is installed that stores and manages all the data models on an enterprise-wide basis, there can be an improvement in the first phase, Requirements Analysis and Design as well. Finally, if we implement only after four to six design iteration cycles of a prototype, a good bit of the last phase, Implementation, Operations, and Maintenance can be eliminated. Quality business information system generators can import a database’s design and generate a working business information system in a few hours. From that first generation, the generated business information system can be “electronically pruned” to a form that is suitable for demonstration. After several iterations of demonstration, modification, and regeneration, a quality specification can be created. The change to the process is illustrated in Figure 2. Shown are four prototype development cycles and one full-implementation cycle. The first “D” (i.e., development) cycle contains the activities noted (that is, Design, ... , Feedback). Each subsequent “D” cycle is dramatically reduced because it only is focused on design changes, regeneration, and the next demonstration. The first cycle contains a requirements analysis and design step. But, that step is reduced to mainly producing the data model. To arrive at the prototype, the business information system generator takes in the data model’s design and generates the first-cut business information system. A day or so is then spent to make it “pretty.” Then, the prototype is demonstrated. In all, only about six weeks will have passed. Each “D” thereafter can occur in one or two weeks. Under this changed approach:
Under either case, the total life cycle cost is the cost of the first cycle plus five times more to account for revisions and extensions. While over time, the subsequent revisions should cost less, several things happen to thwart that. First, as systems developed through traditional design and coding techniques get “old,” their designs and program constructions degrade through the application of ad hoc patches and quick fixes. It takes much longer to accomplish maintenance work than new construction work. The result is the same volume of work. Second, because of staff turnover, new staff must relearn the system from scratch. That’s almost a complete recycle of the Figure 1 processes. It is both painful and long. In addition, there’s the “I’d never have done it that way syndrome” to contend with. Every maintenance task seems to get transformed into a fix-and-enhance task. This naturally takes longer and the enhancement may actually install new bugs. Third, as new features are required, some of these features are real design breakers. Because a production system has already been implemented, changes take much longer because of redesign, recoding, and the very tedious and error-prone data conversion. All the costing is shown in Figure 3. Under the traditional approach, if the first box costs $400,000, the total life cycle costs are 30x, that is, 5 + (5 * 5) or $12 million.
Faced with high costs, the high-pressure demand for immediate and visible results, the most common approach is to trim the first box. Suppose it was trimmed by one-half. In theory then, the total cost of the first business information system implementation cycle would be reduced to 2.5, that is, 0.5 + (4x 0.5). The total life cycle cost would only be reduced to 15 units, or $6 million. That’s a 50% savings. Such savings are, however, false. That’s because the real requirements are still there. The only thing that has changed is that the analysis to discover the real requirements has been reduced by 50%. In short, 50% of the real requirements are undiscovered. Now, if the requirements and design efforts are reduced by 50% from 1.0 to 0.5, and given the GAO error allocation of 41% of all information technology errors if this step is underdone or done wrong, then the “4x” (the amount to accomplish what is really “required”) is likely to be 12x. That is because it’s done wrong, it has to be undone and then done right. Hence, three times the “4x” number. In that case, the first business information system implementation cycle cost is not the original 5 nor the 2.5, but is 6.5 ( 2.5 (done wrong) + 4.0 (now done right) ). The overall life cycle unit quantity becomes 39, that is, 6.5 + (6.5 * 5). This is really a “pay me now or pay me more later” situation. If a unit costs $400,000, the life cycle cost is now $15.6 million instead of the original $12 million. In short, a requirements analysis of design cut of $200K results in an overall cost increase of $3.6 million. Clearly, these savings are costly indeed. For a real impact, the savings must be made to the 4x component (that is, Detailed Design, ... , Training). If the 4x component is reduced to 2x, then the life cycle costs are reduced to $7.2 million, that is, 400K * 3 + (400K * 3) * 5. In addition, if due to the business information system generator approach, three of the major business information systems evolution cycles are eliminated, then the whole life cycle cost is reduced to $3.6 million, that is, 400K * 3 + (400K * 3) * 2. That represents dramatic savings of about 66% from the original $12 million. Now, in contrast to the $200K savings that cost $3.6 million, these are real savings Simply, this means that three business information systems can be implemented for the price of one, or that the productivity goes up by 300%. A compelling case. But can it be done? Yes, and this has been standard best practice for the data-driven Clarion community for the past 20 years. The steps for a quality business information system generator approach are these:
Thereafter, present the prototype to users, gather changes, and repeat the steps above until the users say, “enough already.” Then, and only then, create the production version. Once the production version is created, there are the evolution and maintenance cycles. The code-generator approach dramatically affects maintenance as well. There were three reasons cited earlier that cause maintenance to either remain level or take more time under the traditional approach. In contrast to the traditional maintenance approach, under a quality business information system generator approach, many of the first type of evolution and maintenance problems are eliminated outright. The second type of evolution and maintenance problem is of course, unavoidable, but because 90% or more of any business information system’s code is generated, it does not have to be learned. In fact, because the ultimate program’s code sort of doesn’t really exist, it never has to be addressed at all. Quality business information system generators operate at a meta-design level. The actual program code is generated from this meta-design level. Changes are made at the meta-design level as well. So, once the changes are made, the ultimate code is regenerated. If it wasn’t necessary to deal with real code at first, there is no need now. Finally, the third type of evolution and maintenance problem largely disappears because there are four or more design iterations before first implementation. The five cycles of maintenance caused by an immature design are eliminated because the design has matured through prototyping before it is implemented. SummaryThere is no down-side to the adoption of this 9-step approach. The overall information technology organization and the functional organizations that are supported by this approach are more productive, less costly, higher quality, and lower risk because more work is done in a non-redundant, integrated manner. More work products are able to be re-used because they are created with re-use in mind, stored in a metadata format, and reside in a multiple-user Metabase that has an enterprise-wide perspective. Data semantics are able to be harmonized which eliminates whole classes of data transformation and reloading business information systems and logic. Again, there is no downside to adoption provided doing more faster at a lower cost and lower risk are the objectives. The U.S. Army uses a starter slide for many of its briefings. The slide’s header contains the string, “BLUF.” Here’s this article’s BLUF. With this approach, you can both save at least 40% on all your business information systems’ development, but also you can create these business information systems within an enterprise-data framework. As stated many times, there is no downside to this approach. It is squarely based on database’s first principle: Define once, use many-times. That’s this article’s BLUF. In the Army, BLUF means: Bottom Line Up Front. It is hoped that careful consideration will be given to this approach and that it will be applied to bring about some reality to the myth of data processing. That is, all things in a blink of an eye. If you are a member of a project team, accomplish the article’s strategies. When you are successful and your manager wants to know why, have him/her read the article. If you are a project manager, make sure your project adopts the methods in this article so you can have a positive impact on your organization. When your boss notices and asks what you are doing differently, give him/her the article to read. In short, keep passing this article to higher levels in your organization until enterprise-wide changes in the business information system development are accomplished. Go to Current Issue | Go to Issue Archive Recent articles by Michael M. Gorman
Michael M. Gorman -
Michael, the President of Whitemarsh Information Systems Corporation, has been involved in database and DBMS for more than 40 years. Michael has been the Secretary of the ANSI Database Languages
Committee for more than 30 years. This committee standardizes SQL. A full list of Whitemarsh's clients and products can be found on the website. Whitemarsh has developed a very comprehensive Metadata CASE/Repository tool, Metabase, that supports enterprise architectures, information systems planning,
comprehensive data model creation and management, and interfaces with the finest code generator on the market, Clarion ( www.SoftVelocity.com). The Whitemarsh website makes available data management books, courses, workshops, methodologies, software, and metrics. Whitemarsh prices
are very reasonable and are designed for the individual, the information technology organization and professional training organizations. Whitemarsh provides free use of its materials for
universities/colleges. Please contact Whitemarsh for assistance in data modeling, data architecture, enterprise architecture, metadata management, and for on-site delivery of data management
workshops, courses, and seminars. Our phone number is (301) 249-1142. Our email address is: mmgorman@wiscorp.com. |