THE DATA ADMINISTRATION NEWSLETTER – TDAN.com
ROBERT S. SEINER – PUBLISHER

The relentless drive to reduce business timeframes is making for strange new bedfellows these days. Every segment of IT feels the pressure, including data warehouse (DW) and business intelligence (BI). Now you might feel DW and BI are far afield of business rules. You shouldn’t.

A current focus in DW/BI is on active or real-time data warehouses. Different people have different interpretations, but all generally emphasize “data freshness” or “data latency.” The thinking is that if you want more payback and direct use of the DW, the data it contains must be as up-to-date as possible, even up to the minute. However, as Richard Hackathorn points out in a recent article¹, the problem is that “managers continue to struggle with the issue of how lower data latency can be of value to the business.”

Why? I believe they’re asking the wrong question. It isn’t the latency of the data that matters, it’s the latency of what you can do with the data that matters.

Hackathorn discusses a very useful tool to explore this. He points out that the business value you derive from the data in a DW/BI context is a function of the elapsed time between some business event occurring (in the real world) and some related business action being taken as a result (also in the real world). As illustrated on the left in Figure 1, there are three milestones within that elapsed time period: data ready for analysis, information delivered and action taken.

Figure 1: The Latency Curve

There are three corresponding kinds of latency, as follows:

• Capture latency. This is where active or real-time DW really focuses – how you get data from operational sources into the DW fast, with all the cleansing, synthesis and consolidation inevitably required.
  
  
• Analysis latency. The assumption is that there must be some value-add in analyzing the data – this is where BI and analytic tools come to bear.
  
  
• Decision latency. This gets to the most interesting part. Let’s look closer.

First, the action to be taken should not be something trivial – for example, merely reporting the results of the consolidation and analysis. Such focus brings us right back to the problem of trying to assess the value of the data itself. That’s a solution in search of a problem. Rather, the focus needs to be on some better-informed decision you make as a result. As Neil Raden and James Taylor put it, “Making information more readily available is important, but making better decisions based on information is what pays the bills.”²

Second, if this decision is truly operational, it’s not something you do infrequently or only once – it’s something that probably needs doing hundreds or thousands of times a day. As discussed in my previous TDAN article, meaningful analysis of ROI requires staying focused on the volume of the decision – not the kind of decision per se. For example, the decision might have to do with cross-sell or up-sell opportunities, or proactive problem resolution, or optimal pricing for a given type of business transaction … or a million other things your business does daily hundreds or thousands of times.

So the fundamental question is how well you deploy analytical insight from data you capture back into the business as operational decisions. Here’s where we hit the bad news. Most organizations have no automated infrastructure to conduct such deployment in any high-volume, rapid, repeatable fashion. You can spend all the money you want on improving the timeliness of data capture and analysis, but if the resulting insight has no quick operational deployment, what have you really gained? That’s like building near-real-time customer information for the Pony Express. Hackathorn correctly writes, “… decision latency will increasingly become the limiting factor in improving response latency.”

He also notes, “Decision latency … is driven primarily by the cultural aspects of how decisions are made in a company.” Human and organizational factors do indeed loom large in the way decisions are currently deployed in most organizations. Specifically, insights arising through analysis of data generally go in one of either two directions for decision deployment, as follows. Neither is effective by today’s standards of scale and urgency.

• The insights are fed to operational business people to act on directly. Without any real infrastructure, the volume and pace are overwhelming. Errors, delays and lost opportunities are inevitable – and quite costly at scale.
  
  
• The insights are fed to IT as “requirements.” That subjects them to all the usual problems of backlogs and priorities. Worse, these are “requirements” that continually change, an anathema for traditional methods.

Just because that’s the way things are mostly done today doesn’t mean it has to be that way. Indeed, business rules, decisioning and related technologies offer a ready, proven solution to decision latency. The potential business value is suggested in the bottom graph in Figure 1.

Ask yourself this. How much does your organization currently spend on DW or BI? Whatever the answer, I believe it should be spending at least that much on business rules and decisioning. In fact, given the current imbalance evident in so many organizations today, I’d say it should probably be spending even more!

For more information on decisioning, business rules, ROI and related ideas you can access free recent webinars in the EDM Summit webinar archives (http://www.edmsummit.com/webinars.php) by Ron Ross (November 13, 2007) and Neil Raden (December 13, 2007).

References:

1. “Active Data Warehousing: From Nice to Necessary” by Richard Hackathorn, Teradata Magazine, June 2006, AR-4835.
   
   
2. Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions, Neil Raden and James Taylor, Prentice-Hall, June 2007, ISBN: 0132347962, p.337. Raden and Taylor are Co-Chairs of the first annual Enterprise Decision Management Summit (October 26-30, Orlando), www.EDMSummit.com.