THE DATA ADMINISTRATION NEWSLETTER – TDAN.com
ROBERT S. SEINER – PUBLISHER

A MetaData Coalition Introduction

The MetaData Coalition was founded by a group of industry-leading vendors aimed at defining a tactical set of standard specifications for the access and interchange of meta data between different software tools.

What follows is an overview of Version 1.0 of the MetaData Interchange Specification (MDIS) initiative taken by the MetaData Coalition. Goals of the MetaData Interchange Specification Initiative

Situation Analysis

The volatility of our global economy and an increasingly competitive business climate are driving companies to leverage their existing resources in new, creative, and more effective ways. Enterprise data, once viewed as merely fodder for the operational systems that ran the day-to-day mechanics of business, is now being recognized not only as one of these valuable resources but as a strategic business asset.

However, as the rate of change continues to accelerate-in response to both business pressures and technological advancement-managing this strategic asset and providing timely, accurate, and manageable access to enterprise data becomes increasingly critical. This need to find faster, more comprehensive and efficient ways to provide access to and manage enterprise data has given rise to a variety of new architectures and approaches, such as data warehouses, distributed client/server computing, and integrated enterprise-wide applications.

In these new environments, meta data, or the information about the enterprise data, is emerging as a critical element in effective data management. Vendors as well as users have been quick to appreciate the value of meta data, but the rapid proliferation of data manipulation and management tools has resulted in almost as many different "flavors" and treatments of meta data as there are tools.

The Challenge

To enable full-scale enterprise data management, different tools must be able to freely and easily access, and in some cases manipulate and update, the meta data created by other tools and stored in a variety of different storage facilities. The only viable mechanism to enable disparate tools from independent vendors to exchange this meta data is to establish at least a minimum common denominator of interchange specifications and guidelines to which the different vendors' tools can comply.

Establishing and adhering to a core set of industry meta data interchange specifications will enable IS managers to select what they perceive as "best of breed" to build the tool infrastructure that best fits their unique environment needs. In choosing the interchange-compliant tools, they can be assured of the accurate and efficient exchange of meta data essential to meeting their users' business information needs.

The MetaData Coalition was established to bring industry vendors and users together to address a variety of difficult problems and issues with regard to exchanging, sharing, and managing meta data. This is intended as a coalition of interested parties with a common focus and shared goals, not a traditional standards body or regulatory group in any way.

Terminology and Basic Assumptions

The MetaData Interchange Specification (MDIS) draws a distinction between:

The Application Metamodel - the tables, etc., used to "hold" the meta data for schemas, etc., for a particular application; for example, the set of tables used to store meta data in Composer may differ significantly from those used by the Bachman Data Analyst.

The MetaData Metamodel - the set of objects that the MetaData Interchange Specification can be used to describe. These represent the information that is common (i.e., represented) by one or more classes of tools, such as data discovery tools, data extraction tools, replication tools, user query tools, database servers, etc. The meta data metamodel should be:

• Independent of any application metamodel
• Character-based so as to be hardware/platform-independent
• Fully qualified so that the definition of each object is uniquely identified

Basic Assumptions

The MetaData Coalition has made the following assumptions:

• Because users' information needs are growing more complex, the IS organization would ideally like the interchange specification to support (to the greatest extent possible) the bidirectional interchange of meta data so that updates can be made in the most natural place. For example, the user might initially specify the source-to-target mapping between a legacy database and a RDBMS target in a CASE tool but, after using a data extraction tool to generate and execute programs to actually move the data, discover that the mapping was somehow incorrect. The most natural place to test out the "fix" to this problem is in the context of the data extraction tool. Once the correction is verified, one updates the metamodel in the CASE tool, rather than having to go to the CASE tool, change the mapping, and trigger the meta data interchange between the CASE tool and the data extraction tool before being able to test the new mapping.
• Vendors would like to support the MetaData Interchange Specification with a minimum amount of additional development. In light of these assumptions, the meta data model must be sufficiently extensible to allow a vendor to store the entire metamodel for any application. In other words, MDIS should provide mechanisms for extending the meta data model so that additional (and possibly encrypted) information can be passed. An example of when a vendor might want encryption is in the case of a tool that generates parameters for invoking some internal routine. Because these parameters might provide other vendors with information regarding what is considered a proprietary part of their tool, the vendor may wish to encrypt these parameters.

If one assumed that all updates to the model occurred in the context of a single tool, e.g., the CASE tool in the example above, the MDIS would not benefit from "carrying along" any of the tool-specific meta data. However, as the above example indicates, this assumption is not the "natural" meta data interchange flow. Consequently, some type of mechanism for providing extensions to the type of information exchanged by the interchange specification is necessary if one hopes to achieve bidirectional interchange between vendor applications.

The MetaData Interchange Framework

For Version 1.0, the MetaData Council is recommending the ASCII-based batch approach so that vendors can implement support for the specification with minimum overhead and the customer benefits from the availability of meta data interchange as quickly as possible.

ASCII Batch Approach

An ASCII Batch approach relies on the ASCII file format that contains the description of the common meta data components and standardized access requirements that make up the interchange specification meta data model. In this approach, the entire ASCII file containing the MDIS schema and access parameters is reloaded whenever a tool accesses the meta data through the specification API.

This approach requires only the addition of a simple import/export function to the tools and would not require updating the tool in the event of meta data model changes, because the most up-to-date schema will always be available through the access framework. This eliminates the amount of retrofitting required to enable tools to remain compliant with the MDIS, because the burden for update stays primarily within the framework itself.

The MetaData Interchange Specification

There are two basic aspects of the proposed specification:

• Those that pertain to the semantics and syntax used to represent the meta data to be exchanged. These items are those that are typically found in a specifications document.
• Those that pertain to some framework in which the specification will be used. This second set of items is two file-based semaphores that are used by the specification's import and export functions to help the user of the specification control consistency.

• Components defining the semantics and syntax that define the specification:

The Metamodel

The MetaData Interchange Specification Metamodel describes the entities and relationships that are used to directly represent meta data in the MDIS. The goal in designing this metamodel is twofold:

• To choose the set of entities and relationships that represents the objects that the majority of tools require.
• To provide some mechanism for extensibility in the case that some tool requires the representation of some other type of object. Section 5 describes the metamodel for Version 1.0 of the MetaData Interchange Specification. In the rest of this document the entities that are directly represented by the specification are referred to as objects in the "public view," while any other meta data stored in the interchange file is referred to as "private meta data" (i.e., tool-specific meta data).

The Mechanism for Extending the Metamodel

The mechanism chosen to provide extensibility to the specification is analogous to the "properties" object found in LISP environments: a character field of arbitrary length that consists of a set of identifiers and a value, where the identifiers are used by the import function of the specification to locate and identify the private meta data in question and the value is the actual meta data itself. Note that because some tools may consider their private meta data proprietary, the actual value for this meta data may be encrypted.

The MDIS Access Framework

Version 1.0 of the MDIS includes information which will support a bidirectional flow of meta data while maintaining meta data consistency.

Three types of information are required:

• Versioning information in the header of the file containing the meta data
• A Tool Profile which describes what type of data elements a tool directly represents and/or updates
• A Configuration Profile which describes the "legal flow of meta data." For example, although source-to-target mapping may be specified in the context of some analysis tool, once that meta data has been exported to ETI*EXTRACT and the mapping is changed because of errors found in expected data, one may want to require that all future changes to mapping originate in ETI*EXTRACT. If the configuration profile is set properly, the import function for ETI*EXTRACT would err off if asked to import a conversion specification from the analysis tool with a version number greater than the version number of the one originally imported from the mapping tool.