THE DATA ADMINISTRATION NEWSLETTER – TDAN.com
ROBERT S. SEINER – PUBLISHER

Published in TDAN.com April 2005

Within the past year or so there has been some excitement about “social network” web sites – organizations that attempt to establish the value of relationships between individuals for the purposes of exploiting those relationships. For example, there is a web site intended to help establish links between people interested in exploring business opportunities. Once a person has registered, he or she is able to browse through the collection of other registrants for either of two purposes: to see if any acquaintance has already registered, or to find someone with whom he or she would like to make a new connection. There is a third option, which is to invite other acquaintances to register as well.

Each time you link to another individual, you are contributing to a growing network. When you want to meet a “target someone” to whom you are not connected, the application checks to see how closely you are connected to that target. If that target is linked to one of your connections, you can make contact referred through the third party; otherwise, you are on your own to make contact. But once you establish a link to your target, you are then only one hop away from all of your target’s direct connections.

An interesting phenomenon I noticed while I was navigating the set of people registered on this site was that I actually already knew a lot of the people, but mostly segregated into different communities. There were a bunch of folks I knew from my early career working in the High Performance Computing industry, a collection of people I had met while consulting with New Media companies, and a large number from my more recent focus on information management consulting. The interesting part is that some of my acquaintances were connected to me through more than one community.

Okay, so you are all thinking “Six Degrees of Separation,” either the play, movie, or perhaps the late web site that tried to achieve a similar result. Ten years ago the idea may have been premature, but in today’s world, insightful people see the value in connectivity. Our most particular and touchy issues revolve around networks, and I am not talking about your Ethernet cables. Instead, I am referring to the way that people, places, things, etc. all relate to each other in the kinds of ways that emerge as patterns when viewed properly (for a quick scare, terrorists, money laundering, fraud, and illegal drug trade all rely on the use of social networks).

This idea manifests itself in numerous and different ways. Whether we call it “social networks,” “viral marketing,” “spheres of influence,” “phone tree,” or some other trendy catch-phrase, we are talking about ways that groups of things from a large set aggregate into self-similar smaller, strongly connected patterns. It is a very compelling notion when we look at how the network is used, how it can be exploited, and how the value dissipates when a network becomes overloaded.

A simple example revolves around equity markets. One analyst may have an interest in a particular stock, and communicates the tip to her immediate colleagues, all of whom invest in that stock, which drives the price of the stock up slightly. As each of those individuals tips off their friends, the purchase frenzy grows, as does the price of the stock. At some point, the original buyers sell and make their profit, but the “hot tip” takes on a life of its own as it travels through the network. But at some point, the number of individuals in the community overloads, subset communities break out, and as soon as people understand that the increase in the stock price is due to their own pressure, and not related to any intrinsic value, they begin to sell, driving the stock price down.

Irrational exuberance? Perhaps, but consider that this pattern is not unique to the late 1990’s – do a web search for information on the following phrases to learn more: “South Sea Bubble,” “Tulipomania,” and the “Mississippi Company Bubble.” My point is that a pattern exists, some people exploit it and benefit, while those who ignore it get burned.

What does this have to do with data? A few thoughts:

1. The data management industry has invested a lot of intellectual energy in analyzing and formalizing the structure of information, but while a significant amount of knowledge lies embedded within unstructured data, only recently has there been any reasonable focus on extracting knowledge from unstructured content. Even the semi-official name for data mining is “Knowledge Discovery in Databases,” clearly indicating a preference for structured information.
2. There are many different ways to establish a connection between two things. A straightforward one is the simple reference to two concepts within the same chunk of content. This implies that we can scan content for the purposes of identifying and extracting evidence of concepts, determining the name of each concept, as well as providing additional contextual and semantic meaning to each concept. But even more intriguing is delineating the different ways and forms that a relationship between two things may take. Being able to do this requires a means for cataloguing and managing linkage and connectivity.
3. I keep using the term “things” because there are so many different kinds of things that one might care about connecting, it would be unfair to try to enumerate them. In fact, so many things can be viewed as more than one thing (a Chevy is a car is a motor vehicle is a …).

Thought number 1 indicates a need for fully establishing text mining and unstructured data analysis into the realm of information management. With so much information locked inside documents, one can only imagine the knowledge that can be accumulated from that resource. Thought number 2 implies that we can manage the information about the relationships and connectivity that can be extracted – but isn’t this what metadata should be all about (and not just about fancy data dictionaries). Thought number 3 should inspire you to consider how entities semantically live within a taxonomical hierarchy, and how that taxonomy relates to the way you do business. None of these are new ideas, but perhaps they might be more innovative ways to apply them.

Copyright © 2005 Knowledge Integrity, Inc.