New Computing Architectures Driven by RFID-Type Sources

RFID is a powerful new tool. But to use it effectively, manufacturers might have to build new computing infrastructures. Why? RFID deployments can generate enormous amounts of data and flood an existing business's system with more data than it can handle. New computing architectures will be needed by all participants in the value chain to provide information to each other in real or near-real time. It is estimated that Wal-Mart's in-store implementation of RFID alone will generate as much as 7 terabytes (7 trillion bytes) of data per day. Use in the supply chain will generate even more. As a point of relative magnitude, consider that the entire U.S. Library of Congress contains 20 terabytes of data. Wal-Mart could therefore generate as much as the equivalent of a Library of Congress' worth of RFID data every three days. Obviously, such a volume of data could cripple business systems not equipped to handle data volumes of this magnitude and defeat the very purpose of deploying RFID in the first place. What is needed is an advanced technology infrastructure designed to handle this Inescapable Data.

"The good news is that large-scale, real-time computing architectures are not new," claims Mark Palmer, ObjectStore's vice president of complex event processing and RFID. "Instant decision making on high-velocity data streams is a challenge that large trading systems have faced in the financial services industry, military command and control systems have faced, and network management applications have faced in the telecommunications industry." One example of a large-scale real-time architecture is Amazon.com, whose used book and CD system handles real-time inventory updates from hundreds of thousands of third parties, but caches it in a real-time data-management system that services up to 50 million inventory queries an hour during the Christmas season. "Many technologists hear about Amazon.com's system and say: I don't have their problem," continues Palmer, "but even a fraction of the Amazon problem is a huge problem, and the challenge is approaching more and more businesses faster than they realize."

RFID is driving Amazon-like data streams into manufacturing and retailing information systems, which, for the most part, are presently not designed to handle the barrage. "Traditional data management approaches using conventional databases like Oracle cannot handle this new load," claims Palmer. By contrast, real-time computing architectures have ways to handle large volumes of fast-changing data and are now being used more frequently by information technologists. The first RFID Reference Architecture proposed at the MIT's Auto-ID center in 2003 prescribed a real-time, in-memory event database (RIED)," continues Palmer. "The reason for the RIED was simplephysics: Memory is 1,000 times faster than disk, and in-memory event databases are 1,000 times faster than relational databases for RFID data. Although commercial implementations of event-driven databases have been available for years, their deployment has not been pervasiveuntil now."

Complex event processing (CEP) is a new field that deals with the task of processing multiple streams of simple events with the goal of identifying the meaningful events within those streams. An example of a stream of simple events could include a church bell ringing, the appearance of a man in a tuxedo, and rice flying through the air. The related complex event is what one infers from the string of simple events: A wedding is taking place. CEP helps discover complex, inferred events by analyzing other events: the bells, the man in a tux, and the rice flying through the air.

CEP's pioneer David Luckham of Stanford University, the author of The Power of Events, an Introduction to Complex Event Processing in Distributed Enterprise Systems, defines a complex event query language that treats event time and order, in addition to event data, as the basic elements of data processing. According to Gartner, CEP will become a common computing model within 5 to 10 years. But developers aren't sitting stillyou can build CEP systems today in languages such as Java or C++.

One of the main elements of CEP is a new language, the Event Processing Language (EPL). An EPL query is like an SQL query in that it looks for relationships between objects. With and EPL query, the objects are events and the main query metaphor is time and causality, not relationships between data objects. That is, EPLs answer questions such as:A0"Let me know when these 2 events happen, in this order, within 30 seconds of each other."A0SQL is not designed to handle those types of queries. So, this is both a new type of language, but also a new style of computing: event-driven computing.

Event-driven computing will become more prominent as Inescapable Data technologies are integrated into mainstream computing infrastructures. CEP is one of the key elements that extracts intelligence from Inescapable Data streams. Combinations of CEP, event-driven architectures, real-time data management, and federated, distributed middleware, can all be combined to unlock intelligence.A0In many cases, intelligence cannot be gleaned in real time from Inescapable Data streams without all four.

Real-time event databases are designed for streams of data not sets of data (a fundamental computing paradigm shift introduced by the availability of Inescapable Data). Associations and relationships between objects contained within event streams are based on time and proximity, not primary and foreign data keys. Event databases are designed to store dynamic data, not static data. The difference is fundamental and significant, hence the need for new architectures. The developers of event-driven systems will need to adopt these techniques to keep up with Inescapable Data sources such as RFID.

"While the problem isn't new, the tools are new to many, and evolving quickly; venture capital in the United States is heavily investing in them, innovators are beginning to adopt them, and systems are being deployed more quickly because of them," said Palmer. It would not be prudent to force-fit traditional processing architectures to this radically new stream of data. Fortunately, the computing industry wrestled with similar data streams recently and has developed new tools that can now be applied to RFID and other Inescapable Data streams.