Chapter XX: Critical and Future Trends in Data Mining-A Review of Key Data Mining TechnologiesApplications

Chapter XX - Critical and Future Trends in Data Mining A Review of Key Data Mining Technologies/Applications
Data Mining: Opportunities and Challenges
by John Wang (ed)
Idea Group Publishing 2003

	Brought to you by Team-Fly

Jeffrey Hsu, Fairleigh Dickinson University

USA

Every day, enormous amounts of information are generated from all sectors, whether it be business, education, the scientific community, the World Wide Web (WWW), or one of many readily available off-line and online data sources. From all of this, which represents a sizable repository of data and information, it is possible to generate worthwhile and usable knowledge. As a result, the field of Data Mining (DM) and knowledge discovery in databases (KDD) has grown in leaps and bounds and has shown great potential for the future (Han & Kamber, 2001). The purpose of this chapter is to survey many of the critical and future trends in the field of DM, with a focus on those which are thought to have the most promise and applicability to future DM applications.

MAJOR TRENDS IN TECHNOLOGIES AND METHODS: WEB MINING

Web mining is one of the most promising areas in DM, because the Internet and WWW are dynamic sources of information. Web mining is the extraction of interesting and potentially useful patterns and implicit information from artifacts or activity related to the WWW (Etzioni, 1996). The main tasks that comprise Web mining include retrieving Web documents, selection and processing of Web information, pattern discovery in sites and across sites, and analysis of the patterns found (Garofalis, Rastogi, Seshadri & Shim, 1999; Kosala & Blockeel, 2000; Han, Zaiane, Chee, & Chiang, 2000).

Web mining can be categorized into three separate areas: web-content mining, Web-structure mining, and Web-usage mining. Web-content mining is the process of extracting knowledge from the content of documents or their descriptions. This includes the mining of Web text documents, which is a form of resource discovery based on the indexing of concepts, sometimes using agent-based technology. Web-structure mining is the process of inferring knowledge from the links and organizations in the WWW. Finally, Web-usage mining, also known as Web-log mining, is the process of extracting interesting patterns in Web-access logs and other Web-usage information (Borges & Levene, 1999; Kosala & Blockeel, 2000; Madria, Bhowmick, Ng, & Lim, 1999).

Web-content mining is concerned with the discovery of new information and knowledge from web-based data, documents, and pages. According to Kosala and Blockeel (2000), there are two main approaches to Web-content mining: an information retrieval view and a database (DB) view. The information retrieval view is designed to work with both unstructured (free text, such as news stories) or semistructured documents (with both HTML and hyperlinked data), and attempts to identify patterns and models based on an analysis of the documents, using such techniques as clustering, classification, finding text patterns, and extraction rules (Billsus & Pazzani, 1999; Frank, Paynter, Witten, Gutwin & Nevill-Manning, 1998; Nahm & Mooney, 2000). The other main approach, which is to content mine semi-structured documents, uses many of the same techniques as used for unstructured documents, but with the added complexity and challenge of analyzing documents containing a variety of media elements (Crimmins & Smeator, 1999; Shavlik & Elassi-Rad, 1998).

There are also applications that focus on the design of languages, which provide better querying of DBs containing web-based data. Researchers have developed many web-oriented query languages that attempt to extend standard DB query languages such as SQL to collect data from the WWW, e.g., WebLog and WebSQL. The TSIMMIS system (Chawathe et al., 1994) extracts data from heterogeneous and semistructured information sources and correlates them to generate an integrated DB representation of the extracted information (Maarek & Ben Shaul, 1996; Han, 1996; Meldelzon, Mihaila, & Milo, 1996; Merialdo, Atzeni, & Mecca, 1997).

Other applications focus on the building and management of multilevel or multilayered DBs. This suggests a multilevel-DB approach to organizing web-based information. The main idea behind this method is that the lowest level of the DB contains primitive semistructured information stored in various Web repositories, such as hypertext documents. At the higher level(s), metadata or generalizations are extracted from lower levels and organized in structured collections such as relational or object-oriented DBs. Kholsa, Kuhn, and Soparkar (1996) and King and Novak (1996) have done research in this area.

Web-structure mining. Instead of looking at the text and data on the pages themselves, Web-structure mining has as its goal the mining of knowledge from the structure of websites. More specifically, it attempts to examine the structures that exist between documents on a website, such as hyperlinks and other linkages. For instance, links pointing to a document indicate the popularity of the document, while links coming out of a document indicate the richness or perhaps the variety of topics covered in the document. The PageRank (Brin & Page, 1998) and CLEVER (Chakrabarti et al., 1999) methods take advantage of the information conveyed by the links to find pertinent Web pages. Counters of hyperlinks, into and out of documents, retrace the structure of the Web artifacts summarized.

Web-usage mining. Yet another major area in the broad spectrum of Web mining is Web-usage mining. Rather than looking at the content pages or the underlying structure, Web-usage mining is focused on Web user behavior or, more specifically, modeling and predicting how a user will use and interact with the Web. In general, this form of mining examines secondary data, or the data that are derived from the interaction of users (Chen, Park, & Yu, 1996). There are two main thrusts in Web-usage mining: general access-pattern tracking and customized-usage tracking. General access-pattern tracking analyzes Web logs in order to better understand access patterns and trends. Customized-usage tracking analyzes individual trends. Its purpose is to customize websites to users. The information displayed, the depth of the site structure, and the format of the resources can all be dynamically customized for each user over time, based on their patterns of access (Kosala & Blockeel, 2000).

Srivastava, Cooley, Deshpe, and Tan. (2000) have produced a taxonomy of different Web-mining applications and have categorized them into the following types:

• Personalization. The goal here is to produce a more "individualized" experience for a Web visitor, including making recommendations about other pages to visit based on the pages he/she has visited previously.

• System improvement. Performance and speed have always been important factors when it comes to computing systems, and through Web-usage data, it is possible to improve system performance by creating policies and using such methods as load-balancing, Web caching and network transmission.

• Site modification. It is also possible to modify aspects of a site based on user patterns and behavior. After a detailed analysis of a user's activities on a site, it is possible to make design changes and structural modifications to site to enhance a user's satisfaction and the site's usability.

• Business intelligence. Another important application of Web-usage mining is the ability to mine for marketing intelligence information. Buchner and Mulvenna. (1998) used a data hypercube to consolidate Web-usage data together with marketing data to obtain insights with regards to e-commerce.

• Usage characterization. There is a close relationship between DM of Web-usage data and Web-usage characterization research. This area is focused more on such topics as interaction with the browser interface, navigational strategies, the occurrence of certain types of activities, and models of Web usage. Studies in this area include Catledge and Pitkow (1995), Doorenbos, Etzioni, and Weld (1996), and Arlitt and Williamson (1997).

Yet another area that has been gaining interest is agent-based approaches. Agents are intelligent software components that "crawl through" the Internet and collect useful information, much like the way a virus-like worm moves through systems wreaking havoc. Generally, agent-based Web-mining systems can be placed into three main categories: information categorization and filtering, intelligent search agents, and personal agents.

Information filtering/categorization agents try to automatically retrieve, filter, and categorize discovered information by using various information-retrieval techniques. Agents that can be classified in this category include HyPursuit (Weiss et al., 1996) and Bookmark Organizer (BO). Intelligent search agents search the Internet for relevant information and use characteristics of a particular domain to organize and interpret the discovered information. Some of the better known include ParaSite, and FAQ-Finder. Personalized Web agents try to obtain or learn user preferences and discover Web information sources that correspond to these preferences, and possibly those of other individuals with similar interests, using collaborative filtering. Systems in this class include Netperceptions, WebWatcher (Armstrong, Freitag, Joachims, & Mitchell, 1995), and Syskill & Webert (Pazzani, Muramatsu & Billsus, 1996).

	Brought to you by Team-Fly