A Mixed-Initiative Planning Approach to Exploratory Data Analysis

• Authors: R. St. Amant
• Venue: A Mixed-Initiative Planning Approach to Exploratory Data Analysis
• Year: 1996

Exploratory data anlysis (EDA) has come to play an increasingly important role in statistical analysis. Modern computer-based statistics packages contain a rich set of operations, suitable for almost any EDA application. One can fit lines and higher order functions to relationships, identify and describe clusters, transform and reduce data to meet the specific requirements of a domain, among many other possiblities, in seeking to understand patterns in data. Unfortunately, EDA can be difficult. Conventional statistics packages offer the user hundreds of operations, which must often be combined in lengthy sequences to produce useful results. In addition, the application of these operations often depends on the user''s knowledge of what the data mean. In other words, EDA is too large a problem for a human analyst to solve alone, but complete automation of the process is not feasible either because domain-specific knowledge is required. This dissertation describes an assistant for intelligent data exploratin called AIDE. AIDE is mixed-initiative, autonomously pursuing it''s own goals, but always allowing the user to review an possibly override its decisions. AIDE''s design as a knowledge-based planning system allows it to detect and evaluate suggestive features in the data identify appropreate strategies for extracting the patterns, apply the strategies incrementally, and combine the results in a coherent whole. An experimental evaluation compared the performance of human subjects analyzing data with and without AIDE''s assistance. Although the subjects worked with AIDE for only a couple of hours, each, it clearly influenced the efficiency and coherence of their explorations. Analysis of the experimental results tuned up suggestive evidence that AIDE facilitates data analysis primarily by helping users navigate through the space of relations among variables. This research provides a novel look at automated support for data analysis. Conventional systems tend to take over the talk completely or rely on the user for every step of the analysis. AIDE''s mixed-initiative planning approach provides an alternative in which control changes hands flexibly between the user and the system. This arrangement capitalizes on the strengths of both: the system takes over low-level search and statistical computations, while the user remains responsible for strategic, knowledgeabe guidance of the process.