Research Environment for Data Analysis Tool Allocators

  • Authors:
  • Jeff Wilkinson;Robert Levinson

  • Affiliations:
  • Inprise Corporation, Scotts Valley, CA. jwilkinson@netsafe.net;Department of Computer Science, University of California, Santa Cruz, CA 95060. levinson@cse.ucsc.edu

  • Venue:
  • Applied Intelligence
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

Intelligent data analysis implies the reasoned application of autonomousor semi-autonomous tools to data sets drawn from problem domains. Automation of this process of reasoning about analysis (based on factorssuch as available computational resources, cost of analysis, risk offailure, lessons learned from past errors, and tentative structural modelsof problem domains) is highly non-trivial. By casting the problem ofreasoning about analysis (MetaReasoning) as yet another data analysisproblem domain, we have previously [R. Levinson and J. Wilkinson, inAdvances in Intelligent Data Analysis, edited by X. Liu, P. Cohen, and M. Berthold, volume LNCS 1280, Springer-Verlag, Berlin, pp. 89–100, 1997] presented a design framework, MetaReasoning for Data Analysis Tool Allocation (MRDATA).Crucial to this framework is the ability of a Tool Allocator to trackresource consumption (i.e. processor time and memory usage) by the Toolsit employs, as well as the ability to allocate measured quantities ofresources to these Tools. In order to test implementations of the MRDATAdesign, we now implement a Runtime Environment for Data Analysis ToolAllocation, RE:DATA. Tool Allocators run as processes under RE:DATA, areallotted system resources, and may use these resources to run their Toolsas spawned sub-processes. We also present designs of native RE:DATAimplementations of analysis tools used by MRDATA: K-Nearest NeighborTables, Regression Trees, Interruptible (“Any-Time”) Regression Trees, and “Hierarchy Diffusion” Temporal Difference Learners. Preliminary results are discussed and techniques for integration with non-native tools are explored.