A single-pass online data mining algorithm combined with control theory with limited memory in dynamic data streams

Authors:
Yanxiang He;Naixue Xiong;Xavier Défago;Yan Yang;Jing He
Affiliations:
The State Key Lab of Software Engineering, Computer School, Wuhan University, P.R. China;The State Key Lab of Software Engineering, Computer School, Wuhan University, P.R. China;School of Information Science, Japan Advanced Institute of Science and Technology (JAIST), Japan;Computer School, Wuhan university of science and technology, P.R. China;Department of Computer Science, Utah State University, Logan, UT
Venue:
GCC'05 Proceedings of the 4th international conference on Grid and Cooperative Computing
Year:
2005

Citing 38
Cited 0

The space complexity of approximating the frequency moments

STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
Online aggregation

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Tracking join and self-join sizes in limited storage

PODS '99 Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
A framework for measuring changes in data characteristics

PODS '99 Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Random sampling techniques for space efficient online computation of order statistics of large datasets

SIGMOD '99 Proceedings of the 1999 ACM SIGMOD international conference on Management of data
The space complexity of approximating the frequency moments

Journal of Computer and System Sciences
Efficient mining of emerging patterns: discovering trends and differences

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Synopsis data structures for massive data sets

Proceedings of the tenth annual ACM-SIAM symposium on Discrete algorithms
Testing and spot-checking of data streams (extended abstract)

SODA '00 Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms
Mining high-speed data streams

Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
On computing correlated aggregates over continual data streams

SIGMOD '01 Proceedings of the 2001 ACM SIGMOD international conference on Management of data
Mining time-changing data streams

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Scalable flow control for multicast ABR services in ATM networks

IEEE/ACM Transactions on Networking (TON)
Mining data streams under block evolution

ACM SIGKDD Explorations Newsletter
Models and issues in data stream systems

Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Maintaining stream statistics over sliding windows: (extended abstract)

SODA '02 Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms
Continuous queries over data streams

ACM SIGMOD Record
Using Control Theory to Achieve Service Level Objectives In Performance Management

Real-Time Systems
Mining Very Large Databases

Computer
Computing Iceberg Queries Efficiently

VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Surfing Wavelets on Streams: One-Pass Summaries for Approximate Aggregate Queries

Proceedings of the 27th International Conference on Very Large Data Bases
Finding Frequent Items in Data Streams

ICALP '02 Proceedings of the 29th International Colloquium on Automata, Languages and Programming
Frequency Estimation of Internet Packet Streams with Limited Space

ESA '02 Proceedings of the 10th Annual European Symposium on Algorithms
A simple algorithm for finding frequent elements in streams and bags

ACM Transactions on Database Systems (TODS)
What's hot and what's not: tracking most frequent items dynamically

Proceedings of the twenty-second ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
One-Pass Wavelet Decompositions of Data Streams

IEEE Transactions on Knowledge and Data Engineering
An Approximate L1-Difference Algorithm for Massive Data Streams

FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
Clustering data streams

FOCS '00 Proceedings of the 41st Annual Symposium on Foundations of Computer Science
Stable distributions, pseudorandom generators, embeddings and data stream computation

FOCS '00 Proceedings of the 41st Annual Symposium on Foundations of Computer Science
A Rate-Based End-to-End Multicast Congestion Control Protocol

ISCC '00 Proceedings of the Fifth IEEE Symposium on Computers and Communications (ISCC 2000)
Online Data Mining for Co-Evolving Time Sequences

ICDE '00 Proceedings of the 16th International Conference on Data Engineering
Distributed top-k monitoring

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Fjording the Stream: An Architecture for Queries Over Streaming Sensor Data

ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Single-Pass Algorithms for Mining Frequency Change Patterns with Limited Space in Evolving Append-Only and Dynamic Transaction Data Streams

EEE '04 Proceedings of the 2004 IEEE International Conference on e-Technology, e-Commerce and e-Service (EEE'04)
Multi-dimensional regression analysis of time-series data streams

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
How to summarize the universe: dynamic maintenance of quantiles

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
A framework for clustering evolving data streams

VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Data transmission rate control in computer networks using neural predictive networks

ISPA'04 Proceedings of the Second international conference on Parallel and Distributed Processing and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper addresses a fundamental problem that arises in data streaming scenarios, namely, today’s data mining is ill-equipped to handle data streams effectively, and pays little attention to the network stability and the fast response [36]. To the question, we present a control-theoretic explicit rate (ER) online data mining control algorithm (ODMCA) to regulate the sending rate of mined data, which accounts for the main memory occupancies of terminal nodes. The proposed method uses a distributed proportional integrative plus derivative controller combined with data-mining, where the control parameters can be designed to ensure the stability of the control loop in terms of sending rate of mined data. We further analyze the theoretical aspects of the proposed algorithm, and simulation results show the efficiency of our scheme in terms of high main memory occupancy, fast response of the main memory occupancy as well as of the controlled sending rates.