A generic framework for data acquisition and transmission

Authors:
Wenruo Bai;Ningbo Wang;Junchao Zhu;Baofeng Zhang
Affiliations:
State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin University, Tianjin 300072, PR China;Wind Power Technology Center of Gansu Electronic Power Company, Lanzhou 730050, PR China;Tianjin Key Laboratory for Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin 300384, PR China;Tianjin Key Laboratory for Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin 300384, PR China
Venue:
Advances in Engineering Software
Year:
2014

Citing 12
Cited 0

Advanced Linux Programming

Advanced Linux Programming
Operating Systems Design and Implementation (3rd Edition)

Operating Systems Design and Implementation (3rd Edition)
VLBI_UDP: An application for transporting VLBI data using the UDP protocol

Future Generation Computer Systems
SDRT: A reliable data transport protocol for underwater sensor networks

Ad Hoc Networks
Extending applications using an advanced approach to DLL injection and API hooking

Software—Practice & Experience
Flower

Bioinformatics
Incremental Information Extraction Using Relational Databases

IEEE Transactions on Knowledge and Data Engineering
A scientific data extraction architecture using classified metadata

The Journal of Supercomputing
GeoDADIS: A framework for the development of geographic data acquisition and dissemination servers

Computers & Geosciences
A formal framework for query decomposition and knowledge integration in data warehouse federations

Expert Systems with Applications: An International Journal
Solutions in XML data exchange

Journal of Computer and System Sciences
A virtual mart for knowledge discovery in databases

Information Systems Frontiers

Quantified Score

Hi-index	0.00

Visualization

Abstract

Online applications such as decision support systems depend on data collected from multiple sources. We develop a generic data acquisition and transmission framework by modularizing the repetitive functions. Other than data acquisition and distribution with necessary transformation, the framework can act as a middle storage when the sources cannot connect to the destination directly. We designed three types of data extractors to accommodate the data acquisition from the file system, the net protocol and the database. After being collected by the extractors, the data is processed by an assembler module, to fit the target's data structure. The data is inserted into the database by a loader module, which gets data from the assembler module. The assembler module and the loader module are controlled by a monitor and controller module. These modules are highly configurable and they form a 3-Level hierarchy. Taking advantage of modular design and shared library technique, the framework is extensible and flexible.