The stream machine: a data flow architecture for real-time applications
ICSE '85 Proceedings of the 8th international conference on Software engineering
A preliminary architecture for a basic data-flow processor
ISCA '75 Proceedings of the 2nd annual symposium on Computer architecture
| Hi-index | 12.05 |
We present the design and development of a flow-based software system that became necessary for the study of a class of options-based price-estimators in the financial markets. Because a combination of factors-cost, reliability, uniformity and convention-made it virtually impossible to obtain historical options data, we developed a data-flow system to capture, process and analyze streaming data over a period of over a year. The system utilizes distributed processing nodes with checkpointing to process input data streams and reliably compute a variety of estimator updates for studying relationships between equity prices and the functions of corresponding option-related variables. The flow-based architecture is designed to support the high-volume data characteristics of the options market, along with the severely taxing computational requirements of hypothetical option-pricing so that experimental investigation of novel estimators becomes possible with current and historical data. Features such as high-volume stream-processing, intermediate checkpointing and load distribution make the design viable for more general streaming data processing applications. .