Object serialization analysis and comparison in Java and .NET
ACM SIGPLAN Notices
Server I/O networks past, present, and future
NICELI '03 Proceedings of the ACM SIGCOMM workshop on Network-I/O convergence: experience, lessons, implications
Aurora: a new model and architecture for data stream management
The VLDB Journal — The International Journal on Very Large Data Bases
Optimizing 10-Gigabit Ethernet for Networks of Workstations, Clusters, and Grids: A Case Study
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
The 8 requirements of real-time stream processing
ACM SIGMOD Record
The CQL continuous query language: semantic foundations and query execution
The VLDB Journal — The International Journal on Very Large Data Bases
Providing resiliency to load variations in distributed stream processing
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Power provisioning for a warehouse-sized computer
Proceedings of the 34th annual international symposium on Computer architecture
The causes of bloat, the limits of health
Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications
SPADE: the system s declarative stream processing engine
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
SODA: An Optimizing Scheduler for Large-Scale Stream-Based Distributed Computer Systems
Middleware '08 Proceedings of the ACM/IFIP/USENIX 9th International Middleware Conference
Communications of the ACM - A Blind Person's Interaction with Technology
COLA: optimizing stream processing applications via graph partitioning
Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware
Queue - Development
Communications of the ACM
ACM SIGOPS Operating Systems Review
Scalable performance of system S for extract-transform-load processing
Proceedings of the 3rd Annual Haifa Experimental Systems Conference
StreamCloud: A Large Scale Data Streaming System
ICDCS '10 Proceedings of the 2010 IEEE 30th International Conference on Distributed Computing Systems
A comparison of high-level full-system power models
HotPower'08 Proceedings of the 2008 conference on Power aware computing and systems
A Roadmap to 100G Ethernet at the enterprise data center
IEEE Communications Magazine
MASCOTS '12 Proceedings of the 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems
Automatic optimization of stream programs via source program operator graph transformations
Distributed and Parallel Databases
| Hi-index | 0.00 |
Building scalable back-end infrastructures for data-centric applications is becoming important. Applications used in data-centres have complex, multilayer software stacks and are required to scale to a large number of nodes. Today, there is increased interest in improving the efficiency of such software stacks. In this paper, we examine the efficiency of such a stack used for distributed stream processing, an important application domain. We use a specific streaming system, Borealis [10], and extensively hand-tune the end-to-end data path. We focus on parts of the stack that are related to intra- and inter-node communication and data exchange, a central component of many software stacks. We find that application-independent code in stream processing middleware employs operations for communication that consume significant amount of CPU cycles and are not strictly necessary. We first categorize these operations based on the protocol function they support. We then proceed to remove these operations by producing a functionally equivalent software stack in terms of application processing. Our results show that restructuring the data path achieves up to 5x higher throughput, reduces energy consumption by up to 60% and saves infrastructure cost by up to 40%. Finally, we project that with 1024-core processors per node, stream processing applications will demand up to 2 TBits/s/node of networking throughput.