The nature of data center traffic: measurements & analysis
Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
On the feasibility of multi-site web search engines
Proceedings of the 18th ACM conference on Information and knowledge management
SLA-driven planning and optimization of enterprise applications
Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering
Micro-pages: increasing DRAM efficiency with locality-aware data placement
Proceedings of the fifteenth edition of ASPLOS on Architectural support for programming languages and operating systems
Toward automatic policy refinement in repair services for large distributed systems
ACM SIGOPS Operating Systems Review
Proceedings of the 7th ACM international conference on Computing frontiers
Characterizing cloud computing hardware reliability
Proceedings of the 1st ACM symposium on Cloud computing
Experiences in initiating concurrency software research efforts
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Rethinking DRAM design and organization for energy-constrained multi-cores
Proceedings of the 37th annual international symposium on Computer architecture
The impact of management operations on the virtualized datacenter
Proceedings of the 37th annual international symposium on Computer architecture
Energy proportional datacenter networks
Proceedings of the 37th annual international symposium on Computer architecture
Fiber optic communication technologies: what's needed for datacenter network operations
IEEE Communications Magazine
Lightning: self-adaptive, energy-conserving, multi-zoned, commodity green cloud storage system
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Providing a cloud network infrastructure on a supercomputer
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
SRCMap: energy proportional storage using dynamic consolidation
FAST'10 Proceedings of the 8th USENIX conference on File and storage technologies
Towards energy proportional cloud for data processing frameworks
SustainIT'10 Proceedings of the First USENIX conference on Sustainable information technology
SLIM: network decongestion for storage systems
WIOV'10 Proceedings of the 2nd conference on I/O virtualization
Vision for cross-layer optimization to address the dual challenges of energy and reliability
Proceedings of the Conference on Design, Automation and Test in Europe
DRAM errors in the wild: technical perspective
Communications of the ACM
Challenges in building scalable virtualized datacenter management
ACM SIGOPS Operating Systems Review
Dremel: interactive analysis of web-scale datasets
Proceedings of the VLDB Endowment
LEGUP: using heterogeneity to reduce the cost of data center network upgrades
Proceedings of the 6th International COnference
A cost comparison of datacenter network architectures
Proceedings of the 6th International COnference
GreenHDFS: towards an energy-conserving, storage-efficient, hybrid Hadoop compute cluster
HotPower'10 Proceedings of the 2010 international conference on Power aware computing and systems
KMV-peer: a robust and adaptive peer-selection algorithm
Proceedings of the fourth ACM international conference on Web search and data mining
Dremel: interactive analysis of web-scale datasets
Communications of the ACM
XRDS: Crossroads, The ACM Magazine for Students - Green Technologies: From Pollution to Pixels
Server operational cost optimization for cloud computing service providers over a time horizon
Hot-ICE'11 Proceedings of the 11th USENIX conference on Hot topics in management of internet, cloud, and enterprise networks and services
Optimal power cost management using stored energy in data centers
Proceedings of the ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
Page placement in hybrid memory systems
Proceedings of the international conference on Supercomputing
Optimizing the datacenter for data-centric workloads
Proceedings of the international conference on Supercomputing
Wide area placement of data replicas for fast and highly available data access
Proceedings of the fourth international workshop on Data-intensive distributed computing
Energy efficiency is not enough, energy proportionality is needed!
DASFAA'11 Proceedings of the 16th international conference on Database systems for advanced applications
Curbing energy cravings in networks: a cross-sectional view across the micro-macro boundary
NOCS '11 Proceedings of the Fifth ACM/IEEE International Symposium on Networks-on-Chip
Benefits and limitations of tapping into stored energy for datacenters
Proceedings of the 38th annual international symposium on Computer architecture
Proceedings of the 38th annual international symposium on Computer architecture
Mobile processors for energy-efficient web search
ACM Transactions on Computer Systems (TOCS)
Optimal power cost management using stored energy in data centers
ACM SIGMETRICS Performance Evaluation Review - Performance evaluation review
A cascade ranking model for efficient ranked retrieval
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Energy-price-driven query processing in multi-center web search engines
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Managing data transfers in computer clusters with orchestra
Proceedings of the ACM SIGCOMM 2011 conference
Dimetrodon: processor-level preventive thermal management via idle cycle injection
Proceedings of the 48th Design Automation Conference
Version-consistent dynamic reconfiguration of component-based distributed systems
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
Automatic management of partitioned, replicated search services
Proceedings of the 2nd ACM Symposium on Cloud Computing
Towards realizing a low cost and highly available datacenter power infrastructure
HotPower '11 Proceedings of the 4th Workshop on Power-Aware Computing and Systems
Evaluating the suitability of mapreduce for surface temperature analysis codes
Proceedings of the second international workshop on Data intensive computing in the clouds
Leveraging stored energy for handling power emergencies in aggressively provisioned datacenters
ASPLOS XVII Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems
Bubble-Up: increasing utilization in modern warehouse scale computers via sensible co-locations
Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture
System-level integrated server architectures for scale-out datacenters
Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture
The datacenter needs an operating system
HotCloud'11 Proceedings of the 3rd USENIX conference on Hot topics in cloud computing
Chapter 2: next generation web search
Search Computing
CC'10/ETAPS'10 Proceedings of the 19th joint European conference on Theory and Practice of Software, international conference on Compiler Construction
Towards a distributed search engine
CIAC'10 Proceedings of the 7th international conference on Algorithms and Complexity
Distributed computing in the 21st century: some aspects of cloud computing
Dependable and Historic Computing
A guided tour of data-center networking
Communications of the ACM
The search for energy-efficient building blocks for the data center
ISCA'10 Proceedings of the 2010 international conference on Computer Architecture
Improving server performance on multi-cores via selective off-loading of OS functionality
ISCA'10 Proceedings of the 2010 international conference on Computer Architecture
Studying hardware and software trade-offs for a real-life web 2.0 workload
ICPE '12 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
VMMB: Virtual Machine Memory Balancing for Unmodified Operating Systems
Journal of Grid Computing
ACM Transactions on Architecture and Code Optimization (TACO)
A Guided Tour through Data-center Networking
Queue - Networks
Camdoop: exploiting in-network aggregation for big data applications
NSDI'12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation
Energy storage in datacenters: what, where, and how much?
Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems
The Cloud Adoption Toolkit: supporting cloud adoption decisions in the enterprise
Software—Practice & Experience
Multiple sub-row buffers in DRAM: unlocking performance and energy improvement opportunities
Proceedings of the 26th ACM international conference on Supercomputing
Energy- and Cost-Efficiency Analysis of ARM-Based Clusters
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
SLA-based Optimization of Power and Migration Cost in Cloud Computing
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Fan-speed-aware scheduling of data intensive jobs
Proceedings of the 2012 ACM/IEEE international symposium on Low power electronics and design
Managing distributed ups energy for effective power capping in data centers
Proceedings of the 39th Annual International Symposium on Computer Architecture
Sorting on GPUs for large scale datasets: A thorough comparison
Information Processing and Management: an International Journal
RainMon: an integrated approach to mining bursty timeseries monitoring data
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
NetPilot: automating datacenter network failure mitigation
Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication
Systematic approach of using power save mode for cloud data processing services
International Journal of Ad Hoc and Ubiquitous Computing
Communications of the ACM
Towards energy-efficient database cluster design
Proceedings of the VLDB Endowment
Failure-aware resource provisioning for hybrid Cloud infrastructure
Journal of Parallel and Distributed Computing
Optimizing datacenter power with memory system levers for guaranteed quality-of-service
Proceedings of the 21st international conference on Parallel architectures and compilation techniques
Proceedings of the 9th international conference on Autonomic computing
NetPilot: automating datacenter network failure mitigation
ACM SIGCOMM Computer Communication Review - Special october issue SIGCOMM '12
Toward on-chip datacenters: a perspective on general trends and on-chip particulars
The Journal of Supercomputing
Internet-based Virtual Computing Environment: Beyond the data center as a computer
Future Generation Computer Systems
Future Generation Computer Systems
RAMZzz: rank-aware dram power management with dynamic migrations and demotions
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Measuring interference between live datacenter applications
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
T: a data-centric cooling energy costs reduction approach for big data analytics cloud
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Parallel job scheduling for power constrained HPC systems
Parallel Computing
Adaptive page grouping for energy efficiency in hybrid PRAM-DRAM main memory
Proceedings of the 2012 ACM Research in Applied Computation Symposium
Communications of the ACM
Responding rapidly to service level violations using virtual appliances
ACM SIGOPS Operating Systems Review
Aggressive Datacenter Power Provisioning with Batteries
ACM Transactions on Computer Systems (TOCS)
Towards an agent-based symbiotic architecture for autonomic management of virtualized data centers
Proceedings of the Winter Simulation Conference
Bridging the gap between applications and networks in data centers
ACM SIGOPS Operating Systems Review
Enhancing performance of failure-prone clusters by adaptive provisioning of cloud resources
The Journal of Supercomputing
Review of performance metrics for green data centers: a taxonomy study
The Journal of Supercomputing
CoScale: Coordinating CPU and Memory System DVFS in Server Systems
MICRO-45 Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture
Vector Extensions for Decision Support DBMS Acceleration
MICRO-45 Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture
Zone-based data striping for cloud storage
IBM Journal of Research and Development
Economics of cloud computing for enterprise IT
IBM Journal of Research and Development
Yank: enabling green data centers to pull the plug
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
A financial cost metric for result caching
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Resource efficient computing for warehouse-scale datacenters
Proceedings of the Conference on Design, Automation and Test in Europe
Proceedings of the 40th Annual International Symposium on Computer Architecture
Integrating microsecond circuit switching into the data center
Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
Datacenter Scale Evaluation of the Impact of Temperature on Hard Disk Drive Failures
ACM Transactions on Storage (TOS)
Fast candidate generation for real-time tweet search with bloom filter chains
ACM Transactions on Information Systems (TOIS)
New wine in old skins: the case for distributed operating systems in the data center
Proceedings of the 4th Asia-Pacific Workshop on Systems
High throughput filtering using FPGA-acceleration
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Scalable high-radix router microarchitecture using a network switch organization
ACM Transactions on Architecture and Code Optimization (TACO)
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
Do not blame users for misconfigurations
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Slice embedding solutions for distributed service architectures
ACM Computing Surveys (CSUR)
Exploring hybrid memory for GPU energy efficiency through software-hardware co-design
PACT '13 Proceedings of the 22nd international conference on Parallel architectures and compilation techniques
Racing and pacing to idle: an evaluation of heuristics for energy-aware resource allocation
Proceedings of the Workshop on Power-Aware Computing and Systems
Introducing service-level awareness in the cloud
Proceedings of the 4th annual Symposium on Cloud Computing
CoMETC: Coordinated management of energy/thermal/cooling in servers
ACM Transactions on Design Automation of Electronic Systems (TODAES)
Quasar: resource-efficient and QoS-aware cluster management
Proceedings of the 19th international conference on Architectural support for programming languages and operating systems
Underprovisioning backup power infrastructure for datacenters
Proceedings of the 19th international conference on Architectural support for programming languages and operating systems
A generalized software framework for accurate and efficient management of performance goals
Proceedings of the Eleventh ACM International Conference on Embedded Software
A self-adapting latency/power tradeoff model for replicated search engines
Proceedings of the 7th ACM international conference on Web search and data mining
QoS-Aware scheduling in heterogeneous datacenters with paragon
ACM Transactions on Computer Systems (TOCS)
Power consumption evaluation of all-optical data center networks
Cluster Computing
Mortar: filling the gaps in data center memory
Proceedings of the 10th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
Monitoring and Autoscaling IaaS Clouds: A Case for Complex Event Processing on Data Streams
UCC '13 Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing
Operational experiences with disk imaging in a multi-tenant datacenter
NSDI'14 Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation
Hi-index | 0.08 |
As computation continues to move into the cloud, the computing platform of interest no longer resembles a pizza box or a refrigerator, but a warehouse full of computers. These new large datacenters are quite different from traditional hosting facilities of earlier times and cannot be viewed simply as a collection of co-located servers. Large portions of the hardware and software resources in these facilities must work in concert to efficiently deliver good levels of Internet service performance, something that can only be achieved by a holistic approach to their design and deployment. In other words, we must treat the datacenter itself as one massive warehouse-scale computer (WSC). We describe the architecture of WSCs, the main factors influencing their design, operation, and cost structure, and the characteristics of their software base. We hope it will be useful to architects and programmers of today's WSCs, as well as those of future many-core platforms which may one day implement the equivalent of today's WSCs on a single board. Table of Contents: Introduction / Workloads and Software Infrastructure / Hardware Building Blocks / Datacenter Basics / Energy and Power Efficiency / Modeling Costs / Dealing with Failures and Repairs / Closing Remarks