Communications of the ACM
IEEE Transactions on Computers
On the validity of trace-driven simulation for multiprocessors
ISCA '91 Proceedings of the 18th annual international symposium on Computer architecture
Synthetic Traces for Trace-Driven Simulation of Cache Memories
IEEE Transactions on Computers
Scalable problems and memory-bounded speedup
Journal of Parallel and Distributed Computing
Application scheduling and processor allocation in multiprogrammed parallel processing systems
Performance Evaluation - Special issue: performance modeling of parallel processing systems
The elusive goal of workload characterization
ACM SIGMETRICS Performance Evaluation Review
The impact of job arrival patterns on parallel scheduling
ACM SIGMETRICS Performance Evaluation Review
Simulation Modeling and Analysis
Simulation Modeling and Analysis
A Comparison of Trace-Sampling Techniques for Multi-Megabyte Caches
IEEE Transactions on Computers
A hierarchical approach to workload characterization for parallel systems
HPCN Europe '95 Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking
The ANL/IBM SP Scheduling System
IPPS '95 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
A Batch Scheduler for the Intel Paragon MPP System with a Non-contiguous Node Allocation Algorithm
IPPS '96 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Packing Schemes for Gang Scheduling
IPPS '96 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Workload Evolution on the Cornell Theory Center IBM SP2
IPPS '96 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Towards Convergence in Job Schedulers for Parallel Supercomputers
IPPS '96 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
IPPS '97 Proceedings of the Job Scheduling Strategies for Parallel Processing
Improved Utilization and Responsiveness with Gang Scheduling
IPPS '97 Proceedings of the Job Scheduling Strategies for Parallel Processing
IPPS/SPDP '98 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Metrics and Benchmarking for Parallel Job Scheduling
IPPS/SPDP '98 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
A parallel workload model and its implications for processor allocation
HPDC '97 Proceedings of the 6th IEEE International Symposium on High Performance Distributed Computing
What is worth learning from parallel workloads?: a user and session based analysis
Proceedings of the 19th annual international conference on Supercomputing
Adaptive Parallel Job Scheduling with Flexible Coscheduling
IEEE Transactions on Parallel and Distributed Systems
ICPADS '06 Proceedings of the 12th International Conference on Parallel and Distributed Systems - Volume 1
LOMARC: Lookahead Matchmaking for Multiresource Coscheduling on Hyperthreaded CPUs
IEEE Transactions on Parallel and Distributed Systems
Engineering grid applications and middleware for high performance
WOSP '07 Proceedings of the 6th international workshop on Software and performance
Analysis and modeling of job arrivals in a production grid
ACM SIGMETRICS Performance Evaluation Review
A Co-Plot analysis of logs and models of parallel workloads
ACM Transactions on Modeling and Computer Simulation (TOMACS)
Characterizing resource availability in enterprise desktop grids
Future Generation Computer Systems
Using queue structures to improve job reliability
Proceedings of the 16th international symposium on High performance distributed computing
Locality of sampling and diversity in parallel system workloads
Proceedings of the 21st annual international conference on Supercomputing
Modeling correlated workloads by combining model based clustering and a localized sampling algorithm
Proceedings of the 21st annual international conference on Supercomputing
Inter-operating grids through delegated matchmaking
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Future Generation Computer Systems
The performance of bags-of-tasks in large-scale distributed systems
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
Discovering performance bounds for grid scheduling by using evolutionary multiobjective optimization
Proceedings of the 10th annual conference on Genetic and evolutionary computation
Adaptive work-stealing with parallelism feedback
ACM Transactions on Computer Systems (TOCS)
Inter-operating grids through Delegated MatchMaking
Scientific Programming - Large-Scale Programming Tools and Environments
A multicriteria approach to two-level hierarchy scheduling in grids
Journal of Scheduling
DGSim: Comparing Grid Resource Management Architectures through Trace-Based Simulation
Euro-Par '08 Proceedings of the 14th international Euro-Par conference on Parallel Processing
Achieving efficiency, quality of service and robustness in multi-organizational Grids
Journal of Systems and Software
PastryGrid: decentralisation of the execution of distributed applications in desktop grid
Proceedings of the 6th international workshop on Middleware for grid computing
Performance analysis of allocation policies for interGrid resource provisioning
Information and Software Technology
Workload dynamics on clusters and grids
The Journal of Supercomputing
How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
Trace-based evaluation of job runtime and queue wait time predictions in grids
Proceedings of the 18th ACM international symposium on High performance distributed computing
Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters
Proceedings of the 18th ACM international symposium on High performance distributed computing
Resource Allocation Using Virtual Clusters
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Modeling Job Arrival Process with Long Range Dependence and Burstiness Characteristics
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
POGGI: Puzzle-Based Online Games on Grid Infrastructures
Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Future Generation Computer Systems
Identifying job migration characteristics in decentralized grid scheduling scenarios
PDCS '07 Proceedings of the 19th IASTED International Conference on Parallel and Distributed Computing and Systems
Adaptive job scheduling via predictive job resource allocation
JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
A data locality aware online scheduling approach for I/O-intensive jobs with file sharing
JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
Modeling job arrivals in a data-intensive grid
JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
On grid performance evaluation using synthetic workloads
JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
CROWNBench: a grid performance testing system using customizable synthetic workload
APWeb'08 Proceedings of the 10th Asia-Pacific web conference on Progress in WWW research and development
HiPC'08 Proceedings of the 15th international conference on High performance computing
Probabilistic performance modeling of virtualized resource allocation
Proceedings of the 7th international conference on Autonomic computing
Deadline-guarantee-enhanced co-allocation for parameter sweep application in grid
ICC'09 Proceedings of the 2009 IEEE international conference on Communications
Time and cost trade-off management for scheduling parallel applications on Utility Grids
Future Generation Computer Systems
NBTI-aware DVFS: a new approach to saving energy and increasing processor lifetime
Proceedings of the 16th ACM/IEEE international symposium on Low power electronics and design
A Realistic Integrated Model of Parallel System Workloads
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
An Analysis of Traces from a Production MapReduce Cluster
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
Morco: middleware framework for long-running multi-component applications on batch grids
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
A hybrid Markov chain model for workload on parallel computers
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Scheduling parallel applications on utility grids: time and cost trade-off management
ACSC '09 Proceedings of the Thirty-Second Australasian Conference on Computer Science - Volume 91
Instability in parallel job scheduling simulation: the role of workload flurries
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
The importance of complete data sets for job scheduling simulations
JSSPP'10 Proceedings of the 15th international conference on Job scheduling strategies for parallel processing
On/off-line prediction applied to job scheduling on non-dedicated NOWs
Journal of Computer Science and Technology - Special issue on natural language processing
Towards a profound analysis of bags-of-tasks in parallel systems and their performance impact
Proceedings of the 20th international symposium on High performance distributed computing
Service control with the preemptive parallel job scheduler Scojo-PECT
Cluster Computing
Modeling and synthesizing task placement constraints in Google compute clusters
Proceedings of the 2nd ACM Symposium on Cloud Computing
A similarity measure for time, frequency, and dependencies in large-scale workloads
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Adaptive Executions of Multi-Physics Coupled Applications on Batch Grids
Journal of Grid Computing
Adaptive resource remapping through live migration of virtual machines
ICA3PP'11 Proceedings of the 11th international conference on Algorithms and architectures for parallel processing - Volume Part I
Measuring TeraGrid: workload characterization for a high-performance computing federation
International Journal of High Performance Computing Applications
Parallel computer workload modeling with markov chains
JSSPP'04 Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing
Workload characteristics of a multi-cluster supercomputer
JSSPP'04 Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing
LOMARC — lookahead matchmaking for multi-resource coscheduling
JSSPP'04 Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing
Modeling user runtime estimates
JSSPP'05 Proceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing
Pitfalls in parallel job scheduling evaluation
JSSPP'05 Proceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing
Workload analysis of a large-scale key-value store
Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems
The Journal of Supercomputing
iSPD: an iconic-based modeling simulator for distributed grids
Proceedings of the 45th Annual Simulation Symposium
Coordinated load management in Peer-to-Peer coupled federated grid systems
The Journal of Supercomputing
Coordinated rescheduling of Bag-of-Tasks for executions on multiple resource providers
Concurrency and Computation: Practice & Experience
Failure-aware resource provisioning for hybrid Cloud infrastructure
Journal of Parallel and Distributed Computing
The Journal of Supercomputing
Efficient resource management for virtual desktop cloud computing
The Journal of Supercomputing
High throughput computing over peer-to-peer networks
Future Generation Computer Systems
Concurrency and Computation: Practice & Experience
Heterogeneity and dynamicity of clouds at scale: Google trace analysis
Proceedings of the Third ACM Symposium on Cloud Computing
The characteristics and performance of groups of jobs in grids
Euro-Par'07 Proceedings of the 13th international Euro-Par conference on Parallel Processing
Coarse-grain time slicing with resource-share control in parallel-job scheduling
HPCC'07 Proceedings of the Third international conference on High Performance Computing and Communications
Job scheduling using successive linear programming approximations of a sparse model
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
Scheduling mapreduce jobs in HPC clusters
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
A job scheduling approach for multi-core clusters based on virtual malleability
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
Bacterial foraging based hyper-heuristic for resource scheduling in grid computing
Future Generation Computer Systems
A User-Based Model of Grid Computing Workloads
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
Assessing Green Strategies in Peer-to-Peer Opportunistic Grids
Journal of Grid Computing
Resource provisioning based on lease preemption in InterGrid
ACSC '11 Proceedings of the Thirty-Fourth Australasian Computer Science Conference - Volume 113
Storage and performance optimization of long tail key access in a social network
Proceedings of the 3rd International Workshop on Cloud Data and Platforms
Accelerating population-based search heuristics by adaptive resource allocation
Proceedings of the 15th annual conference on Genetic and evolutionary computation
Configurable performance analysis and evaluation framework for cloud systems
International Journal of Information and Communication Technology
Workload resampling for performance evaluation of parallel job schedulers
Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
Characterization and modeling of PIDX parallel I/O for performance optimization
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Preference---Based Matchmaking of Grid Resources with CP---Nets
Journal of Grid Computing
Deconstructing Amazon EC2 Spot Instance Pricing
ACM Transactions on Economics and Computation
Energy and carbon-efficient placement of virtual machines in distributed cloud data centers
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
Improving user QoS by relaxing resource reservation policy in high-performance grid environments
International Journal of Grid and Utility Computing
TLA: Temporal look-ahead processor allocation method for heterogeneous multi-cluster systems
Journal of Parallel and Distributed Computing
International Journal of Computational Science and Engineering
| Hi-index | 0.00 |
The analysis of workloads is important for understanding how systems are used. In addition, workload models are needed as input for the evaluation of new system designs, and for the comparison of system designs. This is especially important in costly large-scale parallel systems. Luckily, workload data are available in the form of accounting logs. Using such logs from three different sites, we analyze and model the job-level workloads with an emphasis on those aspects that are universal to all sites. As many distributions turn out to span a large range, we typically first apply a logarithmic transformation to the data, and then fit it to a novel hyper-Gamma distribution or one of its special cases. This is a generalization of distributions proposed previously, and leads to good goodness-of-fit scores. The parameters for the distribution are found using the iterative EM algorithm. The results of the analysis have been codified in a modeling program that creates a synthetic workload based on the results of the analysis.