Resource containers: a new facility for resource management in server systems
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
The Spring System: Integrated Support for Complex Real-TimeSystems
Real-Time Systems
Condor-G: A Computation Management Agent for Multi-Institutional Grids
Cluster Computing
Chimera: AVirtual Data System for Representing, Querying, and Automating Data Derivation
SSDBM '02 Proceedings of the 14th International Conference on Scientific and Statistical Database Management
Sun Grid Engine: Towards Creating a Compute Power Grid
CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
An Enabling Framework for Master-Worker Applications on the Computational Grid
HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
BOINC: A System for Public-Resource Computing and Storage
GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
Distributed computing in practice: the Condor experience: Research Articles
Concurrency and Computation: Practice & Experience - Grid Performance
The Anatomy of the Grid: Enabling Scalable Virtual Organizations
International Journal of High Performance Computing Applications
High-Performance Task Distribution for Volunteer Computing
E-SCIENCE '05 Proceedings of the First International Conference on e-Science and Grid Computing
Toward a doctrine of containment: grid hosting with adaptive resource control
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Harnessing grid resources to enable the dynamic analysis of large astronomy datasets
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Pegasus: A framework for mapping complex scientific workflows onto distributed systems
Scientific Programming
The portable batch scheduler and the maui scheduler on linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
Globus toolkit version 4: software for service-oriented systems
NPC'05 Proceedings of the 2005 IFIP international conference on Network and Parallel Computing
Combining batch execution and leasing using virtual machines
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
enabling cross-layer optimizations in storage systems with custom metadata
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
Accelerating large-scale data exploration through data diffusion
DADC '08 Proceedings of the 2008 international workshop on Data-aware distributed computing
Grid resource allocation: allocation mechanisms and utilisation patterns
AusGrid '08 Proceedings of the sixth Australasian workshop on Grid computing and e-research - Volume 82
Toward loosely coupled programming on petascale systems
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Flying Low: Simple Leases with Workspace Pilot
Euro-Par '08 Proceedings of the 14th international Euro-Par conference on Parallel Processing
Towards a Security Model to Bridge Internet Desktop Grids and Service Grids
Euro-Par 2008 Workshops - Parallel Processing
Harnessing parallelism in multicore clusters with the all-pairs and wavefront abstractions
Proceedings of the 18th ACM international symposium on High performance distributed computing
The quest for scalable support of data-intensive workloads in distributed systems
Proceedings of the 18th ACM international symposium on High performance distributed computing
Elastic management of cluster-based services in the cloud
ACDC '09 Proceedings of the 1st workshop on Automated control for datacenters and clouds
Scheduling Strategies for Cycle Scavenging in Multicluster Grid Systems
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
POGGI: Puzzle-Based Online Games on Grid Infrastructures
Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Exploring many task computing in scientific workflows
Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers
Nephele: efficient parallel data processing in the cloud
Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers
OddCI: on-demand distributed computing infrastructure
Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers
Highly scalable genome assembly on campus grids
Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers
GridBot: execution of bags of tasks in multiple grids
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Case studies in storage access by loosely coupled petascale applications
Proceedings of the 4th Annual Workshop on Petascale Data Storage
Experiences with resource provisioning for scientific workflows using Corral
Scientific Programming
Interoperability of BOINC and EGEE
Future Generation Computer Systems
Middleware support for many-task computing
Cluster Computing
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
File-Access Characteristics of Data-Intensive Workflow Applications
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
MRAP: a novel MapReduce-based framework to support HPC analytics applications with access patterns
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Comparison of resource platform selection approaches for scientific workflows
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Data parallelism in bioinformatics workflows using Hydra
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Elastic Cloud Caches for Accelerating Service-Oriented Computations
Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
On the use of clouds for grid resource provisioning
Future Generation Computer Systems
Provenance management in Swift
Future Generation Computer Systems
An elasticity model for High Throughput Computing clusters
Journal of Parallel and Distributed Computing
Towards jungle computing with Ibis/Constellation
Proceedings of the 2011 workshop on Dynamic distributed data-intensive applications, programming abstractions, and systems
Making a case for distributed file systems at Exascale
Proceedings of the third international workshop on Large-scale system and application performance
A model of pilot-job resource provisioning on production grids
Parallel Computing
Grid Interoperability Based on a Formal Design
Journal of Grid Computing
Mediation of Service Overhead in Service-Oriented Grid Architectures
GRID '11 Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing
A Highly Scalable Decentralized Scheduler of Tasks with Deadlines
GRID '11 Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing
AME: an anyscale many-task computing engine
Proceedings of the 6th workshop on Workflows in support of large-scale science
The Aneka platform and QoS-driven resource provisioning for elastic applications on hybrid Clouds
Future Generation Computer Systems
PonD: dynamic creation of HTC pool on demand using a decentralized resource discovery system
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Resource Management for Elastic Cloud Workflows
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Reducing Complexity in Management of eScience Computations
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
A Workflow-Aware Storage System: An Opportunity Study
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
A cost-effective cloud computing framework for accelerating multimedia communication simulations
Journal of Parallel and Distributed Computing
GJMF - a composable service-oriented grid job management framework
Future Generation Computer Systems
Design and analysis of data management in scalable parallel scripting
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Sigiri: uniform resource abstraction for grids and clouds
Concurrency and Computation: Practice & Experience
Design and implementation of GXP make - A workflow system based on make
Future Generation Computer Systems
Designing and Deploying a Scientific Computing Cloud Platform
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
Adapting scientific workflow structures using multi-objective optimization strategies
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
A task routing approach to large-scale scheduling
Future Generation Computer Systems
Provisioning data analytic workloads in a cloud
Future Generation Computer Systems
Hobbes: composition and virtualization as the foundations of an extreme-scale OS/R
Proceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers
Performance evaluation of parallel strategies in public clouds: A study with phylogenomic workflows
Future Generation Computer Systems
Cloud computing for fast prediction of chemical activity
Future Generation Computer Systems
A family of heuristics for agent-based elastic Cloud bag-of-tasks concurrent scheduling
Future Generation Computer Systems
Autonomic load balancing mechanisms in the P2P desktop grid
Proceedings of the 2013 ACM Cloud and Autonomic Computing Conference
SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascale
Proceedings of the High Performance Computing Symposium
Multiple objective scheduling of HPC workloads through dynamic prioritization
Proceedings of the High Performance Computing Symposium
Using Kestrel and XMPP to Support the STAR Experiment in the Cloud
Journal of Grid Computing
Data-Intensive Cloud Computing: Requirements, Expectations, Challenges, and Solutions
Journal of Grid Computing
EventWave: programming model and runtime support for tightly-coupled elastic cloud applications
Proceedings of the 4th annual Symposium on Cloud Computing
An agent based network resource planner for workflow applications
Multiagent and Grid Systems - Agent Based Computing: From Model to Implementation
Hi-index | 0.00 |
To enable the rapid execution of many tasks on compute clusters, we have developed Falkon, a Fast and Light-weight tasK executiON framework. Falkon integrates (1) multi-level scheduling to separate resource acquisition (via, e.g., requests to batch schedulers) from task dispatch, and (2) a streamlined dispatcher. Falkon's integration of multi-level scheduling and streamlined dispatchers delivers performance not provided by any other system. We describe Falkon architecture and implementation, and present performance results for both microbenchmarks and applications. Microbenchmarks show that Falkon throughput (487 tasks/sec) and scalability (to 54,000 executors and 2,000,000 tasks processed in just 112 minutes) are one to two orders of magnitude better than other systems used in production Grids. Large-scale astronomy and medical applications executed under Falkon by the Swift parallel programming system achieve up to 90% reduction in end-to-end run time, relative to versions that execute tasks via separate scheduler submissions.