A worldwide flock of Condors: load sharing among workstation clusters
Future Generation Computer Systems - Special issue: resource management in distributed systems
A security architecture for computational grids
CCS '98 Proceedings of the 5th ACM conference on Computer and communications security
The grid: blueprint for a new computing infrastructure
The grid: blueprint for a new computing infrastructure
Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination
NOSSDAV '01 Proceedings of the 11th international workshop on Network and operating systems support for digital audio and video
Chord: A scalable peer-to-peer lookup service for internet applications
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
A scalable content-addressable network
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Wide-area cooperative storage with CFS
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Squirrel: a decentralized peer-to-peer web cache
Proceedings of the twenty-first annual symposium on Principles of distributed computing
Managing security in high-performance distributed computations
Cluster Computing
PUNCH: An architecture for Web-enabled wide-area network-computing
Cluster Computing
Fine-Grain Access Control for Securing Shared Resources in Computational Grids
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
Middleware '01 Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg
Borg: a hybrid protocol for scalable application-level multicast in peer-to-peer networks
NOSSDAV '03 Proceedings of the 13th international workshop on Network and operating systems support for digital audio and video
Matchmaking: Distributed Resource Management for High Throughput Computing
HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Resource Management through Multilateral Matchmaking
HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
Exploiting the synergy between peer-to-peer and mobile ad hoc networks
HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
A secure environment for untrusted helper applications confining the Wily Hacker
SSYM'96 Proceedings of the 6th conference on USENIX Security Symposium, Focusing on Applications of Cryptography - Volume 6
INFOCOM'96 Proceedings of the Fifteenth annual joint conference of the IEEE computer and communications societies conference on The conference on computer communications - Volume 2
Self-Organizing Agents for Grid Load Balancing
GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
General parallel computations on desktop grid and P2P systems
LCR '04 Proceedings of the 7th workshop on Workshop on languages, compilers, and run-time support for scalable systems
Addressing the trust asymmetry problem in grid computing with encrypted computation
LCR '04 Proceedings of the 7th workshop on Workshop on languages, compilers, and run-time support for scalable systems
Accurate autonomous accounting in peer-to-peer Grids
MGC '05 Proceedings of the 3rd international workshop on Middleware for grid computing
A Benchmark Suite for SOAP-based Communication in Grid Web Services
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
A self-organizing flock of Condors
Journal of Parallel and Distributed Computing
Task distribution with a random overlay network
Future Generation Computer Systems - Collaborative and learning applications of grid technology
CycleMeter: detecting fraudulent peers in internet cycle sharing
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
DGRID: a DHT-based resource indexing and discovery scheme for computational grids
ACSW '07 Proceedings of the fifth Australasian symposium on ACSW frontiers - Volume 68
A case for cooperative and incentive-based federation of distributed clusters
Future Generation Computer Systems
GWiQ-P: an efficient decentralized grid-wide quota enforcement protocol
ACM SIGOPS Operating Systems Review
How web community organisation can help grid computing
International Journal of Web Based Communities
Task distribution with a random overlay network
Future Generation Computer Systems - Collaborative and learning applications of grid technology
WaveGrid: a scalable fast-turnaround heterogeneous peer-based desktop grid system
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
A Read-Only Distributed Hash Table
Journal of Grid Computing
An integration of global and enterprise grid computing: gridbus broker and xgrid perspective
GCC'05 Proceedings of the 4th international conference on Grid and Cooperative Computing
GCC'05 Proceedings of the 4th international conference on Grid and Cooperative Computing
A peer-to-peer mechanism for resource location and allocation over the grid
ISPA'04 Proceedings of the Second international conference on Parallel and Distributed Processing and Applications
Wave scheduler: scheduling for faster turnaround time in peer-based desktop grid systems
JSSPP'05 Proceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing
Resource and service discovery in the igrid information service
ICCSA'05 Proceedings of the 2005 international conference on Computational Science and Its Applications - Volume Part III
Coordinated load management in Peer-to-Peer coupled federated grid systems
The Journal of Supercomputing
| Hi-index | 0.00 |
Condor provides high throughput computing by leveraging idle-cycles on off-the-shelf desktop machines. It also supports flocking, a mechanism for sharing resources among Condor pools. Since Condor pools distributed over a wide area can have dynamically changing availability and sharing preferences, the current flocking mechanism based on static configurations can limit the potential of sharing resources across Condor pools. This paper presents a technique for resource discovery in distributed Condor pools using peer-to-peer mechanisms that are self-organizing, fault-tolerant, scalable, and locality-aware. Locality-awareness guarantees that applications are not shipped across long distances when nearby resources are available. Measurements using a synthetic job trace show that self-organized flocking reduces the maximum job wait time in queue for a heavily loaded pool by a factor of 10 compared to without flocking. Simulations of 1000 Condor pools are also presented and the results confirm that our technique discovers and utilizes nearby resources in the physical network.