PVM: a framework for parallel distributed computing
Concurrency: Practice and Experience
The design and implementation of tripwire: a file system integrity checker
CCS '94 Proceedings of the 2nd ACM Conference on Computer and communications security
A worldwide flock of Condors: load sharing among workstation clusters
Future Generation Computer Systems - Special issue: resource management in distributed systems
A security architecture for computational grids
CCS '98 Proceedings of the 5th ACM conference on Computer and communications security
The grid: blueprint for a new computing infrastructure
The grid: blueprint for a new computing infrastructure
Gallop: the benefits of wide-area computing for parallel processing
Journal of Parallel and Distributed Computing
Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination
NOSSDAV '01 Proceedings of the 11th international workshop on Network and operating systems support for digital audio and video
Chord: A scalable peer-to-peer lookup service for internet applications
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
A scalable content-addressable network
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Wide-area cooperative storage with CFS
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Matchmaking: An extensible framework for distributed resource management
Cluster Computing
The Legion Resource Management System
IPPS/SPDP '99/JSSPP '99 Proceedings of the Job Scheduling Strategies for Parallel Processing
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
Middleware '01 Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg
Borg: a hybrid protocol for scalable application-level multicast in peer-to-peer networks
NOSSDAV '03 Proceedings of the 13th international workshop on Network and operating systems support for digital audio and video
Software infrastructure for the I-WAY high-performance distributed computing experiment
HPDC '96 Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing
Matchmaking: Distributed Resource Management for High Throughput Computing
HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Resource Management through Multilateral Matchmaking
HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
A Peer-to-Peer Approach to Resource Location in Grid Environments
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
A Decentralized, Adaptive Replica Location Mechanism
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
XenoSearch: Distributed Resource Discovery in the XenoServer Open Platform
HPDC '03 Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
SHARP: an architecture for secure resource peering
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Grid-computing portals and security issues
Journal of Parallel and Distributed Computing - Scalable web services and architecture
Canon in G Major: Designing DHTs with Hierarchical Structure
ICDCS '04 Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS'04)
A Self-Organizing Flock of Condors
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
ATEC '04 Proceedings of the annual conference on USENIX Annual Technical Conference
Exploiting the synergy between peer-to-peer and mobile ad hoc networks
HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
A secure environment for untrusted helper applications confining the Wily Hacker
SSYM'96 Proceedings of the 6th conference on USENIX Security Symposium, Focusing on Applications of Cryptography - Volume 6
INFOCOM'96 Proceedings of the Fifteenth annual joint conference of the IEEE computer and communications societies conference on The conference on computer communications - Volume 2
Towards a grid-wide intrusion detection system
EGC'05 Proceedings of the 2005 European conference on Advances in Grid Computing
Scribe: a large-scale and decentralized application-level multicast infrastructure
IEEE Journal on Selected Areas in Communications
Automatic grid assembly by promoting collaboration in peer-to-peer grids
Journal of Parallel and Distributed Computing
Resource Sharing Architecture For Cooperative Heterogeneous P2P Overlays
Journal of Network and Systems Management
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
The Journal of Supercomputing
A workflow model for heterogeneous computing environments
Future Generation Computer Systems
Hybrid Resource Discovery Mechanism in Ad Hoc Grid Using Structured Overlay
ARCS '09 Proceedings of the 22nd International Conference on Architecture of Computing Systems
Constructing trusted virtual execution environment in P2P grids
Future Generation Computer Systems
A novel multi-agent reinforcement learning approach for job scheduling in Grid computing
Future Generation Computer Systems
Super-peer-based coordinated service provision
Journal of Network and Computer Applications
Decentralized proactive resource allocation for maximizing throughput of P2P Grid
Journal of Parallel and Distributed Computing
Dynamic grid load sharing with adaptive dissemination protocols
The Journal of Supercomputing
The mini-grid framework: application programming support for ad-hoc, peer-to-peer volunteer grids
GPC'10 Proceedings of the 5th international conference on Advances in Grid and Pervasive Computing
Capacity and load-aware service discovery with service selection in peer-to-peer grids
Future Generation Computer Systems
PonD: dynamic creation of HTC pool on demand using a decentralized resource discovery system
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Resource aggregation and workflow with webcom
HPCC'07 Proceedings of the Third international conference on High Performance Computing and Communications
Future Generation Computer Systems
On reducing energy management delays in disks
Journal of Parallel and Distributed Computing
A proximity-aware load balancing in peer-to-peer-based volunteer computing systems
The Journal of Supercomputing
| Hi-index | 0.00 |
Condor enables high throughput computing using off-the-shelf cost-effective components. It also supports flocking, a mechanism for sharing resources among Condor pools. Since Condor pools distributed over a wide area can have dynamically changing availability and sharing preferences, the current flocking mechanism based on static configurations can limit the potential of sharing resources across Condor pools. This paper presents a technique for resource discovery in distributed Condor pools using peer-to-peer mechanisms that are self-organizing, fault-tolerant, scalable, and locality-aware. Locality-awareness guarantees that applications are not shipped across long distances when nearby resources are available. Measurements using a synthetic job trace show that self-organized flocking reduces the maximum job wait time in queue for a heavily loaded pool by a factor of 10 compared to without flocking. Simulations of 1000 Condor pools are also presented and the results confirm that our technique discovers and utilizes nearby resources in the physical network.