Information Retrieval on an SCI-Based PC Cluster

Authors:
Sang-Hwa Chung;Hyuk-Chul Kwon;Kwang Ryel Ryu;Yoojin Chung;Hankook Jang;Cham-Ah Choi
Affiliations:
School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea shchung@hyowon.pusan.ac.kr;School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea;School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea;School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea;School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea;School of Electrical and Computer Engineering, Pusan National University, Pusan 609-735, Korea
Venue:
The Journal of Supercomputing
Year:
2001

Citing 9
Cited 1

A generic machine for parallel information retrieval

Information Processing and Management: an International Journal
Network design for the implementation of text searching using a multicomputer

Information Processing and Management: an International Journal - Special issue on parallel processing and information retrieval
Information retrieval on the connection machine: 1 to 8192 gigabytes

Information Processing and Management: an International Journal - Special issue on parallel processing and information retrieval
U-Net: a user-level network interface for parallel and distributed computing

SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
A high-performance, portable implementation of the MPI message passing interface standard

Parallel Computing
Information Retrieval

Information Retrieval
Fast Messages: Efficient, Portable Communication for Workstation Clusters and MPPs

IEEE Parallel & Distributed Technology: Systems & Technology
Myrinet: A Gigabit-per-Second Local Area Network

IEEE Micro
Active Messages: a Mechanism for Integrated Communication and

Active Messages: a Mechanism for Integrated Communication and

Efficient parallel Text Retrieval techniques on Bulk Synchronous Parallel (BSP)/Coarse Grained Multicomputers (CGM)

The Journal of Supercomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This article presents an efficient parallel information retrieval (IR) system which provides fast information service for the Internet users on low-cost high-performance PC-NOW environment. The IR system is implemented on a PC cluster based on the scalable coherent interface (SCI), a powerful interconnecting mechanism for both shared memory models and message-passing models. In the IR system, the inverted-index file (IIF) is partitioned into pieces using a greedy declustering algorithm and distributed to the cluster nodes to be stored on each node's hard disk. For each incoming user's query with multiple terms, terms are sent to the corresponding nodes which contain the relevant pieces of the IIF to be evaluated in parallel. The IR system is developed using a distributed-shared memory (DSM) programming technique based on the SCI. According to the experiments, the IR system outperforms an MPI-based IR system using Fast Ethernet as an interconnect. Speed-up of up to 5.0 was obtained with an 8-node cluster in processing each query on a 500,000-document IIF.