DP9: an OAI gateway service for web crawlers

Authors:
Xiaoming Liu;Kurt Maly;Mohammad Zubair;Michael L. Nelson
Affiliations:
Old Dominion University, Norfolk, VA;Old Dominion University, Norfolk, VA;Old Dominion University, Norfolk, VA;NASA Langley Research Center, Hampton, VA
Venue:
Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries
Year:
2002

Citing 1
Cited 6

The open archives initiative: building a low-barrier interoperability framework

Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries

Combined searching of web and oai digital library resources

Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries
GRID Based Federated Digital Library

Proceedings of the 2nd conference on Computing frontiers
Downloading textual hidden web content through keyword queries

Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries
Search Engine Coverage of the OAI-PMH Corpus

IEEE Internet Computing
Metadata harvesting for content-based distributed information retrieval

Journal of the American Society for Information Science and Technology
Generating a meta-DL by federating search on OAI and non-OAI servers

Journal of Intelligent Information Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Many libraries and databases are closed to general-purpose Web crawlers, and they expose their content only through their own search engines. At the same time many researchers attempt to locate technical papers through general-purpose Web search engines. DP9 is an open source gateway service that allows general search engines, (e.g. Google, Inktomi) to index OAI-compliant archives. DP9 does this by providing consistent URLs for repository records, and converting them to OAI queries against the appropriate repository when the URL is requested. This allows search engines that do not support the OAI protocol to index the "deep Web" contained within OAI compliant repositories.