Intelligent Spider for Internet Searching

  • Authors:
  • Hsinchun Chen;Yi-Ming Chung;Marshall Ramsey

  • Affiliations:
  • -;-;-

  • Venue:
  • HICSS '97 Proceedings of the 30th Hawaii International Conference on System Sciences: Information Systems Track—Internet and the Digital Economy - Volume 4
  • Year:
  • 1997

Quantified Score

Hi-index 0.00

Visualization

Abstract

As the World-Wide Web ( WWW) basedInternet services become more popular, informationoverload also becomes a pressing research problem.Difficulties with searching on Internet get worse asthe amount of information that available on the internetincreases. A scalable approach to support Internetsearch is critical to the success of Internet services andother current or future National Information Infrastructure(NII) applications. A new approach to buildintelligent personal spider (agent), which is based onautomatic textual analysis of Internet documents, isproposed in this paper. Best first search and geneticalgorithm have been tested to develop the intelligentspider. These personal spiders are able to dynamicallyand intelligently analyze the contents of the users selectedhomepages as the starting point to search forthe most relevant homepages based on the links andindexing.An intelligent spider must have the capability tomake adjustments according to progress of searchingin order to be an intelligent agent. However, the currentsearching engines do not have the communicationbetween the users and the robots. The spiderpresented in this paper use Java to develop the userinterface such that the users can adjust the controlparameters according to the progress and observe theintermediate results. The performances of the geneticalgorithm based and best first search based spiders arealso reported.