CrawlingStrategies

Crawling Strategies define in which order a crawler should crawl pages on the Internet. This is important when the crawler cannot crawl the entire domain desired due to space/time demands, and so the focus is then on crawling the best (highest quality) pages earlier.

Several crawling strategies are in common use today: PageRank, BreadthFirstCrawling, DepthFirstCrawling.

last edited 2007-03-27 11:29:36 by ErikGraf