E-Business Engineers
Grey Colour ThemeBlue Colour ThemeOrange Colour Theme
HomeAbout UsApproachCase StudiesContact UsSitemap

EBE Knowledge Base

Home > SEO Service > Search Engine Optimisation Glossary > Crawler

Crawler

A search engine that is crawler based such as Google, Yahoo, Alta Vista etc uses web spiders (robots – bots) to crawl the intertnet from website to website.

There job is to mainly copy or take a snap shots of each web page then bring the copies back to the main search engine for processing.

Example:
Google has two main crawlers or search robots. These are called:

Freshbot – Regularly crawls the World Wide Web looking for new websites and updated page content.
Deepbot – Exactly how it sounds Deepbot does a deep crawl of web content once a month.



Back   Go to Top

Client Login
Reset Form
EBE News
Silver Surfers Portal ...
Thu, 24 Apr 2008 11:44:00 GMT
Silver Surfers Guide.com is a Web 2.0 style over 50s portal dealing with finance, dating, health, holiday plus lots mo ...
* * * * * * * * * * * * * * * * * * * * * *
NHS Healthcare Databa ...
Fri, 07 Mar 2008 12:08:00 GMT
E-Business Engineers Ltd provides consultancy and database build services to the NHS...
* * * * * * * * * * * * * * * * * * * * * *
Live Tech Feed
Alarm at Google Yahoo ...
Sat, 10 May 2008 00:10:24 GMT
US advocacy groups urge regulators to block any deal Google and Yahoo might strike after a two-week experiment.
* * * * * * * * * * * * * * * * * * * * * *
Burma's emergency tel ...
Fri, 09 May 2008 16:54:11 GMT
Foreign aid workers dedicated to delivering emergency telecoms are prevented from going into Burma.
* * * * * * * * * * * * * * * * * * * * * *