SEO Forums  

Go Back   SEO Forums > Search Engines > Google > Google Optimization
FAQ Members List Calendar Search Today's Posts Mark Forums Read

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 06-04-2008, 11:00 AM
tecktalk tecktalk is offline
SubmitShop Newbie
 
Join Date: May 2008
Posts: 4
Default How Google Indexes our site

A "bot" is a piece of software from a search engine that is built to go through every page of your site, categorize it, and place it into a database.
Google has three well known bots: The Adsense bot, the Freshbot and the DeepCrawl.
The Adsense bot is used for publishers who have Adsense on their sites. As soon as a new page is created, the JavaScript within the Adsense code sends a message to the Adsense bot, and it will come within 15 minutes to index the page so that it can serve up the most relevant ads.

The Freshbot crawls the most popular pages on our website. Sites like Amazon.com and CNN.com have pages that are crawled every ten minutes, since Google has learned that those pages have that amount of frequent changes. A typical site should expect to have a freshbot visit every 1 to 14 days, depending on how popular those pages are.Freshbot visit is that it finds all of the deeper links in your site. It places those links into a database so that when the DeepCrawl occurs, it has a reference.Once a month, the DeepCrawl bot visits your site and goes over all the links found by the Freshbot. This is the reason why it can take up to a month for your entire site to be indexed in Google - even with the addition of a Google Sitemap.
Reply With Quote
  #2  
Old 07-28-2008, 07:20 PM
pageupseo pageupseo is offline
SubmitShop Newbie
 
Join Date: Jul 2008
Posts: 6
Default

Thanks for this info
Reply With Quote
  #3  
Old 09-10-2008, 08:08 AM
andylewis andylewis is offline
SubmitShop Newbie
 
Join Date: Sep 2008
Posts: 3
Default

Thanks for providing us such a valuable information on google crawlers.
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is difference between site map and Google site map joshpe Google Optimization 3 12-17-2009 06:48 AM
Google launches Government-only search site jagan SEO News & Articles 0 06-16-2006 06:33 AM
Yahoo Site Map like Google Site Map jagan Yahoo 0 04-03-2006 02:46 PM
Google Sitemap www vs non-www versions of a site megri Google Optimization 2 01-01-2006 07:08 AM


All times are GMT. The time now is 07:03 PM.


Powered by vBulletin® Version 3.5.7
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.