First we are disscussing
about a "bot" is a piece of software
from a search engine that is built to go through
every page of your site, included at particular
category, and stores into a database.
Google has three well
known bots: "The Adsense Bot", "the
Fresh Bot" and "the Deep Crawl ".
The Adsense Bot is used
for publishers who have Adsense on their sites.
within the Adsense code sends a message to the
Adsense bot, and it
will come within 15 minutes to index the page
so that it can serve up the most relevant ads.
But, we are only mainly
concerned about the DeepCrawl and the Freshbot.
The Freshbot crawls
the most popular pages on your website. It does
not a matter if that is one page or hundreds or
thousands. Sites like Amazon.com , BBC.com and
CNN.com have pages that are crawled every ten
minutes, since Google has learned that those pages
have that amount of frequent changes. A typical
site should expect to have a freshbot visit every
weeken days, depending on how popularize those
What happens to your
site on a Freshbot visit is that it finds all
of the deeper links in your site. It places those
links into a database so that when the DeepCrawl
occurs, it has a reference. Once a month, the
DeepCrawl bot visits your site and goes over all
the links found by the Freshbot. This is the reason
why it can take up to a month for your entire
site to be indexed in Google - even with the addition
of a Google Sitemap.Once you are google sitemap
created sucessfully for your website. Google sitemap
also useful to get indexed all pages of Your website
from google searchengine.
So, be patient and keep
on adding content to your site, and work on getting
valuable in-bound links to your site - Google
will reward you for it.
Last year Google set
up the sitemaps program to help webmasters make
it easier for their site's pages to be found by
the Google spider. To participate, you have to
construct an XML file with information about each
of your site's pages and upload this to the main
directory of your website. XML files are the same
kinds of files used in RSS syndication, but in
this case the XML file is meant only to provide
information for the Google robot.
If you don't know how
to make an XML file or you have a very large website
with hundreds of pages it is best to use an XML
sitemap generator. A free, online Google Sitemaps
generator can be found at http://www.auditmypc.com/free-sitemap-generator.html
Once you have made your
XML sitemap and have uploaded it to your server,
go to www.google.com/webmasters/sitemaps and set
up a sitemaps account with Google. When you log
into your account you will see a tab at the top
of the page that says Add. Click it and then check
the box, add "General Web Sitemap".
You will be asked to put the url of your sitemap.
Normally the url of your sitemap will be http://www.yoursite.com/sitemap.xml
The Google sitemaps
program is particularly important if you have
a large, data base driven site or ecommerce website
with many pages that you want to be available
in Google searches.
If you want to increase
traffic to your site, check and see how many of
your pages have actually been included in the
various search engines, and then take steps to
make sure that all of them are included in future
search engine updates. Your efforts to improve
the search engine friendliness of your site will
be well rewarded.