|
|
| |
Search Engine Optimisation
> Robots Directives
|
|
Robots Directives
Q. What will a spider do when it gets to your website?
A. You cannot tell!
However there are many spiders and robots out there
on the internet and most will behave in predictable
fashions when given the right directives. Unfortunately
there are still some rogues out there which will
just ignore the robots directives standards, for
the most part these are not of consequence to your
internet marketing strategy. Many are just email
snaggers, targetting email addresses for addition
to emailing lists.
Anyway for all those well behaved spiders there
are a number of things you can do to ensure that
your site is well looked upon. These include ensuring
that your robots.txt file is correctly written,
ensuring that you have included the correct robots
meta tags and that the instructions given to the
robots are such that they will fully index your
site if this is your wish.
Many spiders will just
up and leave if you do not provide the appropriate
information. This will leave you wondering just
why your site is being ignored by some listings
and included in others when you know you have submitted
to them all.
|
 |
|
| << Back to Search Engine Optimisation - View our SEO Site>> |
|
|
