Search engines, traffic and spam

How many times have you found on the Internet offer to register your website into hundreds or even thousands of search engines? Have you actually ever wondered if there is any sense to?
The real answer is that it is in fact a total nonsense. Why?

1. Today, there are only 4 search engines that generates traffic (alphabetically): Ask, Google, MSN & Yahoo!
2. The remaining "honest" robots only steal bandwidth - this is especially important to know if you are on a hosting plan with limited bandwidth.
3. The rest of the bots only spam your site and that you don't even know - see more detailed your server log files, in them you will find guaranteed all possible forms of spam.
4. Email spam through your registration email is not counted - you'll get that as well like an "additional award".

How to solve the problem?

- Maximally avoid such registration services - both paid and free.

- Type the following lines in your robots.txt file:

User-agent: Googlebot
Disallow:
User-agent: Slurp
Disallow:
User-agent: msnbot
Disallow:
User-agent: Teoma
Disallow:
User-agent: Jeeves
Disallow:
User-agent: *
Disallow: /

- Type the following lines in your .htaccess file:

order allow,deny
deny from env=bad_bot
SetEnvIfNoCase User-Agent "^libwww" bad_bot
SetEnvIfNoCase User-Agent "^libwww-perl" bad_bot
SetEnvIfNoCase user-agent "Indy Library" bad_bot
SetEnvIfNoCase user-agent "noxtrumbot" bad_bot
allow from all

All of the above will not solve your problems 100%, but they will be dramatically reduced.

ask.com:

google.com:

live.com:

yahoo.com: