Computing.Net > Forums > Web Development > Robots file

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Robots file

Reply to Message Icon

Name: ian_ok
Date: August 27, 2004 at 02:19:28 Pacific
OS: win xp 98 & 2k
CPU/Ram: p3 512
Comment:

Hi all,

Stupid question here but do I have to upload a file called robots.txt to my webserver or is it simply a matter of putting the following mata tag?

<META NAME="robots" CONTENT="index,follow">

If I have to put a robots.txt file on my server what code etc do I need to put in the file?

Ian

My Website - Accommodation in Andalucia - Spain



Sponsored Link
Ads by Google

Response Number 1
Name: barelysane
Date: August 27, 2004 at 08:37:58 Pacific
Reply:

Most search engines crawlers probably obey META entries for robot rules but if you want to be sure its best to put a robots.txt in your root directory as well. See here for details of the format: http://www.w3.org/TR/REC-html40/appendix/notes.html#h-B.4.1


0

Response Number 2
Name: smbotans
Date: August 28, 2004 at 04:09:19 Pacific
Reply:

the following may be of help to you too
http://www.1-hit.com/all-in-one/tool-robots.txt-generator.htm

serge


0

Sponsored Link
Ads by Google
Reply to Message Icon

Related Posts

See More







Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Web Development Forum Home


Sponsored links

Ads by Google


Results for: Robots file

PHP - saving a file www.computing.net/answers/webdevel/php-saving-a-file/2142.html

including .js file into jsp page www.computing.net/answers/webdevel/including-js-file-into-jsp-page-/2794.html

Cannot write to a text file in PHP www.computing.net/answers/webdevel/cannot-write-to-a-text-file-in-php/1414.html