Sample Robot Exclusion Code

===

Code to exclude robots from any URL starting with "/cyberworld/map/" or "/tmp/:

# robots.txt for http://www.site.com/

User-agent: *
Disallow: /cyberworld/map/ # 
	This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear

===

Home Home PREVIOUS Outline

URL: http://www.lanw.com/myw/search11.htm
Copyright © 1997 LANWrights, Inc.
Layout, design & revisions by LANWrights
Webmaster: Natanya A. Pitts
Revised -- March 5, 1997