Spider instructions: download Spider instructions related software
-
Robots.txt editor 2
This program is a visual editor for Robot Exclusion Files and log analyzer software. It allows the user to quickly and easily create the robots.txt files required to instruct web search engines which parts of a web site are not to be indexed.
site optimization, log file analysis, spider visits, robots.txt editor, spider instructions