Robots.txt: download Robots.txt related software
-
Robots.txt 2.0
Visually generate industry standard robots.txt files and analize webservers log.
robotstxt, spiders, site, search, visits, spider, analyzer, html, robots.txt, log analyzer
-
Robots.txt editor 2
This program is a visual editor for Robot Exclusion Files and log analyzer software. It allows the user to quickly and easily create the robots.txt files required to instruct web search engines which parts of a web site are not to be indexed.
site optimization, log file analysis, spider visits, robots.txt editor, spider instructions
-
Website Toolbox Pro 1.1
Over 30 generators, Web 2.0 elements, tools and scripts for webmasters and web designers, including: color palettes, gradient images, stripe backgrounds, rounded corners, .htaccess and mod_rewrite generators, regular expressions and more...
webmaster, webdesigner, tools, color palettes, generators, htaccess, htpasswd, mod_rewrite, gradient, stripe backgrounds
-
Keyword Crawler 1.1
Online website optimization tools: explore and see your site like webengines. Count and display broken and internals links. Get Google pagerank for each file and generate a SiteMaps file in XML format. KWC observes the directives of the robots.txt.
keyword, finder, search, website, crawl, spider, pagerank, sitemap, xml, google
