Google has discharged an upgraded robots.txt testing tool in
Webmaster Tools. The apparatus might be found in the Crawl segment.
The point of the new form of the device is to make it
simpler to make and keep up a "right" robots.txt document, and make it less demanding to
discover the mandates inside a vast record that are or were blocking individual
Urls.
Google is including robots.txt error checking tool it means
Google give more trust to those sites which has blunder free robots.txt file in
our root folder.
