Robots.Txt Testing Tool
This week I had an issue with a robots.txt file. I have to be honest. I can't remember the last time I edited a robots file. Most of the sites I work on generally speaking we want all the content crawlable. So, we don't have a lot of advanced things set up. But, in this case. I needed to see what file Google had cached as well if one of my URLs was craw…
Keep reading with a 7-day free trial
Subscribe to Stack Snacks to keep reading this post and get 7 days of free access to the full post archives.

