Here is another PSA from Gary Illyes of Google. In short, if you serve a 4xx status code with your robots.txt file, then Google will ignore the rules you have specified in that file. Why? Well, 4xx ...
Google has published a new help document explaining how different HTTP status codes affect how a site appears in search results. A recent tweet suggests Google’s Gary Illyes had a hand in putting this ...
Matt Cutts on How Google Handles 404 & 410 Status Codes In a recent webmaster help video, Google's Matt Cutts talks about the nuances of 404 vs. 410 status codes. While Google's crawlers treat 404s ...