The SEO Starter Guide to Google Search ConsoleChapter ten

How to Fix Crawl Errors

Site Errors

Site Errors are issues that affect your entire website, and should be prioritized. Below is an example of what you want to see – no site errors.

Note: You can click on all screenshots below to view at a larger size.

no site errors gsc

Possible site errors include:

DNS Errors

DNS errors are a major problem – this means that Googlebot couldn’t connect with your domain due to a  DNS lookup or DNS timeout issue.

Server Connectivity Errors

Server errors usually mean that your site is taking too long to respond, so the request times out. This means that Googlebot can connect to your site but can’t load the page.

Robots.txt Fetch Errors

A Robots error means that the Googlebot cannot retrieve your robots.txt file from [example.com]/robots.txt. However, you only need a robots.txt file if you don’t want Google to crawl certain pages. If you don’t have a robots.txt file, the server will return a 404 and Googlebot will simply continue to crawl your site.

robots txt errors in gsc

URL Errors

URL Errors affect individual pages of your site and are categorized by Desktop or Smartphone.

Types of URL errors include:

Server Errors

Server errors happen when Googlebot couldn’t access your URL, the request timed out, or your site was busy.

Soft 404s

A Soft 404 status happens when a page displays as 200 (found) when it should display as not found (404). For pages that don’t exist, they should either be 404d or 301 redirected to a relevant page. Any live pages that display as soft 404 should be audited to ensure that there isn’t thin content or content that might indicate a 404 page.

Not Found (404)

A 404 means that Google tried to crawl a page on your site that does not or no longer exists. Googlebot finds 404 pages when other sites or pages link to that non-existent page.

If the page is an important page, you should address the issue immediately.

Access Denied

This error often indicates that the URL requires a login or your hosting provider is blocking Google from accessing your site.

Not Followed

Not Followed errors can often indicate crawling issues from JavaScript, Flash or other forms of active content or improper redirects.

DNS Error

DNS errors indicate that Googlebot couldn’t connect with the URL due to a  DNS lookup or DNS timeout issue.

How to Fix 404 Errors in Webmaster Tools

If you find a 404 error, there are a few ways to resolve it using Google Search Console.

Step 1: Log in to Search Console, click on Crawl, then click Crawl Errors

Step 2: Click on the Not Found tab to view 404 errors

not found errors in gsc

Step 3: Click on the first URL displayed to show a popup with two tabs: Error Details and Linked from

linked from tab for url errors in gsc

Step 4: In the Linked from tab, you can see what internal and external links are pointing to the 404 pages.

If the URL was a previously existing page, you could 301 redirect the old URL to a relevant page. If there is a misspelling or error, you can reach out to the supplier of the incorrect URL asking them to update their link to the correct URL.

Step 5: If a page is properly 404d, you can either ignore it or select Mark as Fixed to remove the 404 from your Search Console error report.

However, if Google is still finding the URL (through links, in your sitemap, etc…) it may recrawl this page at a later date and will reappear in the Crawl Error report.

mark url errors as fixed in gsc