How to Fix Submitted URL Seems to be a Soft 404 Error?

Google crawls your entire website and shows all crawling related errors in Google Search Console. You can find different types of errors under “Coverage” section of your Google Search Console account. Recently, we have noticed Google did some changes which resulted in many soft 404 errors. In this article, we will explain what is soft 404 and hot to fix submitted URL seems to a be soft 404 error in Google Search Console.

What is Soft 404?

HTTP status codes are useful for troubleshooting the response received from the web server for request sent from the browser. When the URL of the requested webpage is available, web server will send a HTTP status code “200 – OK”. In this case, you will see the rendered webpage on browser without any problem.

On other hand, when the requested URL is not available on the server, you will receive “404 – Not Found” error. It could happen due to many reasons like the webpage is moved to another location or permanently deleted.

Now, that soft 404 is a case that the web server returns 200 status code while Google thinks it is a 404 page. Having a very thin content on the page is a typical example of this case.  

Where Can I See Soft 404 Errors?

Login to your Google Search Console account. You might have added and verified your site in order to use the features of Search Console. Choose your website property and go to “Coverage” section. Here, you will see all type of errors encountered by Googlebot while crawling your website. Check whether you have an item for “Submitted URL seems to be a soft 404” along with number of affected pages. Click on the link to see the list of individual URLs affected by this error on your website.

Why I See Increased Soft 404 Errors?

With mobile first indexing, Google shows all crawling errors in Google Search Console only based on the crawling status of Google smartphone. This could cause problem when your page is looking good in desktop while the resources on the page are loading slow or delayed in mobile devices. You may suddenly see spike in soft 404 errors, when you did some mobile related changes on your site. For example, you might have added an advertisement banner in the above fold area of an already thin content page. Though, there will not be any issue in desktop device, Google might think the content is ignorable on mobile view. However, the server will send 200 success status code. This will confuse Googlebot and results in increased soft 404 errors under “Coverage > Submitted URL seems to be a soft 404” section.   

Why You Should Fix Soft 404 Errors?

There are some errors in Google Search Console account that you can safely ignore. For example, you can ignore the pages under “Crawled but not indexed” section which mostly contains URLs like internal search pages. However, you should fix soft 404 errors similar to 404 or 403 cases. Google will instantly remove soft 404 pages from search results resulting in decreased traffic to your site.

Your site’s traffic will get a big hit, if your top landing pages are affected by soft 404 error.

How to Fix Soft 404 Errors?

There are multiple ways to fix the error depending upon the root cause.

1. Page is Permanently Deleted

There is nothing much to worry if you have really deleted those soft 404 pages permanently. However, make sure to setup proper redirection to send 404, 403 or 410 response. This will help to classify the pages to proper category in Google Search Console. In all these cases, Google will permanently remove the pages from Search results.  

2. Content Moved to Another Page

If you find the content of soft 404 pages are moved to another page, then setup 301 redirect. This will pass all search engine ranking and authority of the deleted page to your new page. Google will index the final destination URL and will not show soft 404 errors.

3. Page is Available – Check with URL Inspection Tool

This is the problematic scenario that you have not done any changes on your site but found increased soft 404 errors. The most probable reasons could be the mobile loading issue as explained above. However, you need to spend some time on troubleshooting before deciding the next action.

The first thing you can do is to analyze the soft 404 page using URL Inspection tool in Google Search Console. You can do this by clicking on the small lens icon when hovering over the affected URL. Alternatively, you can copy and paste the URL in the top search box and hit enter.

You will see the message “URL is not on Google” with an option to test the live page. Remember the result from URL Inspection tool is based on already indexed page in Google. Hence, testing the live page may result in different result especially if you have changed something on the page.

Below the status of the page, you can see details of the page fetch status, last crawled date, crawled as and other details under “Coverage” section.

Click on the “Test Live URL” button to fetch the live page using Google Smartphone crawler. It will take few minutes for Google to crawl the live page and show the updated results. Click on the “View Tested Page” link to see how the Google Smartphone bot rendered your page.

The tested page will show three sections under HTML, Screenshot and More Info. First, click on “More Info” tab and check the “HTTP Response” code.

Check “Page resource” section which will show the number of blocked resources. As you see in the above screenshot, Google could not load 12 resources out of 35 resources available on the page. Click on the “Page resources” to get more details of the blocked resources on the page. It is evident that Google could not load all resources to successfully render the page in mobile devices. However, server sends “200 OK” HTTP response which is the reason Google showing the URL as a soft 404.

3.1. Advertisements and Other Cached Content

Third party advertisements are one of the major reasons for blocked resources problem. Google AdSense or any other JavaScript code placed the header section will block the page loading and will create problems like soft 404. Since Google removes the affected pages from search results, you have to adjust the position of your advertisement before submitting for reindexing.

Similarly, make sure to remove all render blocking resources due to the cached content loading on the header section of your site. This happens with content management systems like WordPress where you can use caching plugins.

As mentioned, Google uses Googlebot Smartphone for fetching and showing the result in Search Console account. Therefore, make sure your page loads properly on mobile regardless of the desktop view.

3.2. Blocked Googlebot

Other issue could be that you mistakenly blocked Googlebot from crawling your site. Remember, Googlebot Smartphone has multiple IP address and blocking one of them will result in errors like 403 or soft 404. Check the IP addresses and user agents you have blocked and make sure that there are no Googlebot IP addresses blocked. In addition, you check your robots.txt file that you have no blocking rules for Googlebot.

3.3. Thin Content in Mobile View

If you are not using scripts in header, then the next possible reason is that you have no or thin content on the page. For example, you may have only few sentences on the page which is also pushed down by an affiliate banner. In this case, Google thinks the page should be considered as 404 and your server wrongly sends 200 status code. Make sure to add more content at least 300 words for Google to understand the page. In addition, ensure the text content is available in above the fold area visible when the page loads in a mobile device.  

3.4. Feed, Taxonomy and Archive Pages

Thin or no content situation can occur without your knowledge. For example, WordPress and many other blogging platforms use taxonomies to categorize the blog posts. You can use category and tags to group similar posts for showcasing on the index pages. In addition, there will be RSS feed for each existing page on the site by adding /feed/ suffix to the URL. This will help feed readers to syndicate your content and offer to subscribed readers. there could also be other archived pages for search results, author and date based. These pages can create soft 404 problems in following situations:

• You have a tag or category with no blog posts assigned thus creating a page with header, footer and sidebar with no content. This could happen for any other feed or archive pages like author and date based archives.
• Generally, index pages will show the latest 10 posts. If you have 11 posts assigned to a tag, the second page will have only one post link creating thin content.

The best option here is to noindex all archive pages on your site. you can also disable feed if you are not using feed readers as a primary source for traffic. This will help to remove soft 404 errors by avoiding thin content issue. However, you will see all noindexed pages in Google Search Console under different coverage section. This should not be an issue as anyway you don’t want those pages to appear in search results while not resulting in soft 404 cases.      

Final Words

After fixing the problem, you can test the live page URL using URL Inspection tool. Make sure the screenshot section shows properly rendered content and there are less or no blocked resources. If everything looks fine, request the page for reindexing. This will help you to bring the page back in search results and fix submitted URL seems to be a soft 404 problem in Google Search Console account.