7 min read
Soft 404: what Google means by it, and how to fix one
By CrawlWarden, which builds the monitoring described at the foot of this article.
Published .
A soft 404 is a page that reads as missing but answers with HTTP 200 instead of 404. Google describes it as a URL that "returns a user-friendly 'not found' message but not a 404 HTTP response code". Google will not index one, and keeps re-fetching it, because 200 says there is something there. Two different problems carry the label: a wrong status code, which is a configuration fix, and a page Google judges to be empty, which is not.
What Google counts as a soft 404
The mechanical case is the one the documentation defines. The page is gone, the server answers 200, and the body says "not found". Google puts it as a URL that "returns a user-friendly 'not found' message but not a 404 HTTP response code" (Search Console Help, Page Indexing report).
The second case is a judgement, and the Search Console wording carries the hedge in the open: the status fires when the request returns "what we think is a soft 404". An empty category, a search results page with no results, an expired listing that now renders a stub - all answer 200 with real markup and are still filed here. No status code change on your side clears that verdict.
The rest of this page keeps the two apart, because the fixes do not overlap. If the URL already returns 404 and Search Console still says soft 404, you are in the second case and nothing in the server configuration will help.
Why a 200 on a missing page costs more than a 404
A 404 is not an admission of failure. It is the only way to tell a crawler that a URL is finished, and Google documents precisely what each answer does.
| Response | What Google does with it | Use it when |
|---|---|---|
200 with a "not found" body | Search Console flags a soft 404 and the URL is not indexed. Google keeps coming back, because 200 means there is content here. | Never on purpose. |
404 Not Found | Google "doesn't index URLs that return a 4xx status code, and URLs that are already indexed and return a 4xx status code are removed from the index". | The page is gone, and the URL might be reused one day. |
410 Gone | Treated as a 4xx like the rest. It reads as deliberate rather than as a fault. | Pruning in bulk, where nothing is coming back. |
301 to a real replacement | The URL moves and the signals follow it. Redirecting everything missing to the home page instead produces soft 404s, because the destination does not answer the request that was made. | A genuine one-to-one replacement exists. |
503 or 429 | These "prompt Google's crawlers to temporarily slow down with crawling". Indexed URLs are "preserved in the index, but eventually dropped". | The outage is real and you expect it to end. |
The four ways a site starts producing them
Every quotation in the table above comes from Google's HTTP status codes and network errors reference. The causes below are in no documentation at all, because they are accidents of how sites get built.
- A client-side router. A single-page app resolves an unknown path in the browser and renders its own "page not found" view, but the server already answered 200 for the document. Every typo, stale link and deleted route becomes a soft 404, at scale, silently.
- A catch-all redirect to the home page, usually added during a migration. It converts a finite set of honest 404s into an unbounded set of duplicates of the front door.
- Empty templates: a category with no products, a tag with no posts, an internal search page that is indexable. The template renders, so the status is 200, and there is nothing on the page.
- Expired or out-of-stock items that start rendering a stub. The URL outlives the thing it described.
How to confirm one in under a minute
- Ask the server about a path that cannot exist:
curl -o /dev/null -s -w "%{http_code}" https://example.com/this-page-does-not-exist. Anything other than 404 or 410 means the mechanical kind, site-wide, for every missing URL you have. - Run the same command against the URL Search Console flagged. If it already answers 404, Search Console is reporting the judgement kind, and the work is content or removal rather than configuration.
- Use URL Inspection then Test Live URL, and read the rendered HTML Google captures rather than your own view-source. A router that swaps in the error view after hydration looks fine in view-source and wrong to a crawler.
- Check for a redirect hiding the same defect:
curl -sI https://example.com/gone | head -n 1. A 301 into a page that then says "not found" is a soft 404 with extra steps, and it is also how a redirect chain tends to start.
Fixing each cause
Google's advice runs the same way in both cases: make the distinction easier rather than hide it. Return the real status for pages that do not exist, and give a page that does exist enough clarifying content that it does not read as an error.
| Cause | Fix | What not to do |
|---|---|---|
| Router renders a 404 view on a 200 | Return a real 404 status for unmatched paths from the server or the edge, and keep rendering the same view. | Adding noindex to the view. It stops the indexing symptom and leaves the status wrong for every other consumer of the URL. |
| Catch-all redirect to the home page | Return 404 or 410 where there is no genuine replacement. Redirect only one-to-one matches. | Keeping the redirect because the bounce rate reads better. |
| Empty category, tag or search page | noindex the empty state, or return 404 when the set is empty and the URL is not a real section of the site. | Padding the template with filler text so it stops looking thin. |
| Expired or out-of-stock product | 410 if it is never coming back. If it is, keep it at 200 with real content and links to the alternatives. | Redirecting every dead SKU to the category root, which recreates the problem one level up. |
Why this is worth watching rather than auditing once
Nobody introduces a soft 404 on purpose. It arrives with a framework upgrade, a move to a new host, an edge rewrite, or a redirect rule written in good faith, and on the day it arrives the site looks correct in a browser. Almost no deploy pipeline asserts on what the server says about a URL that is not supposed to exist.
Search Console will tell you in the end. The Page Indexing report is where this surfaces, its data lags by a few days, and most people open it after something else has already gone wrong. That gap between the deploy and the report is the same one behind a page quietly dropping out of the XML sitemap and a canonical tag changing under you.
The checks that report this
CrawlWarden watches a list of URLs you choose and reports what changed. It does not diagnose and it does not score. Severity is a property of the check rather than a setting: the number below is how many consecutive runs must see the new value before it becomes an email, and it is never less than two, because one observation is noise.
Real 404 on a missing pageconfirmed over 3 consecutive runs
Requests a path that cannot exist and reports when the answer stops being a real 404. That catches an edge rewrite or a router change on the day it ships rather than when a report notices.
HTTP status codeconfirmed over 2 consecutive runs
Reports when a URL you watch stops returning the status it used to, in either direction.
Sitemap sample returns 200confirmed over 4 consecutive runs
Off by default, because it costs extra requests. Samples the URLs your sitemap lists and reports when they stop resolving.
Every check is available on every plan, including the free one. CrawlWarden pricing, per site rather than per URL.
Questions
- Does a soft 404 hurt the rest of the site?
- Not as a penalty. The cost is crawl budget spent re-fetching URLs that are finished, and, where the cause is site-wide, a large set of near-identical pages for Google to sort out. Across a few hundred URLs it is close to harmless. Across a large catalogue it is not.
- Should I use 404 or 410?
- Either. Google treats both as 4xx and removes indexed URLs that return them. 410 says you are certain, which is useful when pruning in bulk; 404 is the safer default if the URL might come back.
- Search Console says soft 404 but the page is real and useful.
- That is the judgement case. Confirm the page returns 200 and renders its content server-side, check it is not near-identical to another page, and make sure something other than the sitemap links to it. There is no button that overrides the verdict.
- Will CrawlWarden list every soft 404 on my site?
- No. It watches whether a missing page still returns a real 404, and whether the URLs you listed still return the status they used to. Enumerating every soft 404 on a site is a crawler's job, and a desktop crawler does it better than any monitor will.
Related
- redirect chains, which a catch-all redirect usually creates alongside
- a page dropping out of the XML sitemap
- noindex, the other way a page leaves the index without warning
- the maintenance page that should answer 503 instead
- every failure mode CrawlWarden watches for
- every check it runs, and what a change in it means
Sources
What was checked, and where. An entry with no link is something that could not be verified, named here rather than left out.
Google Search Console Help: Page Indexing report
The definition of the Soft 404 status, the "what we think is a soft 404" wording, and the full list of not-indexed reasons.
Google Search Central: HTTP status codes and network errors
How Google treats 4xx, 410, 5xx and 429, including removal of indexed 4xx URLs and the crawl slowdown on server errors.
Not verified: search volume for this query
No keyword volume tool was available. Demand was judged from the live US SERP, which carries no ads and ranks a Reddit thread and a Google Help forum post inside the top five.
Not verified: any claim that soft 404s carry a ranking penalty
Google documents indexing and crawl effects only. Nothing in the primary sources supports a penalty, and this page does not claim one.
Know the moment this happens to you
Give CrawlWarden a domain, keep the pages it suggests, and the first check runs immediately. Free forever for one site.