403 Forbidden Error: Find Who Is Blocked and Fix the Cause

A 403 forbidden error means the server received the request, understood it and refused to answer. The page may exist, your files may be fine, and yet the visitor sees a blunt "Forbidden" message. What makes 403 harder than a 404 is that it often affects only some people: one network, a crawler, or a single folder. You can load the page while a customer, a monitoring tool or Googlebot cannot.

This guide is for site owners and developers who have a 403 in a report, a log, a customer complaint or Search Console. It shows how to find out who is being refused, which layer is refusing them (file permissions, web server rules, an application, a CDN or a firewall) and how to correct it without opening the site wider than you intend. You will end with a repeatable test, a root cause and a clear decision on whether the 403 should exist at all.

Short answer: A 403 forbidden error means the server understood the request but refuses to serve it, even though it knows who is asking. Test the URL with different networks and user agents, find which layer sends the refusal (server rule, permissions, application or firewall), then remove or narrow that rule. Google does not index pages that return 403.

Before you begin

You need to run curl from a machine that is not your own server, access to the web server or hosting configuration, the access logs, and, if you use a CDN or web application firewall (WAF), its event log. Search Console access helps if the problem was reported there.

Treat the fix as a security change. A 403 is usually a rule doing its job in the wrong place, so the goal is a narrower rule, not no rule. Back up the configuration before editing, change one thing at a time and note each change. Allow about an hour.

What does a 403 mean, and how is it different from 401, 404 and 429?

Four codes are routinely confused. The differences point to different fixes:

Code	Means	Typical cause	Who can fix it
401	The client has not authenticated, or the credentials are invalid.	Login or Basic authentication required.	The visitor, by signing in.
403	The server knows the client's identity (or has decided on a rule) and refuses anyway.	Permissions, an access rule, a firewall decision.	Only the site owner, by changing a rule.
404	Nothing found at this URL, or the server pretends so.	Missing page.	The site owner, by restoring or redirecting.
429	The client sent too many requests in a given time.	Rate limiting.	The client, by slowing down.

MDN notes that a server may return 404 instead of 403 when it does not want to reveal that a restricted resource exists, and that repeating a 403 request without changing it should be expected to fail again.

What does a 403 do to search visibility?

Google treats 403 like other 4xx responses: it ignores the content, does not index the URL and removes URLs that were indexed. The Page indexing report in Search Console has a reason called "Blocked due to access forbidden (403)". Its documentation says that Googlebot never sends credentials, so a 403 is considered incorrect for a page you want indexed, and the fix is to let non-signed-in users reach the page or to explicitly allow Googlebot after verifying its identity.

Two other effects are worth knowing:

robots.txt. If /robots.txt returns 403, Google treats it like any 4xx other than 429: as if no valid robots.txt exists, so it assumes there are no crawl restrictions. A forbidden robots.txt therefore does not protect anything.
Crawl rate. Google says 401 and 403 should not be used to limit crawl rate, because 4xx codes other than 429 have no effect on it.

None of this predicts what a page will do in rankings. It describes whether Google can fetch and keep the URL.

Step-by-step instructions

Step 1: Reproduce the 403 and find out who is blocked

Start from outside your own network, so you do not test with an address your server trusts. Run:

# Normal request
curl -s -o /dev/null -w '%{http_code}\n' https://example.com/page

# Same request claiming to be Googlebot (a test of rules, not proof of access)
curl -s -o /dev/null -w '%{http_code}\n' \
  -A 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' \
  https://example.com/page

# Request with an outside referrer, which hotlink rules look at
curl -s -o /dev/null -w '%{http_code}\n' -e https://other.example/ https://example.com/image.jpg

Then repeat from a second network, such as a phone on mobile data. Build a small table: which requests get 200 and which get 403. If only the Googlebot user agent is refused, a bot rule is the likely cause. If everything outside your office is refused, look at IP allow lists. If a single URL or folder fails for everyone, look at permissions or a path-specific rule.

A user-agent test is only a hint. Anyone can send that header, and a firewall that checks crawler identity by IP address will treat your test differently from the real crawler. Crawvix's crawler-access check works the same way and says so: it requests your home page with Googlebot and Bingbot user agents from its own servers, and it cannot prove that the real crawlers, arriving from their own addresses, are allowed.

What success looks like: You have a short table showing exactly which kinds of request get a 403 and which get 200.

Step 2: Identify which layer sent the response

Several layers can produce the same code. Look for evidence of the sender:

The response body and headers. A server's default error page, an application's own page and a CDN's block page usually look different, and a CDN block page often names the product that produced it.
The logs. Find the request by timestamp. If it appears in the web server's access log with 403, the refusal came from the server or application. If it never reaches the origin log, a CDN, WAF or host-level firewall answered first.
The error log next to it, which usually states the reason, such as a denied rule or a permissions problem.

For a CDN or WAF, search its security events by the client IP or the request identifier shown on the block page; the event normally names the feature or rule that fired.

What success looks like: You can name the layer (origin, application, CDN, WAF or host firewall) and the rule or setting that responded.

Step 3: Check permissions and the folder index

If the refusal comes from the origin and affects one file or folder for everyone, check the basics. The web server process needs permission to read the file and to enter every folder above it. A request for a folder URL with no default index file, on a server set not to list folder contents, can also end in a refusal. The error log normally states which of these it was. Add the index file, or point the URL somewhere real, rather than turning folder listings on.

What success looks like: The file is readable by the server process, and the folder has an index document or a deliberate redirect.

Step 4: Check access rules in the server configuration

Look for rules written for one purpose that also match public pages. Typical examples:

# nginx: prefix match, so this also catches /admin-guide-for-teachers
location /admin {
    allow 203.0.113.0/24;
    deny  all;
}

# Apache 2.4: deny one address, allow everyone else
<RequireAll>
    Require all granted
    Require not ip 192.0.2.44
</RequireAll>

nginx evaluates allow and deny in order and the first match wins, so a broad deny all placed before an allow rule locks out everyone. In Apache 2.4 a lone Require not cannot stand on its own; it needs a grant beside it, as above. Apache's documentation also marks the older Order, Allow and Deny directives as deprecated, and warns that blocking by User-Agent is unreliable because clients can send any value. Rewrite rules with the [F] flag return 403 as well, so search .htaccess and the main configuration for it.

Narrow the rule: use an exact path (location = /admin or location ^~ /admin/), name only the addresses you mean to block, and test again after reloading the configuration.

What success looks like: The rule matches only the area it was meant to protect, and the public URL returns 200.

Step 5: Review the firewall, CDN and bot protection

If the origin log shows nothing, or the 403 depends on the user agent or network, go to the security layer. Find the event, read which rule or managed ruleset fired, and change that rule's scope instead of disabling the whole feature. If the blocked client is a search crawler, confirm it is genuine before allowing it: Google documents a reverse DNS lookup that must return a name in googlebot.com, google.com or googleusercontent.com, followed by a forward lookup that returns the original address, and also publishes IP range files. Allow verified crawlers, not user-agent strings.

Details for the two common cases are in Cloudflare Blocking Googlebot or Bingbot and Verify Googlebot and Bingbot. Hotlink-protection rules, which refuse image requests with another site's referrer, should be tested the same way: request your own images with and without a referrer and confirm your pages, your CDN and your sitemaps still load them. See also Why Are My Images Not Loading?.

What success looks like: The security event no longer appears for verified crawlers and legitimate visitors, and unwanted traffic is still blocked.

Step 6: Decide whether the page should return 403 at all

Ask what the URL is for. If it is a private area, 403 or 401 is correct and the page should not be linked in your sitemap or your navigation. If it is a public page, remove the block. If the content is gone, use 404 or 410 instead, and if it moved, use a 301 redirect; 404 vs 410 explains the choice. Do not leave a public page returning 403 because it was convenient to hide it.

What success looks like: Every URL has the status code that matches its purpose.

Step 7: Retest, then watch Search Console

Repeat the Step 1 commands from both networks. Then test the URL with Search Console's URL Inspection tool, as described in URL Inspection Tool Live Test. The Page indexing report updates after Google recrawls, and no timing is promised, so keep your own test table as the primary evidence.

What success looks like: The same requests that got 403 now get the intended response, and the table is saved for the next incident.

Final verification checklist

You know whether the 403 affects visitors, crawlers or both, and from which networks.
You named the layer that sends it and the specific rule or permission.
Public pages, images, /robots.txt and the sitemap return 200 without credentials.
Verified Googlebot and Bingbot are allowed; the rule does not depend on a user-agent string alone.
Private areas still return 403 or 401 and are not in the sitemap or internal navigation.
The fix was a narrower rule, not a removed protection.
The test table and the changed configuration lines are written down.

Common mistakes to avoid

Treating 403 as a missing page. The page exists; a rule refuses it. Restoring files will not help.
Testing only from the office or the server. Trusted addresses pass rules everyone else hits.
Allowing a crawler by name. A user-agent string can be faked. Verify with DNS or the published IP ranges.
Switching off the firewall to see if it helps. Find the rule and narrow it instead.
Using 403 to hide pages that moved or were deleted. Use 301, 404 or 410.
Using 403 on robots.txt. Google reads it as no restrictions, so it protects nothing.
Forgetting the CDN cache. A cached 403 can outlast the fix; purge and retest.

Frequently asked questions

Does a 403 error hurt my SEO?

A 403 on a page you want indexed keeps that page out of Google's index, and indexed URLs that begin returning it are removed. Google does not describe a site-wide penalty for a few intended 403s on private pages. The concern is public pages returning it by mistake.

Why do I get 403 but my colleague does not?

A rule is probably matching your network, country, user agent, cookie or referrer. Compare requests from both of you in the access log and look for the rule they hit.

Will fixing a 403 get the page indexed?

It lets Google fetch the page again. Whether and when it is indexed is up to Google, which gives no timing.

Should I use 403 or 404 for private pages?

Either can be right: 403 says the page is restricted, while 404 can hide that it exists, as MDN notes. Do not list either in your sitemap.

Is a 403 the same as being blocked in robots.txt?

No. A robots.txt rule asks well-behaved crawlers not to fetch a URL, and the server answers normally if they do. A 403 is the server refusing the request. See Blocked by robots.txt: Fix It.

Can a 403 appear only for Googlebot?

Yes. Bot protection, rate limits and security plugins sometimes challenge or refuse crawler requests. Compare what a browser gets with what the URL Inspection tool reports.

Official references

Next step: Try the free website sample to see how your own site does on a few of these checks. This guide provides technical information, not a guarantee of rankings, traffic or index inclusion.