How to fix “Blocked due to unauthorized request (401) / access forbidden (403)” on Drupal
If Search Console shows “Blocked due to unauthorized request (401) / access forbidden (403)” on your Drupal site, you are in the right place. Googlebot could not read the page because it asked for login or access was denied. This page covers the causes and a step-by-step fix for Drupal. A flexible, module-based CMS. The Metatag, Simple XML Sitemap and Redirect modules provide indexing control.
Contents
What does this status mean?
The server answered Googlebot with 401 (login required) or 403 (forbidden). Googlebot does not log in, so the page cannot be indexed.
Likely causes on Drupal
- The page is members-only or password-protected.
- A firewall, WAF or bot protection blocks Googlebot IPs.
- A country or IP restriction covers Google IPs.
- A hotlink or user-agent rule rejects Googlebot.
- Drupal note: Edit robots.txt in the web root or manage it with the RobotsTxt module. On multisite setups each site needs its own file.
Step-by-step fix on Drupal
- In your Drupal admin go to: Configuration > Search and metadata (Metatag, Simple XML Sitemap, Redirect).
- Edit robots.txt in the web root or manage it with the RobotsTxt module. On multisite setups each site needs its own file.
- On the Performance screen enable page and block caching and CSS/JS aggregation; the dynamic page cache lowers server load.
- If the page should be public, remove the protection; if it should stay private this line is expected.
- Allow-list Googlebot as a verified crawler in the WAF and firewall.
- Allow Googlebot by verifying with reverse DNS rather than trusting the user agent.
- Confirm the response code with a live test in URL inspection.
- After fixing, start validation and send the URL.
Extra checks for Drupal
- sitemap: Install Simple XML Sitemap, pick content types and submit the resulting /sitemap.xml. Leave out types that should not be indexed.
- noindex and visibility: In the Metatag module check the "robots" value per content type; a noindex left over from a fresh install can spread to every node.
After fixing
In Search Console start "Validate fix" for this status and resubmit the page with URL inspection. First confirm the blocker is gone with the free Free indexability checker tool; for many URLs follow Bulk URL indexing: how to send thousands of pages to Google. Google makes the indexing decision itself and timing ranges from days to weeks.
More information
Frequently asked questions
Can Google see members-only content?
No. Previews need structured data and paywalled-content markup.
How do I tell if I am blocking Googlebot?
Look for 403 entries for Googlebot requests in your firewall logs.
Where are these settings on Drupal?
Configuration > Search and metadata (Metatag, Simple XML Sitemap, Redirect).
Related pages
- Drupal: URL is unknown to GoogleFixes
- Drupal: Indexed, not submitted in sitemapFixes
- Drupal: Crawled - currently not indexedFixes
- Magento: Blocked due to unauthorized request (401) / access forbidden (403)Fixes
- OpenCart: Blocked due to unauthorized request (401) / access forbidden (403)Fixes
- PrestaShop: Blocked due to unauthorized request (401) / access forbidden (403)Fixes