Go to Configuration > Spider and ensure "Crawl All Subdomains" is checked only if necessary.
Screaming Frog flagged 184 URLs with a “never”/noindex status — a red flag for crawl efficiency and index coverage. Audit those pages to confirm intentional noindexing, fix accidental meta robots tags, remove staging or parameterized URLs from sitemaps, and resolve canonical chains or redirects. Re-crawl and resubmit affected pages once corrected to restore crawl budget and indexing health. screaming frog seo spider 184 neverb link
: This version also fixed specific URL encoding issues that previously prevented some pages from loading during JavaScript rendering . Key Features and Capabilities Go to Configuration > Spider and ensure "Crawl
: Finding broken links (404 errors) and server errors (5xx). Re-crawl and resubmit affected pages once corrected to
If you want a version for a tweet, blog intro, or Slack alert, tell me which and I’ll adapt.
Updates to the Chromium engine to better handle heavy client-side frameworks.
One of the most common forms of the "neverb" link is the one blocked by directives. Version 18.4 provides a dedicated "Blocked by Robots.txt" report. This allows SEOs to identify high-value pages that are internally linked but effectively invisible to search engine crawlers. The software highlights the dissonance between the site architecture (which suggests importance through linking) and the server directives (which deny access).