How a scan works

A scan of your whole site runs when you add_site, when you ask your assistant to check with target: site, and on its own through the weekly re-scan, a sitemap change or your deploy webhook. It goes through these stages:

1. Discovery finds your pages from your sitemap and a crawl.
2. Templates groups pages that share a URL pattern and picks representatives.
3. Rule checks and AI judgment run on every page.
4. Representative pages are loaded again in a real browser and their speed is measured.
5. Page judgments compare pages across the whole site.
6. Internal link suggestions look for links worth adding between your pages.
7. The health score and the fix-first ranking update as results come in.

You can also scan a single page with check (pass the URL or finding). That runs steps 3 and 4 for that one page.

DiscoveryCopy Anchor Link

Loomaly starts with what your site says about itself, then crawls for anything it missed.

• robots.txt and sitemaps. It reads /robots.txt, then /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml and any sitemap your robots.txt lists, following sitemap index files down to nested sitemaps. It keeps up to 10,000 URLs, along with each page's lastmod date. Your homepage is always included. A site with at least 10 discovered pages and no sitemap is no sitemap. When a sitemap exists, indexable pages the crawl found that are missing from it (after ignoring a trailing slash, www and query strings) are sitemap leaves pages out. Those two do not fire together. Sign-in and app pages that answer 200 without noindex are sign-in or app page can be indexed. A robots.txt that is only comments is a notice that the host manages the file.
• Crawl. Next it follows links from your homepage, up to three clicks deep, to find pages the sitemap doesn't list. The crawl visits up to 200 pages on the Free plan and up to 1,000 on paid plans.
• Sites built with JavaScript. If the sitemap and crawl together find fewer than 20 pages, the site probably builds its links in the browser, so Loomaly crawls a second time in a real browser that runs your JavaScript.
• What's left out. Only pages on your domain count (www and non-www are treated as the same site). Files that aren't web pages, such as images, PDFs, feeds, and .txt, .xml or .json files, are skipped. If you set a path scope such as /gb, only pages under that path are kept.

Pages your robots.txt disallows (for all crawlers or for LoomalyBot) are marked robots blocked and aren't fetched at all: no rule checks, no AI judgment, no performance test, no page judgments. The site asked crawlers to stay out.

User-agent: CloudflareBrowserRenderingCrawler
Allow: /

If discovery finds nothing at all (a blocked sitemap, or a site that redirects visitors from our region elsewhere), fix the sitemap or unblock the crawler, then ask your assistant to check with target: site again.

Page templates and representativesCopy Anchor Link

Most sites are built from a handful of templates: every product page, every blog post and every category page shares one layout. Loomaly groups your pages by URL pattern, so /products/blue-mug and /products/red-teapot both belong to the template /products/*. A page whose path is unique, like /about, is a template of its own.

From each template Loomaly picks representative pages, spread evenly across the template rather than bunched at the start of the list:

Representatives are where Loomaly loads pages in a real browser and measures speed (see below). Every page, representative or not, gets the same rule checks and AI judgment.

Every page: rule checks and AI judgmentCopy Anchor Link

Every page Loomaly found (up to 1,500 per scan), except those robots.txt disallows, goes through the same two steps. The page is read as the server sends it, without running JavaScript.

Rule checksCopy Anchor Link

Code settles everything that can be decided from the page's HTML, the same way every time:

• the title and meta description (missing, too short, too long)
• the canonical link (missing, or pointing at another URL), the viewport tag, noindex and nofollow, and the lang attribute on <html>
• headings: a missing H1, more than one H1, skipped levels
• images: missing or empty alt text, and pages with four or more images where none is lazy-loaded
• links: no internal links, fewer than three, or more than 100 external links
• structured data: no JSON-LD, JSON-LD with errors, no Organization schema on the home page, no breadcrumb schema on deeper pages
• Open Graph tags and og:image, and a favicon
• hreflang: a missing self-reference or x-default when the page declares hreflang
• HTTP errors, pages served over http://, pages that are an empty app shell without JavaScript, and pages under 300 words (Chinese, Japanese and Korean text is counted by character)

The Rules reference marks each of these rules Rule check.

AI judgmentCopy Anchor Link

Some things can only be judged by reading the page. For each page, Loomaly puts the page's path, title, headings, a short opening excerpt, a description of its length and the buttons and call-to-action links it found in front of an AI model and asks one set of questions at once:

Code decides which answers apply to which kind of page, so a product page is never flagged for having no author. Only confident answers become issues: the model has to be about 90% sure the answer is "no" (or, for the FAQ question, "yes"). Anything less certain is dropped: it isn't shown and doesn't affect the score. The page's text is only ever given to the model as data to judge, never as instructions.

Sign-in, cart, checkout, search, legal and contact pages, noindex pages, pages that redirect and pages that return an error aren't judged. Neither are pages with under 150 characters of text in the served HTML: there's nothing to read. When such a page is a sizeable document (20 KB of HTML or more), the rule check reports it as Page renders only client-side instead, because that near-empty page is what crawlers that don't run JavaScript see, including most AI crawlers.

Every explanation and fix prompt comes from Loomaly's own templates, so the same problem is described the same way on every scan. Your assistant gets those prompts on the top todo item (fixPrompt) and via problem when you need more detail; it only invents extra wording when you ask it to draft a change.

Rule checks don't count against your pages on any plan. AI judgment doesn't either: on Free it runs on the top 20 pages of each scan, and on a paid plan it runs on every page. If AI judgment couldn't run for a page, that page's earlier AI-judgment issues stay open rather than being treated as fixed, until a later scan judges it again.

todo and problem show each problem once, with the page templates it appears on, so a problem on 1,000 product pages shows up as one thing to fix in the template. A page that redirects to another website isn't treated as one of your pages: it's dropped from the list and its issues are ignored automatically.

Representative pages and performanceCopy Anchor Link

Representative pages are scanned on their own rather than in the site-wide pass. For each one, Loomaly:

1. Loads the page in a real browser, so content your JavaScript renders is included, as Google sees it.
2. Runs the same rule checks and AI judgment on the rendered page.
3. Measures performance with PageSpeed Insights and real-visitor data (see below).

A single page you scan yourself with check goes through the same three steps.

How many pages are testedCopy Anchor Link

Each representative page takes one of your plan's monthly pages. A scan tests as many representatives as your plan has pages left this month, up to 150 per scan, and never more than the site has. When no pages are left, the scan still runs the rule checks and AI judgment on every page; it just tests no pages in depth until your pages reset or you upgrade. See Plans & pages.

To spread that budget well:

• Breadth first. Every template gets its first representative tested before any template gets a second.
• Oldest first. Representatives that were scanned longest ago go first. On a site with more templates than one scan's budget, repeated scans work through the rest instead of testing the same pages every time.
• In waves. Tests start 40 per minute rather than all at once. A scan of 150 pages starts its last wave about three minutes in.

Why only representativesCopy Anchor Link

Loading a page in a real browser and running PageSpeed Insights on it takes time and costs money for every page, and pages built from the same template almost always perform alike: they load the same scripts, styles and layout. So speed is measured on a few pages per template, while the checks that really differ from page to page (rule checks, AI judgment and page judgments) run across the whole site.

Performance and Core Web VitalsCopy Anchor Link

When a representative page is scanned, Loomaly runs Google PageSpeed Insights on it (mobile) and looks up Chrome UX Report data (real visitors on phones). Wherever real-visitor data exists it's used first, because that's what Google ranks on; otherwise lab measurements are used. A page is flagged when:

LCP, INP, CLS and TTFB are warnings in Google's "needs improvement" band and errors only when real-visitor data is in its "poor" band; lab-only results stay warnings. Each issue says whether its numbers are the page's own, the whole site's (Google's fallback when a page has too few visits) or from the lab. A measurement older than 45 days stops counting and its issue moves to Needs re-measuring. These appear in todo like any other issue. If PageSpeed Insights can't measure a page, the scan reports nothing for it, and it won't mark earlier performance issues on that page as fixed.

Page judgmentsCopy Anchor Link

Some problems only show when you compare pages with each other or look for pages that have quietly broken: is this page competing with that one? Is this product gone? Code picks the likely candidates first and AI judges only those.

After the site-wide pass finishes, a background pass looks at up to 1,500 pages:

• Competing pages. Loomaly compares each page's title, heading and opening text with its closest matches on your site. Pairs that are nearly identical go to AI, which decides whether they are duplicates, variants of one item (another size, colour or language), overlapping, or distinct. Only duplicates and overlapping pairs become an issue, Competes with another page for the same search, and the issue names the other page. Pages that already point at each other through a canonical or hreflang link, or that redirect, are skipped.
• Empty or gone pages. Pages with very little text (under 80 words), or whose title, heading or opening text says something like "sold out" or "not found", go to AI, which decides what the page really is. The answer can become Error page served as a normal page, Unavailable item still indexable or Placeholder page with no real content. Sign-in, cart, legal and noindex pages are skipped, since they are meant to be short or out of search.

Each pass judges at most 150 pairs and 100 suspect pages, most similar and most clearly broken first. Answers below 0.60 confidence are dropped. Answers from 0.60 up to 0.85 are stored and left off todo and the weekly report until you turn on lower-confidence AI judgments under /account; 0.85 and above are shown. They still count toward the health score. Results usually show up in todo a few minutes after the rest of the scan. A page-judgment issue closes only when a later pass judges the page again and no longer finds the problem.

Internal link suggestionsCopy Anchor Link

When the page judgments of a site scan are done, Loomaly looks for internal links worth adding. Nothing to click: it runs after every full scan, and takes a few minutes on a large site. The pass:

1. Takes up to 500 of your most recently scanned pages that have text on them.
2. For each page, finds the pages on your site whose content is closest to it: the likeliest places for a link to it.
3. Looks for an existing sentence on those pages that mentions the topic, and the words in it that would make a natural anchor. Links that already exist are never proposed again.
4. Asks AI whether an editor would actually add that link.

The results are available from todo with view: links. The yes/no judgment treats 0.90 and above as ready to apply. In calibration that bar was never reached — every chosen suggestion sat between 0.50 and 0.90 — so the list shows that whole set instead of an empty one. todo can ask for routing: "act" when you want only the stricter set. Each suggestion shows the page it goes on, the page it points to, and the sentence with the words to turn into a link. After you add a link (or decide not to), triage with decide_link marks it accepted or dismissed. Below the suggestions are the pages on your site that no other page links to. Loomaly never edits your site: accepting a suggestion only marks it done for your team.

The health scoreCopy Anchor Link

The health score is built from five dimensions (indexability, page basics, content and AI readability, structured data, performance). On each page an error fails its dimension, a warning costs 25 points and a notice 5, at most 10 from notices. Each dimension is averaged over the pages checked for it, and the score is their weighted average. How the health score works has the weights, the performance rules and a worked example.

Open issues from every source count, and so do issues marked fixed until a re-check confirms them. Ignored and fixed issues don't. status and report show each dimension's score and what fixing each top problem would add. The score updates as each page finishes, so it moves during a scan, and again when page judgments arrive.

Limits at a glanceCopy Anchor Link