# SEO Audit Tool — Sitemap Crawl & On-Page Checker (`insight.solutions/seo-page-audit`) Actor

Point it at a domain. It reads robots.txt, walks the sitemap (gzip and nested indexes included) or crawls the site, then audits every page: title, description, H1s, canonical, hreflang, schema, images, links, speed. 23 issue codes, a transparent 0-100 score, optional broken-link check. No API key.

- **URL**: https://apify.com/insight.solutions/seo-page-audit.md
- **Developed by:** [Insight Solutions](https://apify.com/insight.solutions) (community)
- **Categories:** SEO tools, Marketing, Developer tools
- **Features:** Limited permissions
- **Stats:** 9 total users, 4 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.60 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit Tool — Sitemap Crawl & On-Page Checker

**Point it at a domain and get every page audited.** This Actor reads a site's `robots.txt`, walks its sitemap — nested indexes, gzip children and relative URLs included — or crawls the site when there is no sitemap, and then audits each page: title, meta description, headings, canonical, `hreflang`, structured data, images, links, response time, page weight. Every page comes back with a list of **issue codes** and a **0–100 score** built from published weights you can check against the table below.

No API key. No browser. No JavaScript execution. **$1.00 per 1,000 pages**, proxy included. `robots.txt`, sitemaps and link checks are free, pages that could not be read are free, and a run that reads nothing costs nothing at all.

### Try it in 30 seconds

**Input** — this is the Store prefill; paste it and run:

```json
{
  "websites": ["basecamp.com"],
  "maxPages": 25
}
```

A bare domain is enough. The run reads `https://basecamp.com/robots.txt`, follows the sitemap it finds, and audits the first 25 URLs listed there — typically well under a minute. Want specific pages instead? Put them in `pageUrls` and nothing is discovered or crawled.

**Output** — the first row is the website's `site` summary: its 0–100 score and the issue codes costing it the most points. Then one row per audited page; the fields you will use most are `url`, `statusCode`, `title`, `metaDescription`, `issues` and `score` (full list under *Output reference*), and, with `checkLinks` on, a `broken-link` row per dead internal link. Anything that could not be read comes back as a free diagnostic row (`ok: false`, `errorType`, `error`) instead of a charge.

**Price** — $1.00 per 1,000 pages on the FREE tier (+$0.002 per site summary, +$0.0002 per broken link found, +$0.001 per run); `robots.txt`, every sitemap file, every link check, a page that could not be read and a run that reads nothing are all free. Pay-per-event, no API key, no browser, limited permissions — works over the Apify MCP server (`mcp.apify.com`) and with agentic (x402) payments.

**From code** — `client.actor("insight.solutions/seo-page-audit").call(run_input={"websites": ["basecamp.com"], "maxPages": 25})` with `apify-client`, or `POST https://api.apify.com/v2/acts/insight.solutions~seo-page-audit/run-sync-get-dataset-items`.

### What comes back

Every website starts with its `site` row — the summary, written first so it is the first thing you see (shape shown; the numbers are illustrative):

```jsonc
{
  "ok": true,
  "rowType": "site",
  "website": "basecamp.com",
  "url": "https://basecamp.com",
  "score": 91,                             // mean of the page scores
  "issues": ["missing-description", "thin-content", "title-too-short"],
                                           // the codes costing the site the most points: pages × weight
  "pagesAudited": 25,
  "pagesFailed": 0,
  "discovery": "sitemap",
  "robotsFound": true,
  "robotsBlocked": 0,
  "sitemapUrls": ["https://basecamp.com/sitemap.xml"],
  "sitemapUrlCount": 174,                  // URLs read from the sitemap, not just the 25 audited
  "statusCounts": { "200": 24, "404": 1 },
  "avgResponseMs": 268,
  "issueCounts": { "missing-description": 6, "thin-content": 2, "title-too-short": 1 },
  "worstPages": [
    { "url": "https://basecamp.com/…", "score": 42, "issues": ["http-error", "missing-h1", "…"] }
  ],
  "linksChecked": 0,
  "linksUnchecked": 0,                     // links left in the queue when the time budget ran out
  "brokenLinkCount": 0,
  "proxyUsed": "datacenter"                // or "residential" — see "Sites that answer nothing"
}
```

Then one `page` row per audited URL. This one is real — read from basecamp.com/pricing on 2026-09-11:

```jsonc
{
  "ok": true,
  "rowType": "page",                       // "page" | "site" | "broken-link" | "diagnostic"

  "input": "basecamp.com",
  "website": "basecamp.com",
  "url": "https://basecamp.com/pricing",
  "source": "sitemap",                     // "sitemap" | "crawl" | "input"
  "sitemapLastmod": "2026-05-26T12:00:00-07:00",

  // the response
  "statusCode": 200,
  "finalUrl": "https://basecamp.com/pricing",
  "redirectChain": [],
  "responseTimeMs": 243,
  "contentType": "text/html",
  "contentLength": 65209,                  // bytes of HTML
  "compressed": true,
  "lastModified": "Wed, 02 Sep 2026 23:45:00 GMT",
  "hsts": false,
  "xRobotsTag": null,

  // the markup
  "title": "Basecamp — Pricing",           // — decoded to what a reader sees
  "titleLength": 18,
  "metaDescription": "Start free with Basecamp — one project, five users, forever free…",
  "metaDescriptionLength": 128,
  "metaRobots": "index, follow",
  "canonical": "https://basecamp.com/pricing",
  "canonicalMatchesUrl": true,
  "lang": "en",
  "h1": ["No per-user fees, everyone's included. Simple fixed prices — another Basecamp exclusive."],
  "h1Count": 1, "h2Count": 9, "h3Count": 3,
  "wordCount": 2060,
  "textToHtmlRatio": 18.88,                // visible chars per 100 chars of markup
  "ogTitle": "Pricing",
  "ogDescription": "Start free with Basecamp — one project, five users, forever free…",
  "ogImage": "https://basecamp.com/assets/images/opengraph/basecamp.png?v=1.5.1",
  "twitterCard": "summary_large_image",
  "hreflang": [],
  "structuredDataTypes": ["WebPage", "WebSite", "ItemList", "SiteNavigationElement"],
  "hasViewport": true,
  "charset": "utf-8",
  "favicon": "https://basecamp.com/assets/images/general/favicon.svg?v=1.5.1",
  "internalLinkCount": 35,
  "externalLinkCount": 12,
  "nofollowCount": 0,
  "imageCount": 1,
  "imagesMissingAlt": 0,
  "imagesMissingAltExamples": [],
  "scriptCount": 10, "stylesheetCount": 48, "inlineStyleCount": 0,
  "noindex": false, "nofollowPage": false,
  "mixedContentCount": 0,
  "mixedContentExamples": [],

  // the verdict
  "issues": ["title-too-short"],
  "issueCount": 1,
  "score": 95,                             // 100 − 5 for title-too-short

  "proxyUsed": "datacenter",
  "error": null,
  "errorType": null,
  "scrapedAt": "2026-09-11T15:44:37.120Z",
  "sourceUrl": "https://basecamp.com/pricing"
}
```

And, with `checkLinks` on, one `broken-link` row per dead internal link, naming the pages that point at it. The run's status message says the same thing in one line for a single-website run — `Audited 25 pages across 1 website — basecamp.com scores 91/100; fix first: missing-description (6 pages), thin-content (2 pages)`.

### The issue codes, and what each costs

Every page starts at **100**. Each issue found subtracts its weight. The total is floored at 0.

```
score = max(0, 100 − Σ weight(issue))
```

That is the whole formula. No curve, no weighting by page importance, nothing calibrated against anything. It is a checklist with a running total, published here so that fixing a missing `<h1>` and predicting exactly ten points is possible — and so that if you disagree with a weight you can recompute the column yourself from the `issues` array, which is the part actually worth acting on.

| Code | −pts | Fires when |
|---|---:|---|
| `http-error` | 40 | The page answered 4xx or 5xx — a 404 in the sitemap, a 500 on a live URL. It still gets a full row: a broken page you can see is the point. (A 401, 403 or 429 is a *refusal*, not a page; those are free diagnostic rows instead) |
| `noindex` | 25 | The robots meta tag or the `X-Robots-Tag` header says `noindex` or `none` |
| `missing-title` | 20 | No `<title>`, or an empty one |
| `missing-description` | 10 | No `<meta name="description">` |
| `missing-h1` | 10 | No `<h1>` on the page |
| `duplicate-title` | 10 | Another audited page on the same site publishes this exact title |
| `thin-content` | 10 | Fewer than **200** words of visible body text |
| `mixed-content` | 8 | An https page loads at least one asset over plain http |
| `missing-viewport` | 5 | No viewport meta tag, so the page will not adapt to a phone |
| `multiple-h1` | 5 | More than one `<h1>` |
| `missing-canonical` | 5 | No `rel="canonical"` link |
| `canonical-mismatch` | 5 | The canonical points somewhere other than the URL that served the page |
| `images-missing-alt` | 5 | At least one `<img>` has no `alt` attribute at all |
| `slow-response` | 5 | The page took longer than **2,000 ms** to answer |
| `duplicate-description` | 5 | Another audited page on the same site publishes this exact description |
| `title-too-long` | 5 | Title longer than **60** characters |
| `title-too-short` | 5 | Title shorter than **20** characters |
| `missing-lang` | 3 | No `lang` attribute on `<html>` |
| `description-too-long` | 3 | Description longer than **160** characters |
| `no-structured-data` | 3 | No JSON-LD on the page |
| `not-compressed` | 3 | The response was sent uncompressed — and only when the headers prove it |
| `redirected` | 3 | The URL in the sitemap (or the one you sent) redirected somewhere else |
| `large-html` | 3 | The HTML document alone is over **1 MB**, before any assets load |

Four of those are more careful than they look, and it matters:

- **`alt=""` is not a missing alt.** It is the correct, deliberate markup for a decorative image. Counting it would file a finding against every site that gets accessibility right. Only a missing attribute counts.
- **`mixed-content` counts assets, not links.** An `<a href="http://…">` is the reader's choice and no browser warns about it. The captured apify.com pricing page contains exactly one `http://` URL — the LinkedIn link in the footer — and correctly reports zero mixed content.
- **`canonical-mismatch` normalises before comparing.** A canonical differing only by a trailing slash or a fragment is not a mismatch.
- **`not-compressed` fires only on proof.** The HTTP client inflates compressed responses transparently and deletes `content-encoding` and `content-length` when it does, so "no encoding header" is not evidence of anything. This fires only when a declared `content-length` matches the body that arrived — meaning nothing was inflated. When the headers do not settle it, `compressed` is `null` and no issue is raised. A false "you are not compressing" on every page of every site would be a worse bug than missing the occasional real one.

### Use cases

- **Pre-launch and post-migration QA** — run the sitemap after a deploy and diff the `issueCounts` histogram against last week's. A migration that quietly shipped `noindex` on 400 pages shows up as a single number.
- **Agency audits and pitches** — walk in with a scored page-by-page table of a prospect's site, sorted by `score`, with the ten worst named on the summary row.
- **Content-team backlog** — export the `issues` view, filter to `missing-description` and `thin-content`, and you have a prioritised list of pages to write.
- **Broken-link sweeps** — `checkLinks: true` on a site you are responsible for, then read the `links` view. Each row names the broken URL and up to ten pages that link to it, which is normally enough to identify the template at fault.
- **Duplicate-content detection** — `duplicate-title` and `duplicate-description` are computed across the whole site, so faceted listings and paginated archives that all share one title are found in one run.
- **Competitive teardown** — audit a competitor's sitemap and compare `wordCount`, `structuredDataTypes` and `textToHtmlRatio` distributions against your own.
- **Monitoring** — schedule it weekly against `maxPages: 100` and watch the site `score` as a single trend line.

### How it works, and why it keeps working

A website tells you which pages it has in two places, and this Actor reads both before it audits anything.

**1. `robots.txt`.** The first request of every website is `GET /robots.txt`. It does three jobs: it proves which of the four candidate addresses (`https://`, `https://www.`, `http://`, `http://www.`) the host actually answers on (a site that answers on none of them is covered under *Sites that answer nothing*, below); it supplies the `Sitemap:` lines; and it supplies the `Disallow` rules, which are obeyed. Rule matching is the same longest-match-wins algorithm search engines use, with `*` and `$` patterns, matched against path *and* query — so `Disallow: /search?` blocks `/search?q=shoes` and leaves `/searching` alone, and `Allow: /private/public-notes` correctly overrides `Disallow: /private/`. Only the `User-agent: *` group is read; obeying Googlebot's rules would be pretending to be Googlebot.

**2. The sitemap.** Whatever `robots.txt` named, or failing that `/sitemap.xml`, `/sitemap_index.xml`, `/sitemap-index.xml` and `/sitemap/sitemap.xml` in that order. A `<sitemapindex>` is followed to its children, three levels deep, up to 50 sitemap files per site. Children ending `.xml.gz` are inflated. `<loc>` values are resolved against the sitemap's own URL, because the spec says they are absolute and real sitemaps disagree — basecamp.com's, captured live, lists `/`, `/2` and `/2/integrations`. `<lastmod>`, `<changefreq>` and `<priority>` ride along onto the page row. A guessed path that is not there is not a fault — a site with no sitemap is the case the crawl fallback exists for, and it produces no diagnostic row; a sitemap that `robots.txt` or a sitemap index *declared* and that then fails does.

**3. No sitemap? Crawl.** A site that answers nothing useful at any of those locations is crawled breadth-first from its home page: same host with `www.` folded away, HTML pages only, shallowest links first, fragments and `mailto:` and asset paths skipped, bounded by `maxPages` and by `robots.txt`.

Then each page is fetched with a browser `accept` header, redirects followed, a 20-second per-request timeout, up to `maxConcurrency` at once — and **150–400 ms apart per host, always**. A 500-page audit arriving as 500 simultaneous requests is a denial-of-service attempt with a friendly user agent; raising `maxConcurrency` overlaps the waiting, not the hammering.

Three decisions are worth spelling out, because they are what separates a useful audit from a noisy one:

**A 404 is a row, not an error.** A URL in a sitemap that answers 404 is one of the most useful things an audit can find. It comes back as a full `page` row — status, title, response time, the `http-error` issue, a score — and it is charged, because it is a finding. Two cases are deliberately *not* rows: a URL that answered nothing at all, and a URL that was **refused** (401, 403, 429, or a Cloudflare origin error). A refusal is very often about this exit IP rather than about the URL, and billing a row that scores 0 because a bot wall answered would be charging for the wall. Both are free diagnostic rows.

**Duplicate checks run at the end of a site.** `duplicate-title` and `duplicate-description` cannot be decided while pages are still arriving, so every page of a website is buffered until the site is finished, the title and description index is built, and then all the page rows are written at once followed by the site summary. The buffer holds extracted fields, never HTML.

**The site row comes first.** It is computed last — it needs every page — but it is written first, so the first row of a website in the dataset, the Overview table and an export is its score and the issues worth fixing first. It is still *charged* last, after the pages it summarises.

**The word count is a word count.** Scripts, styles, SVG, `<template>` and the whole `<head>` are removed before counting, so a client-rendered page shipping 400 KB of JSON state in a script tag does not read as 60,000 words of content. The captured apify.com pricing page is 437 KB of markup carrying 2,109 real words, and `textToHtmlRatio: 2.81` says so.

### Sites that answer nothing, and the residential retry

Some sites drop every connection from cloud and datacenter addresses; some are simply down. Either way the Actor used to find out the slow way — a 15–20-second timeout per request, one after another: a real run on 2026-09-21 spent **2 min 19 s** on robots.txt, the sitemap and the home page of one such site and ended with nothing. Now two rules bound it:

- **One waiting budget per website: 45 s.** robots.txt on every rung of the ladder, the first sitemap requests, the home page and — until one page of the site has answered — the first page requests share 45 s of time spent on requests that got *no answer*. Each request's timeout is cut to what is left, and a timed-out request is not retried in place. Requests that answered — quickly, slowly, or with a 404 — cost the budget nothing, so a slow site that *is* answering is never cut short by it.
- **Two strikes per host.** After two consecutive connection failures on one host (a timeout, a reset, a refused connection), no further URL on that host is requested. A name that does not resolve is not a strike — it is an instant, final answer.

Then, **when the run uses Apify's datacenter proxy (the default)** and the site gave no usable answer — every request timed out, was reset, or met a bot wall (HTTP 403, 429 or 503, or an anti-bot interstitial, after at least one retry from a fresh datacenter IP) — the home page gets **one** 15-second request through a residential exit. If it answers, the rest of that website (robots.txt, sitemap, pages, link checks) is read through residential, and every row of the site says `proxyUsed: "residential"`. If it does not, the website gets **one** free `unreachable` (or `blocked`) row saying exactly what was tried, how each attempt failed, and how long it all took — instead of a row per URL nobody answered. The same check runs over the first pages of a sitemap: until one page of the site answers, two pages that hang in a row stop the audit and the residential request gets its turn, and the pages that failed are re-read through residential first.

Worst cases, end to end:

| The site… | Datacenter | Residential retry | Total | Before |
|---|---|---|---|---|
| answers nothing at all | robots.txt on `https://`, `https://www.`, `http://` — 15 s each, 45 s; the fourth rung is not tried | 15 s | **≈ 60 s**, one free row | 2 min 19 s |
| answers robots.txt, then hangs | two sitemap requests, 15 s each, then the host is not asked again — home page included | 15 s | **≈ 45–50 s** | up to ≈ 2¾ min |
| answers robots.txt and its sitemap, but no page | the first pages, 20 s each, until two in a row have hung | 15 s | **≈ 55 s** | until `maxRunSecs` |
| does not resolve | four instant DNS failures | none — no exit can make a name exist | **≈ 1 s** | ≈ 1 s |

Without a residential retry (your own proxy URLs, no proxy, or already on RESIDENTIAL) take 15 s off; the row then says why no retry was made. Either way the run finishes **SUCCEEDED**, and if no website produced a page it has zero results, a status message naming the diagnostic rows, and a bill of **$0.00**, start fee included.

**What the residential retry costs.** Apify prices residential proxy traffic by the gigabyte (about $8 per GB; the datacenter proxy is not metered that way). This Actor is priced per event with the proxy included, so **that cost is ours, not yours**: a 100-page audit that needed the residential retry costs you the same **$0.103** (FREE tier: 100 pages, one site summary, one run start) as any other 100-page audit. For scale, its residential traffic is roughly 3–6 MB — about $0.02–$0.05 — absorbed by the page price. Link checks on a rescued site go through residential too, at the same unchanged prices.

### How it compares

- **The sitemap is the input, not a crawl budget.** Most page-level auditors make you either paste URLs one at a time or buy a crawler subscription. This takes a domain and does the discovery — including the gzip and nested-index cases that break naive readers.
- **The score is arithmetic you can audit.** Every weight is in the table above and in `src/audit.js` as one object. Nothing is hidden behind a model, and the `issues` array is always there to recompute from.
- **Priced by the page, with the plumbing free.** `robots.txt`, every sitemap file, every gzip child and every link check cost nothing. You pay for audited pages and one summary row.
- **Failures are free and legible.** A dead host, a URL that answers with a PDF, a bot wall, or a site that ran out the clock produces a diagnostic row with an `errorType` you can branch on — and no charge. A run that audits nothing at all finishes **SUCCEEDED with zero results**, a status message that says so, and a free diagnostic row per URL telling you why; it finishes **FAILED** only when no entry was usable, or the Actor itself hit an error.
- **No browser.** Auditing is pure HTTP and targeted regular expressions, which is why a 5,000-page site costs single-digit dollars rather than hundreds — and why a page whose content only exists after client-side rendering is a documented limitation rather than a hidden one.

### Input reference

| Field | Type | Default | What it does |
|---|---|---|---|
| `websites` | array of strings | `["basecamp.com"]` | Bare domains or full URLs. Each is tried `https://` → `https://www.` → `http://`. Per site: robots.txt, then sitemap discovery, then up to `maxPages` page audits. Duplicates are read, and billed, once |
| `pageUrls` | array of strings | `[]` | Audit exactly these URLs — no sitemap, no crawl. URLs on the same host as a `websites` entry are folded into that site's audit rather than becoming a second one |
| `maxPages` | integer 1–5000 | `100` (the form starts at `25`) | Pages audited **per website**. This is the number that sets the price of a run |
| `checkLinks` | boolean | `false` | Also fetch every unique internal link found (a plain `GET`), and write a `broken-link` row for each one that answers 4xx, 5xx or nothing. $0.0002 per broken link; checking is free. Links the time budget cuts off are counted in `linksUnchecked`, never as broken |
| `maxLinksToCheck` | integer 1–5000 | `500` | Ceiling on the link check, per website |
| `respectRobots` | boolean | `true` | Obey the site's own `robots.txt`. Skipped URLs are counted as `robotsBlocked` and never fetched or billed. Turn off only for a site you own |
| `maxSitemapUrls` | integer 1–5000 | `5000` | Ceiling on URLs collected from sitemaps. Collection also stops as soon as there are enough to fill `maxPages` |
| `maxConcurrency` | integer 1–16 | `8` | Pages of one website in flight at once. Each worker keeps its own proxy session. Per-host spacing of 150–400 ms applies regardless |
| `maxRunSecs` | integer 30–3600 | `240` | Whole-run wall-clock budget. When it runs out the Actor keeps and bills what it has, and files a free diagnostic row saying how many URLs it never reached |
| `proxyConfiguration` | object | Apify datacenter | Datacenter is the default because it is cheaper and ordinary websites accept it. On it, a site that answers nothing over datacenter gets one residential request to its home page, and is read through residential if that answers (*Sites that answer nothing*). Switch the group to `RESIDENTIAL` to use residential for everything |

At least one of `websites` and `pageUrls` must be non-empty. A run with neither finishes FAILED, having charged nothing.

### Output reference

Every row carries the same keys, whatever its type. `ok: true` is a `page`, `site` or `broken-link` row; `ok: false` is a free diagnostic row.

**Common to every row**

| Field | What it is |
|---|---|
| `ok`, `rowType` | Whether this row is a result, and which of the four kinds it is |
| `input`, `website` | The entry you supplied, and the registrable domain it resolved to |
| `url`, `sourceUrl` | The URL this row is about |
| `source` | Where the URL came from: `sitemap`, `crawl` or `input` on a page row; `site` on a summary; `link-check` on a broken link |
| `proxyUsed` | Which exit read this website: `datacenter`, `residential` (your setting, or the residential retry), `custom` (your own proxy URLs) or `none`. The price is the same whichever it is |
| `error`, `errorType`, `scrapedAt` | Why a row failed, if it did, and when the row was written |

**`page` rows**

| Field | What it is |
|---|---|
| `statusCode`, `finalUrl`, `redirectChain`, `responseTimeMs` | What the server answered, where it ended up, and how long it took |
| `contentType`, `contentLength`, `compressed`, `lastModified`, `hsts`, `xRobotsTag` | Response headers worth an audit column. `contentLength` is bytes of HTML received |
| `title`, `titleLength`, `metaDescription`, `metaDescriptionLength`, `metaRobots` | The tags search results are built from, entity-decoded |
| `canonical`, `canonicalMatchesUrl`, `lang`, `charset`, `hasViewport`, `favicon` | Indexing and rendering signals |
| `h1`, `h1Count`, `h2Count`, `h3Count` | Heading text and structure |
| `wordCount`, `textToHtmlRatio` | Visible body text, and how much of the document it is |
| `ogTitle`, `ogDescription`, `ogImage`, `twitterCard` | Social cards |
| `hreflang`, `structuredDataTypes` | Alternates as `{lang, href}`, and every JSON-LD `@type` including nested ones |
| `internalLinkCount`, `externalLinkCount`, `nofollowCount` | Unique links, split by host with `www.` folded away |
| `imageCount`, `imagesMissingAlt`, `imagesMissingAltExamples` | Images, and up to twenty `src` values with no `alt` attribute |
| `scriptCount`, `stylesheetCount`, `inlineStyleCount` | Front-end weight |
| `noindex`, `nofollowPage`, `mixedContentCount`, `mixedContentExamples` | Indexing directives and http assets on an https page |
| `issues`, `issueCount`, `score` | The verdict: every issue code found, heaviest first, how many, and the 0–100 score |
| `sitemapLastmod`, `sitemapChangefreq`, `sitemapPriority` | What the sitemap said about this URL, when it came from one |

**`site` rows** — the first row of every website

| Field | What it is |
|---|---|
| `pagesAudited`, `pagesFailed` | How many URLs produced a row, and how many could not be read |
| `discovery` | `sitemap`, `crawl` or `input` |
| `robotsFound`, `robotsBlocked` | Whether robots.txt was readable, and how many URLs it kept out |
| `sitemapUrls`, `sitemapUrlCount` | Which sitemap files were read, and how many URLs came out of them |
| `statusCounts`, `avgResponseMs` | Status histogram and mean response time |
| `issueCounts` | How many pages triggered each issue code, heaviest first |
| `score` | The rounded mean of the page scores |
| `issues` | The (at most five) issue codes costing the site the most points — pages affected × weight — worst first. `issueCount` stays empty here |
| `worstPages` | The ten lowest-scoring pages, each with its score and issues |
| `linksChecked`, `linksUnchecked`, `brokenLinkCount` | Link-check totals, zero unless `checkLinks` is on. `linksUnchecked` is what the time budget cut off |

**`broken-link` rows** carry `url`, `statusCode`, `error` and `foundOn` — up to ten audited pages that link to the dead URL.

`errorType` on a diagnostic row is one of:

| Value | Meaning | Charged? |
|---|---|---|
| `unreachable` | Nothing answered — a dead host, a connection that failed, a domain that does not resolve, the proxy's own "host not found" page. When a whole website answered nothing, this is **one** row for the site that lists every attempt, the residential retry and the time spent | No |
| `blocked` | The request was refused — 401, 403, 429, a Cloudflare origin error, or an anti-bot interstitial served in place of the page. When every page and the home page were refused over datacenter *and* residential, this is one row for the site | No |
| `not-html` | The URL answered with a file rather than a page — a PDF, an image, a JSON feed | No |
| `timeout` | The run's `maxRunSecs` budget ran out before this URL was read | No |
| `memory` | The run's memory ran low, so the audit stopped fetching and delivered the pages it had read; the row says how many URLs were left | No |
| `invalid-input` | The entry was not a fetchable website or URL — an email address, an IP, a bare word | No |

### Pricing

**$1.00 per 1,000 pages.** Pay-per-event, with the proxy already inside that number — there is no separate proxy line on your bill for this Actor.

| Event | What triggers it | FREE | Starter | Scale | Business |
|---|---|---|---|---|---|
| **Page audited** *(primary)* | One audited page written to your dataset | $0.001 | $0.001 | $0.0008 | $0.0006 |
| Site summary | One website that produced at least one page | $0.002 | $0.002 | $0.002 | $0.002 |
| Broken link found | One internal link that failed, only when `checkLinks` is on | $0.0002 | $0.0002 | $0.0002 | $0.0002 |
| Run started | Once per run, after the first page is audited | $0.001 | $0.001 | $0.001 | $0.001 |

**Worked example.** A weekly audit of a 500-page site with link checking on, finding 30 broken links:

- 500 pages × $0.001 = **$0.500**
- 1 site summary × $0.002 = **$0.002**
- 30 broken links × $0.0002 = **$0.006**
- 1 run start = **$0.001**
- **Total: $0.509**

What you are never charged for: `robots.txt`, any sitemap file however many there are, any link check (only a *broken* link produces a billed row), a URL that answered nothing, a URL that answered with a file, a URL that was refused with 401/403/429 or behind a bot wall, a URL `robots.txt` disallowed, an entry that was not a website, anything the run never reached before `maxRunSecs`, or the residential traffic of a site that needed the residential retry. If a whole run comes back empty it finishes SUCCEEDED with zero results and a status message naming the free diagnostic rows, and bills **nothing at all**, start fee included.

Set `ACTOR_MAX_TOTAL_CHARGE_USD` on a run and the Actor stops fetching once the ceiling is in sight, rather than handing you rows it cannot bill or billing you for rows it cannot hand over. It finishes SUCCEEDED with the ceiling named in its status message, and everything already delivered is yours.

### Limits, and the ones that might bite

**No browser, so the audit is of the HTML the server returned.** A page whose body copy is rendered client-side will read as `thin-content` with a low `textToHtmlRatio`, and that is an honest description of what a crawler receives — but it is not what a human sees, and treating it as a content problem would be wrong. The `wordCount` and `textToHtmlRatio` columns together are the tell: a low ratio on a big document means a framework, not an empty page.

**`sitemapUrlCount` is what was read, not everything the site publishes.** Sitemap collection stops as soon as there are enough URLs to fill `maxPages`, at 50 sitemap files, at three levels of index nesting, or at `maxSitemapUrls`. Reading a news site's entire sitemap tree to audit 100 pages would be rude as well as slow. Raise `maxPages` and `maxSitemapUrls` together when you want a complete count.

**`redirectChain` has two ends, not every hop.** The HTTP client follows redirects itself and hands back only the URL it finished on, so a page that went through three hops reports the URL requested and the URL delivered. The `redirected` issue and `finalUrl` are correct; the intermediate hops are not visible.

**Duplicate checks are within one run and one site.** `duplicate-title` compares the pages *this run audited on this website*. Two pages sharing a title where only one was audited will not be flagged, and a title shared across two different domains in the same run is not a duplicate.

**`compressed` can be `null`.** See the `not-compressed` note above. When the response carried neither `content-encoding` nor `content-length` and the body was empty, nothing can be concluded, and the column says so rather than guessing.

**Memory: the default 1 GB is enough for the largest audit this Actor allows.** Page rows are buffered per website until the site finishes (the duplicate checks need every page first), and the buffer holds extracted fields, never markup — roughly 20–50 KB per page, so 5,000 pages fit comfortably in the default run memory. If the process ever does run low, the audit stops *fetching* rather than dying: every page already read is still scored, written and summarised, and a free `memory` diagnostic says how many URLs were left and that raising the run's memory or lowering `maxPages` will finish the job. A run that is killed by the platform delivers nothing; this one never gets there.

**A crawl is not a full-site crawl.** The fallback follows internal HTML links breadth-first from the home page up to `maxPages`. It does not submit forms, does not paginate listings by guessing URL patterns, and does not leave the host. A site with no sitemap and a JavaScript-only navigation will yield one page: its home page.

**Scores are a checklist total, not a ranking prediction.** Nothing here measures backlinks, domain authority, search intent, competitor positioning or whether the content is any good. A page can score 100 and rank nowhere. The score is for triage — sort by it, then read the `issues`.

**Sites behind aggressive bot management may refuse the default proxy.** The datacenter default is right for the overwhelming majority of the web and wrong for the small slice that fingerprints every request. When a whole site refuses or ignores datacenter, the residential retry above usually gets through on its own. It is one request per website and it is only made when *nothing* answered: a site whose home page answers over datacenter but that refuses some of its other pages keeps those pages as free `blocked` diagnostic rows — switch `proxyConfiguration` to `RESIDENTIAL` and re-run if you need them. A site that refuses the residential exit too (or for which none could be created) gets one free `blocked` row and no further requests.

**A site that answers nothing still costs about a minute.** That is the waiting budget plus the residential retry (see the timeline above). A list of fifty dead domains is therefore up to fifty minutes, not fifty seconds — raise `maxRunSecs` accordingly; a website the clock does not reach gets a free `timeout` row.

**The upstream format may change.** Sites are rebuilt, sitemaps move, and a site that published a clean `sitemap.xml` last month can start serving an index of gzip children this month. When that happens the discovery path changes rather than breaking — the `discovery` and `sitemapUrls` columns on the site row are how you see which path was taken.

### Use it from an AI agent, or from code

One JSON object in, one flat array out — the shape agent runtimes want. The Actor runs with **limited permissions**, uses **pay-per-event** pricing and never enters Standby, so it works over the Apify MCP server and with x402 agentic payments. The **Integrations** tab pushes results to Slack, a webhook, Zapier, Make, Google Sheets, Snowflake or BigQuery.

```bash
curl -X POST "https://api.apify.com/v2/acts/insight.solutions~seo-page-audit/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"websites":["basecamp.com"],"maxPages":25}'
```

```python
## pip install apify-client
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("insight.solutions/seo-page-audit").call(run_input={
    "websites": ["basecamp.com"],
    "maxPages": 200,
    "checkLinks": True,
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] == "page" and row["score"] < 80:
        print(row["score"], row["url"], "→", ", ".join(row["issues"]))
    elif row["rowType"] == "site":
        print("site score:", row["score"], "over", row["pagesAudited"], "pages")
    elif row["rowType"] == "broken-link":
        print("broken:", row["url"], "linked from", row["foundOn"][0])
```

The four dataset views — **Overview**, **Pages with issues**, **Broken links** and **Site summaries** — are set up in the Console so each row type exports as its own clean table.

### FAQ

**How does it find the pages?**
`robots.txt` first, for the `Sitemap:` lines and the crawl rules. Then those sitemaps, or `/sitemap.xml` and three other usual locations. Sitemap indexes are followed to their children, gzip children are inflated, and relative `<loc>` values are resolved. Only if none of that yields anything does it crawl the site from its home page.

**Does it respect robots.txt?**
Yes, by default, using the `User-agent: *` group and the longest-match-wins rule search engines use. Disallowed URLs are never fetched, counted as `robotsBlocked` on the site row, and never billed. `respectRobots: false` exists for sites you own — for example a staging site that blocks everything.

**How fast is it?**
Requests to one host are spaced 150–400 ms apart, so about three pages a second per site plus the time each page takes to answer, overlapped across `maxConcurrency` workers. The 25 pages the form starts with take seconds; 50 pages of a typical site is well under a minute. A site that answers nothing costs about a minute before its free diagnostic row (*Sites that answer nothing*). `maxRunSecs` is the hard stop.

**Why did a page I know exists not get audited?**
Four possibilities, in order of likelihood: it is not in the sitemap; `maxPages` was reached before the run got to it; `robots.txt` disallows it; or it answered with something that is not a web page. The site row's `sitemapUrlCount`, `pagesAudited` and `robotsBlocked` distinguish the first three, and there is a free diagnostic row for the fourth.

**What counts as a duplicate title?**
An exact string match, after entity decoding and whitespace collapsing, between two pages audited on the same website in the same run. Near-duplicates are not detected — "Shoes | Acme" and "Shoes - Acme" are two different titles.

**Can I audit just a handful of specific pages?**
Yes. Put them in `pageUrls` and leave `websites` empty. No sitemap is fetched and no crawl happens; only `robots.txt` is read, because the Disallow rules still apply.

**Does `checkLinks` check external links too?**
No. Only internal links, deduplicated across the whole site, capped at `maxLinksToCheck`. Checking outbound links would multiply the request count against sites that did not ask to be crawled.

**Can I audit 5,000 pages?**
Yes, that is the per-website ceiling. Raise `maxRunSecs` to match — 5,000 pages at three a second is roughly half an hour — and bound the spend with `ACTOR_MAX_TOTAL_CHARGE_USD`. At $0.001 per page the whole site is $5.

**What happens if one website in a list fails?**
The others still run. The failed one produces a free diagnostic row — one row for the site, saying what was tried and for how long — and the run finishes `SUCCEEDED`. If *every* website fails — a site that does not exist, one that refused us, one whose `robots.txt` disallows every page — the run still finishes `SUCCEEDED`, with zero results, a status message that says so and a free diagnostic row per failure, and you are billed nothing at all, start fee included. It finishes `FAILED` only when nothing you sent is a website or page URL, or the Actor itself hit an error.

**Is the data fresh?**
Live. Every run reads the site at that moment; nothing is cached, and there is no index behind this.

### Legal and data-protection notes

- **Public pages only.** Every URL fetched is one a browser would load without signing in. The Actor never logs in, never accepts cookies or session tokens, never takes an API key belonging to anyone else, and never touches anything behind authentication.
- **It obeys the site's own instructions by default.** `robots.txt` is read before anything else and its `Disallow` rules are respected unless you deliberately turn that off for a site you control.
- **It is a polite crawler.** One request at a time per host with 150–400 ms between them, a 20-second per-request timeout, and a whole-run budget. It does not retry a refusal in a loop.
- **This is not personal data.** An on-page SEO record describes a web page's markup, not a person. Nothing in the output identifies an individual, and no contact details, names or user records are collected.
- **Scores are observations, not advice.** "This page has no meta description" is what was seen. What to do about it depends on the page, and the issue list is there precisely so the number never has to be taken on faith.
- **Not affiliated with any website audited or named in these docs.** All trademarks belong to their respective owners.

### Our other Actors

Every Insight Solutions Actor is pay-per-result with no browser, no login and no API key, and every one of them returns free diagnostic rows instead of billing for failures. Prices are per 1,000 results.

**Video, audio & social**

- [YouTube Transcript Scraper & API — Captions to Text, SRT & VTT](https://apify.com/insight.solutions/youtube-transcript-api) — captions as timed segments, text, SRT or VTT, with language fallback and translation.
- [YouTube Comments Scraper — Comments & Replies with Likes](https://apify.com/insight.solutions/youtube-comments-api) — comments and replies with likes, pinned and hearted flags, newest or top sort.
- [YouTube Channel Scraper — Videos, Shorts & Search](https://apify.com/insight.solutions/youtube-channel-api) — a channel's videos, Shorts and live streams, plus YouTube search.
- [Podcast Search, Episodes & Charts API](https://apify.com/insight.solutions/podcast-api) — Apple Podcasts search, charts and full episode feeds.
- [Bluesky Scraper](https://apify.com/insight.solutions/bluesky-scraper) — profiles, posts, followers and follows from the public AT Protocol API.
- [Telegram Channel Scraper — Posts, Views & Stats](https://apify.com/insight.solutions/telegram-channel-scraper) — posts, views and channel stats from public Telegram channels.
- [Substack Scraper](https://apify.com/insight.solutions/substack-scraper) — posts with full free text, comments and publication profiles.
- [Sponsor & Advertiser Finder — YouTube, Podcasts & Newsletters](https://apify.com/insight.solutions/sponsorship-intelligence-api) — which brands sponsor which YouTube channels, podcasts and Substack newsletters, with promo codes, offers and sponsor links, company-level only.
- [Google Ads Transparency Center API — Advertisers & Ads](https://apify.com/insight.solutions/google-ads-transparency-api) — every ad a verified advertiser runs on Google, from the Ads Transparency Center, by brand name, AR id or domain: format, first and last shown, days shown and preview links, individuals excluded.
- [Hacker News API](https://apify.com/insight.solutions/hacker-news-api) — stories, comments, users, front page and a structured "Who is hiring?" parser from the official HN APIs.
- [GitHub Trending & Repo Scraper — Stars, Releases & Search](https://apify.com/insight.solutions/github-repos-api) — repository facts, trending lists, releases and search, no token needed for small runs.
- [Discourse Forum Scraper — Topics, Posts & Categories](https://apify.com/insight.solutions/discourse-forum-api) — topics, posts and categories from any Discourse community via its own JSON endpoints, usernames only.

**News, documents & the web**

- [Google News Scraper — Search, Topics & Real Article URLs](https://apify.com/insight.solutions/google-news-api) — news search and topic feeds with the publisher's real URL decoded.
- [Website to Markdown Converter & API — for LLMs & RAG](https://apify.com/insight.solutions/website-content-extractor) — any site as clean Markdown, text and heading-aware chunks.
- [WordPress Scraper — Posts & Pages from Any WP Site, No Login](https://apify.com/insight.solutions/wordpress-posts-api) — posts and pages from any WordPress site as clean text or Markdown, with categories, tags and images.
- [Website Metadata Extractor — Schema.org, Open Graph & JSON-LD](https://apify.com/insight.solutions/website-metadata-api) — any URL's meta tags, Open Graph, Twitter cards, hreflang, feeds, JSON-LD and microdata, with normalised Product, Article, Event and FAQ entities.
- [Subdomain Finder — Certificate Logs & DNS, No API Key](https://apify.com/insight.solutions/subdomain-finder-api) — every subdomain of a domain from certificate-transparency logs, resolved over DNS, with an optional live HTTP check.
- [Internet Archive Scraper & API — Search, Metadata & Files](https://apify.com/insight.solutions/internet-archive-api) — archive.org search, item metadata, files and reviews.
- [Wayback Machine Scraper — Archived URLs, Snapshots & Diffs](https://apify.com/insight.solutions/wayback-toolkit) — archived URL inventories, snapshots and text diffs between dates.
- [Website Technology Detector — Tech Stack Checker & Lookup](https://apify.com/insight.solutions/website-tech-detector) — the tech stack behind any site, with the evidence for each detection.
- [Domain Lookup API — DNS, WHOIS/RDAP, SSL & Providers](https://apify.com/insight.solutions/domain-intelligence-api) — DNS, RDAP registration, TLS certificate and HTTP facts in one row per domain.
- [Keyword Suggestions Tool & API — Google, Bing & Amazon](https://apify.com/insight.solutions/keyword-suggestions-api) — Google, YouTube, Bing, Amazon and eBay autocomplete with alphabet and question expansions.
- [Website Contact Scraper — Email, Phone & Socials Extractor](https://apify.com/insight.solutions/website-contact-extractor) — emails, phone numbers and social profiles from any list of websites.
- [Company Email Finder — Published Emails & Phones by Name](https://apify.com/insight.solutions/company-contact-finder) — company names in, each one's verified website with its business emails, phone numbers and social profiles out.
- [Web Search Results API](https://apify.com/insight.solutions/web-search-api) — Bing and DuckDuckGo organic results with snippets, no key, no browser.
- [Company Enrichment API](https://apify.com/insight.solutions/company-enrichment-api) — a domain in, a company profile out: firmographics, contacts, tech stack, DNS and hiring signal.
- [Company Dossier API](https://apify.com/insight.solutions/company-dossier-api) — one company in, twelve sections out: profile, tech, contacts, DNS, open roles, news, SEC filings, federal awards, recalls, YC batch and apps.
- [Press Release Scraper — GlobeNewswire, PR Newswire & More](https://apify.com/insight.solutions/press-releases-api) — GlobeNewswire and PR Newswire releases plus any newsroom feed, by keyword, company, ticker or subject.
- [Federal Register API](https://apify.com/insight.solutions/federal-register-api) — rules, proposed rules, notices and the Public Inspection desk with dockets, comment deadlines and CFR references.
- [Academic Papers Search API](https://apify.com/insight.solutions/academic-papers-api) — OpenAlex, Crossref, arXiv and PubMed in one row per paper: abstract, citations, open-access PDF, authors and venue.
- [RSS & Atom Feed Monitor](https://apify.com/insight.solutions/rss-feed-monitor) — any RSS, Atom or JSON feed (or an OPML file) in, only the new items out, with keyword filters and a webhook.
- [Website Change Monitor](https://apify.com/insight.solutions/website-change-monitor) — watch any pages, diff the text between runs, get change rows with added/removed lines, keyword alerts and a webhook.
- [Wikipedia & Wikidata API](https://apify.com/insight.solutions/wikipedia-api) — article text, search, daily pageviews and Wikidata entity facts, any language edition.

**Business, finance & jobs**

- [Congress Trades Tracker API — STOCK Act PTRs + SEC Form 4](https://apify.com/insight.solutions/congress-insider-trades-api) — STOCK Act periodic transaction reports and SEC Form 4 insider trades in one schema.
- [Federal Contracts, Grants & Lobbying API](https://apify.com/insight.solutions/federal-contracts-grants-api) — SAM.gov opportunities, USAspending awards, Grants.gov notices and Senate lobbying filings in one schema.
- [SEC EDGAR API](https://apify.com/insight.solutions/sec-edgar-api) — filings, XBRL financials and full-text search by ticker or CIK.
- [Litigation Signals API — Lawsuits, Settlements & Investigations](https://apify.com/insight.solutions/litigation-signals-api) — lawsuits, settlements and investigations per public company from SEC 10-K, 10-Q and 8-K disclosures, press wires and SEC, FTC and DOJ notices, company-level only.
- [Startup Funding Rounds API — SEC Form D, Wires & News](https://apify.com/insight.solutions/funding-signals-api) — who just raised money: one row per funding round from SEC Form D filings, press wires and Google News, with amount, round, investors, industry and state, company-level only.
- [Clinical Trials & FDA API](https://apify.com/insight.solutions/clinical-trials-fda-api) — ClinicalTrials.gov studies plus openFDA recalls, labels, approvals, 510(k)s and adverse-event reports.
- [Product & Vehicle Recalls API](https://apify.com/insight.solutions/product-recalls-api) — CPSC, NHTSA, FDA and USDA recalls, vehicle complaints and ratings, plus a VIN decoder.
- [YC Companies List & Directory Scraper — Batches & Founders](https://apify.com/insight.solutions/yc-companies-directory) — the YC directory with founders and social links, filterable by batch, industry and hiring status.
- [Job Scraper for Greenhouse, Lever, Ashby & 10+ ATS Career Sites](https://apify.com/insight.solutions/ats-jobs-api) — jobs straight from Greenhouse, Lever, Ashby, Workable and 10+ other ATS career sites.
- [Workday Jobs Scraper — All Postings from Any Workday Site](https://apify.com/insight.solutions/workday-jobs-api) — every posting on any Workday career site, with facets, search, posted dates and full descriptions.
- [Job Postings Monitor — New & Closed Roles, Hiring Alerts](https://apify.com/insight.solutions/job-postings-monitor) — new, closed and changed postings on the career sites you watch.
- [Hiring Signals API — Open Roles & Hiring Surge by Company](https://apify.com/insight.solutions/hiring-signals-api) — one row per company per run: open roles, what opened and closed, department and seniority breakdowns, and a hiring-surge flag.
- [Careers Page Scraper — Open Roles from Any Company's Site](https://apify.com/insight.solutions/careers-page-jobs-api) — open roles from any company's own careers site: domain in, the platform recognised (Eightfold, Phenom, Jibe, iCIMS, Google, Amazon, 14 ATSs) or the site's structured data read, one row per job.
- [Wellfound Jobs Scraper — Startup Roles, Salary & Equity](https://apify.com/insight.solutions/wellfound-jobs-api) — startup jobs from Wellfound's role listings, by role and city or remote: salary and equity as shown, remote policy, experience, and each startup's size, stage and badges, no recruiter contacts.
- [Remote Jobs API](https://apify.com/insight.solutions/remote-jobs-api) — RemoteOK, Remotive, We Work Remotely, Himalayas, Jobicy and more in one schema, deduplicated.
- [Shopify Products Scraper — Catalogue, Prices & Stock](https://apify.com/insight.solutions/shopify-products-api) — any Shopify store's catalogue, variants, prices and stock signals.
- [WooCommerce Products Scraper — Prices, Variants & Stock](https://apify.com/insight.solutions/woocommerce-products-api) — any WooCommerce store's catalogue, prices, variants and stock through its public Store API.
- [Price Tracker & Monitor — Any Retailer's Prices & Stock](https://apify.com/insight.solutions/price-tracker-api) — price, stock, list price and discount from any product URL, read from the page's structured data, with price-drop and restock rows between runs.
- [Rightmove Scraper — UK Listings for Sale & Rent by Area](https://apify.com/insight.solutions/rightmove-property-api) — UK property for sale and to rent by area from Rightmove's own search API: price, bedrooms, type, tenure, size, coordinates, photos, status and the agent branch, one row per listing.
- [Shopify App Store Scraper — Apps, Pricing, Ratings & Reviews](https://apify.com/insight.solutions/shopify-app-store-api) — Shopify App Store listings by category, collection or developer: pricing plans, ratings and merchant reviews.
- [Luma Events Scraper — Events by City, Category & Calendar](https://apify.com/insight.solutions/luma-events-api) — upcoming Luma events by city, category or calendar: dates, venue, price, capacity and organiser calendar, no host names.
- [Eventbrite Scraper — Events by City, Category & Organizers](https://apify.com/insight.solutions/eventbrite-events-api) — upcoming Eventbrite events by city, category, online or organiser, with the organising companies behind them: dates, venue, price, ticket status, followers and website, no browser, individuals never named.
- [Meetup Scraper — Upcoming Events & Group Stats](https://apify.com/insight.solutions/meetup-events-api) — upcoming events of the Meetup groups you name, and the groups themselves: dates, venue, RSVPs, fee, member count, topics and ratings, no member data.
- [Shopify Store Monitor](https://apify.com/insight.solutions/shopify-store-monitor) — price drops, sales, restocks, sell-outs and new products on any Shopify store, one row per change.
- [Shopify Store Finder — Discover & Verify Stores by Niche](https://apify.com/insight.solutions/shopify-store-finder) — Shopify stores by niche and country, each verified on its own storefront, with product count, currency, country, price range and sample products.
- [Public Tenders API](https://apify.com/insight.solutions/public-tenders-api) — EU TED, UK Find a Tender and Contracts Finder notices by keyword, CPV code, country, stage and deadline.
- [Nonprofit Lookup & IRS 990 API — EIN, Financials, NTEE](https://apify.com/insight.solutions/nonprofit-990-api) — search US nonprofits and get EIN, NTEE code and multi-year Form 990 financials.
- [OpenStreetMap Places API](https://apify.com/insight.solutions/osm-places-api) — businesses and points of interest by category and area from OpenStreetMap: name, address, coordinates, website, phone, opening hours.

**Apps & games**

- [App Store & Google Play Reviews Scraper — Ratings & Replies](https://apify.com/insight.solutions/app-reviews-api) — reviews from both stores with ratings, versions and developer replies.
- [App Store Top Charts & App Search API](https://apify.com/insight.solutions/app-charts-api) — Apple top charts by country and genre, plus app search and details.
- [App Store Keyword Rank Tracker](https://apify.com/insight.solutions/app-store-keyword-rank-tracker) — where any app ranks for any keyword on the App Store and Google Play, with rank changes and ASO suggestions.
- [Steam Reviews Scraper — Reviews & Game Details](https://apify.com/insight.solutions/steam-reviews-api) — Steam reviews with playtime, helpfulness and game details.
- [Steam Game Data API](https://apify.com/insight.solutions/steam-store-stats-api) — prices, tags, review scores, live player counts and top charts.
- [Sports Scores & Schedules API](https://apify.com/insight.solutions/sports-scores-api) — scores, schedules, standings, teams and headlines for the NFL, NBA, MLB, NHL, soccer and more, no key.
- [App Store Monitor](https://apify.com/insight.solutions/app-store-monitor) — App Store and Google Play listings watched between runs: new versions with release notes, price, rating and listing changes, one row per change.

# Actor input Schema

## `websites` (type: `array`):

One entry per site to audit. A bare domain like `basecamp.com` is enough and a full URL works too; each domain is tried on `https://`, then `https://www.`, then plain `http://`. For each site the Actor reads `robots.txt`, follows the `Sitemap:` lines it finds (or four usual sitemap locations), and audits up to **Maximum pages per website** URLs. A site with no sitemap is crawled from its home page instead. Email and IP addresses are rejected for free.

## `maxPages` (type: `integer`):

How many pages to audit per website. This is the number that sets the price of a run: every audited page is one `page` event. The form starts at 25 — a site summary and 25 scored pages in well under a minute; the default when the field is left out is 100, a useful sample of a mid-size site. Raise it to cover a site completely, lower it to spot-check. Sitemap URLs are taken in the order the sitemap lists them; a crawl takes them breadth-first from the home page, shallowest links first.

## `checkLinks` (type: `boolean`):

Fetch every unique internal link found on the audited pages and report the dead ones. Off by default; turn it on for a link-rot sweep. Each link is fetched once with a plain `GET`, and anything answering 4xx, 5xx or nothing at all gets a `broken-link` row naming the pages that link to it. Links are deduplicated across the whole site, and pages that were audited anyway are skipped. Billed at $0.0002 per link that turns out to be broken; checking itself costs nothing.

## `respectRobots` (type: `boolean`):

On by default, and worth leaving on. The Actor reads the `User-agent: *` group of each site's `robots.txt` and skips any URL it disallows, using the same longest-match-wins rule search engines use, with `*` and `$` patterns supported. Skipped URLs are counted on the `site` row as `robotsBlocked` and are never fetched or billed. Turn it off only for a site you own or have permission to crawl — for example to audit a staging site that blocks everything.

## `maxLinksToCheck` (type: `integer`):

The ceiling on the link check, per website. Only used when **Check internal links** is on. Unique internal links are collected across every audited page and then cut to this number.

## `pageUrls` (type: `array`):

Audit exactly these pages and nothing else — no sitemap, no crawl. Use it when you already know which URLs matter: the twelve landing pages you just shipped, the product pages a client complained about, a list exported from your analytics. URLs on the same host as an entry in **Websites** are folded into that site's audit rather than becoming a second one, so they still count toward its duplicate-title check and its site summary row.

## `maxSitemapUrls` (type: `integer`):

How many URLs to read out of a site's sitemaps before stopping. A large news site's sitemap tree runs to millions of entries and reading all of it to audit 100 pages would be rude as well as slow, so collection also stops as soon as there are enough URLs to fill **Maximum pages per website**. Sitemap indexes are followed three levels deep and at most 50 sitemap files are fetched per website.

## `maxConcurrency` (type: `integer`):

How many pages of one website are fetched at once. Each parallel worker keeps its own proxy session, so a page that refuses one exit IP does not burn the address the others are using. Requests to a single host are spaced 150–400 ms apart whatever this is set to, so raising it buys parallel parsing rather than a burst of simultaneous requests. Websites themselves are always audited one at a time.

## `maxRunSecs` (type: `integer`):

Wall-clock budget for the whole run. When it is reached the Actor stops fetching, keeps and bills for every row it already wrote, and files a free diagnostic row saying how many URLs it never reached. Nothing is charged for work that did not happen.

## `proxyConfiguration` (type: `object`):

Ordinary websites do not refuse datacenter address ranges the way the big platforms do, so the default is Apify's datacenter proxy, and its cost is already inside the per-page price. On that default, a site that answers nothing over datacenter (every request times out, is reset, or meets a bot wall) gets one request to its home page through a residential exit; if that answers, the rest of that site is read through residential and its rows say `proxyUsed: "residential"` — at the same per-page price. To use residential for everything, switch the group to RESIDENTIAL here. A refused exit IP is rotated automatically either way. Clearing the field sends requests from the run's own address, with no residential retry.

## Actor input object example

```json
{
  "websites": [
    "apify.com",
    "https://www.gov.uk"
  ],
  "maxPages": 25,
  "checkLinks": false,
  "respectRobots": true,
  "maxLinksToCheck": 500,
  "pageUrls": [
    "https://basecamp.com/pricing",
    "https://basecamp.com/about"
  ],
  "maxSitemapUrls": 5000,
  "maxConcurrency": 8,
  "maxRunSecs": 240,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

One row per audited page — title, description, headings, canonical, structured data, images, links, speed, issue codes and a 0-100 score — plus a summary row per website and a row for every broken internal link. Delivered as JSON items in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "websites": [
        "basecamp.com"
    ],
    "maxPages": 25,
    "checkLinks": false,
    "respectRobots": true,
    "maxLinksToCheck": 500,
    "pageUrls": [],
    "maxSitemapUrls": 5000,
    "maxConcurrency": 8,
    "maxRunSecs": 240,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("insight.solutions/seo-page-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "websites": ["basecamp.com"],
    "maxPages": 25,
    "checkLinks": False,
    "respectRobots": True,
    "maxLinksToCheck": 500,
    "pageUrls": [],
    "maxSitemapUrls": 5000,
    "maxConcurrency": 8,
    "maxRunSecs": 240,
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("insight.solutions/seo-page-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "websites": [
    "basecamp.com"
  ],
  "maxPages": 25,
  "checkLinks": false,
  "respectRobots": true,
  "maxLinksToCheck": 500,
  "pageUrls": [],
  "maxSitemapUrls": 5000,
  "maxConcurrency": 8,
  "maxRunSecs": 240,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call insight.solutions/seo-page-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,insight.solutions/seo-page-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/IbdSbOhXIRfYOPMKq/builds/PyN0U7knDsYWHqcJC/openapi.json