# Sitemap URL Extractor (`leorochasantos/sitemap-extractor`) Actor

Extract every URL from any XML sitemap — support sitemap indexes, gzip, robots.txt discovery, and metadata extraction. Returns clean structured output: one URL per row.

- **URL**: https://apify.com/leorochasantos/sitemap-extractor.md
- **Developed by:** [Leonardo Santos](https://apify.com/leorochasantos) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 96.4% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.65 / 1,000 sitemap url extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Sitemap URL Extractor

Extract every URL from any XML sitemap — including sitemap indexes, gzip-encoded sitemaps, robots.txt discovery, and optional metadata (lastmod, changefreq, priority). Returns clean structured output: one URL per row, LLM/MCP-ready.

### Why this actor?

- **Reliable** — built for production use with retry logic and transparent error handling
- **Fast** — HTTP-first, 256MB, no headless browser. A typical sitemap extracts in seconds
- **Complete** — supports sitemap indexes (recursive), gzip, robots.txt discovery, and domain convention
- **Cheap** — $0.001 per extracted URL, 60x less than the reliable competitor
- **Clean output** — flat, typed JSON with `url`, `sourceSitemap`, `lastModified`, `changeFrequency`, and `priority`

### Input

Provide a sitemap URL or a domain name:

```json
{
  "url": "https://apify.com/sitemap.xml"
}
```

Or let the actor discover the sitemap automatically:

```json
{
  "url": "https://apify.com",
  "followIndexes": true,
  "maxDepth": 3,
  "maxUrls": 10000,
  "extractMetadata": true
}
```

#### Input fields

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | *required* | Sitemap URL or domain name |
| `followIndexes` | boolean | `true` | Recurse into sitemap index children |
| `maxDepth` | integer | `3` | Maximum nesting depth for indexes (1-10) |
| `maxUrls` | integer | `10000` | Maximum URLs to extract (0 = no limit, max 100000) |
| `extractMetadata` | boolean | `true` | Extract lastmod, changefreq, priority |
| `checkRobotsTxt` | boolean | `true` | Check robots.txt when given a domain |
| `timeoutSecs` | integer | `60` | HTTP timeout per request |

### Output

One row per extracted URL:

```json
{
  "url": "https://apify.com/store",
  "sourceSitemap": "https://apify.com/sitemap/pages.xml",
  "lastModified": "2026-07-15",
  "changeFrequency": "daily",
  "priority": 0.8,
  "depth": 0,
  "error": null,
  "errorType": null,
  "scraped_at": "2026-07-31T12:00:00.000Z"
}
```

Error items (for invalid input, not-found, network errors) carry a non-null `error` and `errorType`:

```json
{
  "url": "",
  "sourceSitemap": "https://example.invalid/sitemap.xml",
  "lastModified": null,
  "changeFrequency": null,
  "priority": null,
  "depth": 0,
  "error": "Sitemap not found (HTTP 404)",
  "errorType": "not_found",
  "scraped_at": "2026-07-31T12:00:00.000Z"
}
```

### Pricing

Pay per event, and **the platform usage is on us** — the price you see is the price you pay, with no compute bill on top.

| Event | Price | What one charge buys |
|---|---|---|
| Sitemap URL extracted | **$0.001** | Charged once per extracted URL. Error items (invalid sitemaps, network failures, empty results) are never charged. |
| Actor Start | **$0.002** | One run, whatever it returns. |

Higher Apify subscription tiers pay less on every event (Silver −20%, Gold −35%).

### Use cases

- **SEO analysis** — extract all URLs from a site's sitemap for bulk analysis
- **Site migration** — get a complete URL inventory before restructuring
- **Sitemap validation** — verify all URLs in a sitemap are accessible
- **Agent workflows** — feed structured URL lists into downstream actors

### FAQ

**Does it follow sitemap indexes?** Yes — when `followIndexes` is true, the actor recursively follows `<sitemapindex>` child references up to `maxDepth` levels.

**Can I extract URLs from just a domain name?** Yes — the actor will check `robots.txt` for `Sitemap:` directives and fall back to `/sitemap.xml`.

**What about gzip-encoded sitemaps?** Fully supported — the actor transparently decompresses gzip content.

**Are error items charged?** No — invalid inputs, not-found sitemaps, and network errors are delivered free.

***

*The Sitemap URL Extractor is an independent tool for the Apify platform. Sitemaps are public XML files; this actor parses them without authentication or personal data.*

# Actor input Schema

## `url` (type: `string`):

Full sitemap URL (e.g. https://apify.com/sitemap.xml) or a domain name (e.g. apify.com) to auto-discover the sitemap via robots.txt and convention.

## `followIndexes` (type: `boolean`):

When the sitemap is a sitemap index, recursively follow child sitemaps to extract all URLs. Disable to only extract the index entries themselves.

## `maxDepth` (type: `integer`):

Maximum levels of sitemap index nesting to follow. 0 = direct sitemap only.

## `maxUrls` (type: `integer`):

Maximum number of URLs to extract. Set to 0 for no limit. Capped at 100,000.

## `extractMetadata` (type: `boolean`):

Extract lastmod, changefreq, and priority per URL when available.

## `checkRobotsTxt` (type: `boolean`):

When the input is a domain, try to discover the sitemap from robots.txt before falling back to /sitemap.xml.

## `timeoutSecs` (type: `integer`):

Maximum time per HTTP request.

## `proxyConfiguration` (type: `object`):

Proxy to use for outgoing requests. Sitemaps are datacenter-open and rarely need proxy rotation. Disabled by default for speed.

## Actor input object example

```json
{
  "url": "https://apify.com/sitemap.xml",
  "followIndexes": true,
  "maxDepth": 3,
  "maxUrls": 500,
  "extractMetadata": true,
  "checkRobotsTxt": true,
  "timeoutSecs": 60,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://apify.com/sitemap.xml",
    "maxUrls": 500
};

// Run the Actor and wait for it to finish
const run = await client.actor("leorochasantos/sitemap-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "https://apify.com/sitemap.xml",
    "maxUrls": 500,
}

# Run the Actor and wait for it to finish
run = client.actor("leorochasantos/sitemap-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://apify.com/sitemap.xml",
  "maxUrls": 500
}' |
apify call leorochasantos/sitemap-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,leorochasantos/sitemap-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/AbjRcpzpPdRNJZoUm/builds/zuHF9YOgOzwejUHiO/openapi.json