# Website Link Graph & Outbound Links Crawler (`logiover/website-link-graph-crawler`) Actor
Extract all links from a website to CSV/JSON. Maps internal & outbound link graph with anchor text + nofollow/rel flags. No API, no login.
- **URL**: https://apify.com/logiover/website-link-graph-crawler.md
- **Developed by:** [Logiover](https://apify.com/logiover) (community)
- **Categories:** SEO tools, Developer tools, AI
- **Features:** Limited permissions
- **Stats:** 32 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $3.50 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event
## What's an Apify Actor?
An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.
For examples already wired to this Actor's own input schema, see the [API](#api) section below.
Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).
# README
## Website Link Graph & Outbound Links Crawler β Extract Every Link (No Browser) πΈοΈ
    
**Extract every link from a website and map its link graph.** The **Website Link Graph Crawler** is a pure-HTTP **link extractor** that crawls an entire site and exports **every internal and outbound link as a graph edge** β the **source page**, the **target URL**, the **target domain**, the **anchor text**, the **link type** (internal / subdomain / external) and the **rel flags** (`nofollow`, `sponsored`, `ugc`, plus `target="_blank"`). Map a whole site's internal linking and outbound links from a single start URL. **No login, no API key, no headless browser.**
> ### π Why this link extractor?
>
> **1 row per link** with source, target, anchor text, link type & rel flags Β· **thousands of edges per run** Β· pure HTTP (no browser, no JS render) Β· scope control for internal-only or outbound-only Β· export to JSON / CSV / Excel. The fast, high-volume way to **extract all links from a website**, run an **internal-linking SEO audit** and analyze **outbound / external links** at scale.
***
### β¨ What this Actor does / Key features
- π·οΈ **Full-site crawl** β start from one URL and follow internal links across the whole domain.
- π **Every link as an edge** β `sourceUrl β targetUrl` with anchor text, for internal *and* external links.
- π **Target-domain grouping** β each edge carries `targetDomain`, so you can pivot on the external domains you link to most.
- π·οΈ **Rel & attribute parsing** β the raw `rel` plus parsed `isNofollow`, `isSponsored`, `isUgc` boolean flags and `opensNewTab` (`target="_blank"`).
- π― **Scope control** β export **all** links, **only internal**, or **only outbound/external** links via `linkScope`.
- π§ **Link-type classification** β every edge is tagged `internal`, `subdomain` or `external`.
- β‘ **Faster & higher-volume than a link checker** β it maps the graph without HTTP status checks, so it is ideal for big sites.
- 𧡠**Tunable concurrency** β raise `maxConcurrency` for speed or lower it if the target rate-limits you.
- π **Pure HTTP, no browser** β parses server-rendered HTML for speed and low cost; no login, no API key.
### π Quick start (3 steps)
1. **Configure** β paste one or more website URLs into **Start URLs**, set **Max pages to crawl** (`0` for the whole site) and choose **Which links to export** (all / internal / external).
2. **Run** β click **Start**. The crawler follows internal links and records every `` it finds as an edge.
3. **Get your data** β open the **Output** tab and export the link graph to **JSON, CSV, Excel, HTML or XML**, or pull it via the Apify API.
### π₯ Input
Give the Actor at least one **Start URL**. Everything else is optional.
#### Example β map the full link graph of a site
```json
{
"startUrls": [{ "url": "https://example.com" }],
"maxPagesToCrawl": 5000,
"linkScope": "all"
}
```
#### Example β outbound / external links only (backlink-target report)
```json
{
"startUrls": [{ "url": "https://example.com/blog" }],
"maxPagesToCrawl": 2000,
"linkScope": "external",
"maxConcurrency": 10
}
```
#### Example β internal-linking SEO audit
```json
{
"startUrls": [{ "url": "https://example.com" }],
"maxPagesToCrawl": 0,
"linkScope": "internal"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `startUrls` | array | Websites to crawl. The crawler follows internal links within the same domain and records every link (edge) it finds. **Required.** |
| `maxPagesToCrawl` | integer | Maximum pages to crawl per run. Set `0` for no limit (crawl the whole site). Default `50`. |
| `linkScope` | string | `all`, `internal` (internal links only) or `external` (outbound/external links only). Default `all`. |
| `maxConcurrency` | integer | Number of parallel requests. Lower this if the target site rate-limits you. Default `10`. |
> **Tip:** use `linkScope: "external"` for a pure outbound-link/backlink-target report, and `linkScope: "internal"` to analyze internal link distribution and orphan-page risk. Set `maxPagesToCrawl` to `0` for complete coverage of large sites.
#### Additional input fields
These optional fields are also available in the input form and API:
| Field | Type | Default | Description |
|---|---|---|---|
| `crawlScope` | string | `"same-domain"` | How far to follow links when discovering pages to visit. 'Same domain' stays on the seed hostname, 'Include subdomains' also follows links to sub-domains, and 'Single page only' fetches just the given URLs without crawling. |
| `proxyConfiguration` | proxy | `{"useApifyProxy": true}` | Proxy used to fetch pages. Defaults to Apify Proxy (automatic). You usually don't need to change this. |
### π€ Output
One row per link (edge) β exportable to JSON, CSV, Excel, HTML or XML. Here is a trimmed sample record:
```json
{
"sourceUrl": "https://example.com/blog/seo-guide",
"targetUrl": "https://partner.com/tool",
"targetDomain": "partner.com",
"anchorText": "our favorite SEO tool",
"linkType": "external",
"rel": "nofollow sponsored",
"isNofollow": true,
"isSponsored": true,
"isUgc": false,
"opensNewTab": true,
"crawledAt": "2026-05-25T14:17:17.531Z"
}
```
π Full field reference (click to expand)
| Field | Description |
|-------|-------------|
| `sourceUrl` | The page the link is on |
| `targetUrl` | The link target (absolute, resolved URL) |
| `targetDomain` | Hostname of the target |
| `anchorText` | The clickable text of the link |
| `linkType` | `internal`, `subdomain` or `external` |
| `rel` | Raw `rel` attribute value |
| `isNofollow` | `true` if `rel` contains `nofollow` |
| `isSponsored` | `true` if `rel` contains `sponsored` |
| `isUgc` | `true` if `rel` contains `ugc` |
| `opensNewTab` | `true` if the link has `target="_blank"` |
| `crawledAt` | ISO 8601 timestamp of when the page was crawled |
### π‘ Use cases
- **Internal-linking SEO audits** β see which pages link where and with what anchor text to optimize link equity and spot orphan pages.
- **Outbound / external link analysis** β list every external site you link to and which links are `nofollow`, grouped by `targetDomain`.
- **Site-structure mapping** β build a link graph for visualization, analysis or crawl-budget optimization.
- **Link cleanup & compliance** β find sponsored/UGC links that should carry the right `rel` attributes.
- **Anchor-text analysis** β audit the anchor-text distribution across an entire site.
- **Competitor link mapping** β crawl a competitor's site to see who they link out to and how.
### π₯ Who uses it
SEO specialists & technical-SEO teams Β· content and link-building agencies Β· growth marketers Β· web developers & QA Β· data journalists & researchers Β· site owners auditing internal links, outbound links and rel attributes.
### π° Pricing
This Actor runs on a simple **pay-per-result** model β you pay for the link edges you extract, with no separate Apify platform fees to calculate. Try it on the **free tier** first, then scale up to full-site crawls. See the **Pricing** tab on this page for the current rate.
### β Frequently Asked Questions
**Is it legal to crawl a website's links with this Actor?**
The Actor collects only publicly available links from public web pages. You are responsible for using the data in compliance with the target site's terms and applicable laws.
**Does the site I crawl have an API β is this an API alternative?**
You do not need any third-party link-graph API or SEO-suite subscription. This Actor works directly over HTTP against the public HTML of any site, so it is an API alternative for link extraction that returns structured edges without a provider key.
**Do I need an API key, login or browser?**
No. There is no third-party API or login required, and it uses pure HTTP (no headless browser), so it extracts internal and outbound links fast and cheap. You only need an Apify account.
**How much data can I get?**
You can extract thousands of link edges per run. Set `maxPagesToCrawl` to `0` to crawl the whole site, or cap it for a quick sample; each crawled page contributes one edge per `` it contains.
**How do I extract all links from a website to CSV or JSON?**
Paste your start URL, run the crawl, then export the link graph as CSV, JSON, Excel, HTML or via API β one row per link with source page, target URL, target domain, anchor text and rel flags.
**What's the difference vs the Broken Link Checker?**
This Actor maps the link **graph** (anchors, rel, internal/external) **without** checking HTTP status, so it is faster and produces higher volume. Use the **Broken Link Checker** when you specifically need to find dead links and status codes.
**Does it render JavaScript?**
No β it parses server-rendered HTML for speed and low cost. Links present in the delivered HTML are captured.
#### Can I scrape website links without an API or browser?
Yes. There is no third-party API or login required, and it uses pure HTTP (no headless browser), so it extracts internal and outbound links fast and cheap.
#### Is this an internal-linking audit and outbound-link analysis tool?
Yes. Use `linkScope: "internal"` for an internal-linking SEO audit, or `linkScope: "external"` for an outbound/external link analysis report grouped by `targetDomain`.
#### Does it extract anchor text and nofollow links?
Yes. Every edge records its `anchorText` and parses the `rel` attribute into `isNofollow`, `isSponsored` and `isUgc` boolean flags, plus `opensNewTab`.
#### How do I export a list of all outbound links from a website?
Set `linkScope` to `external` and run the crawl; the Actor lists every outbound link with target domain, anchor text and nofollow flags, exported to CSV or JSON.
#### How do I audit a site's internal linking structure?
Set `linkScope` to `internal` to map every internal link as a source-to-target edge with anchor text, so you can spot orphan pages and optimize link equity.
### π More website & lead-gen tools by logiover
Pair the link graph with the rest of the site-intelligence and lead-gen suite:
| Actor | What it does |
|---|---|
| [Broken Link Checker](https://apify.com/logiover/broken-link-checker) | Crawl a site and find dead links with HTTP status codes |
| [Website SEO Audit Crawler](https://apify.com/logiover/website-seo-audit-crawler) | Full on-page SEO audit for every page |
| [Sitemap to URL Crawler](https://apify.com/logiover/sitemap-to-url-crawler) | Extract all URLs from any sitemap.xml |
| [Website to Text & Markdown Crawler](https://apify.com/logiover/website-text-markdown-crawler) | Clean text + Markdown for AI / RAG |
| [Website Contact Scraper](https://apify.com/logiover/website-contact-scraper) | Extract emails, phones & socials from any site |
| [Website Image & Media Extractor](https://apify.com/logiover/website-image-media-extractor) | Pull every image and media asset |
| [Website Tech Stack Detector](https://apify.com/logiover/website-tech-stack-detector) | Identify the technologies a site runs on |
| [JSON-LD Schema & Meta Tag Extractor](https://apify.com/logiover/json-ld-schema-meta-tag-extractor) | Extract structured data and meta tags |
| [Bulk URL Status Checker](https://apify.com/logiover/bulk-url-status-checker) | Check HTTP status for a list of URLs |
| [Bulk Social Profile Extractor](https://apify.com/logiover/bulk-social-profile-extractor) | Find social profiles from a list of sites |
π Browse all **[logiover scrapers on Apify Store](https://apify.com/logiover)** β 180+ actors across real estate, jobs, crypto, social media & B2B data.
### β° Scheduling & integration
Schedule this Actor on Apify to re-map a site's link graph weekly and diff it against prior runs to catch new outbound links or lost internal links. Export results to JSON, CSV or Excel, sync to Google Sheets, or push edges to your database, BI tools and webhooks through the Apify API. Connect it to **Make, n8n or Zapier** to build automated SEO-monitoring pipelines.
### π€ AI Agents, MCP & RAG Integration
**Website Link Graph & Outbound Links Crawler** works natively as an on-demand data tool for AI assistants and autonomous workflows.
Connect this Actor directly to **Apify AI**, **Claude Desktop**, **Cursor**, **VS Code**, **Windsurf**, **LangChain**, or **LlamaIndex** via the official **Apify MCP Server** β no custom wrapper code needed:
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com?tools=logiover/website-link-graph-crawler"
}
}
}
```
- **Schema-typed inputs & outputs:** Clean JSON records with explicit `.actor/input_schema.json` and `.actor/output_schema.json` definitions so LLM agents infer parameters and parse fields accurately without HTML noise.
- **Bounded & predictable runs:** Built-in volume caps and pay-per-result budget guards prevent runaway agent tool loops.
- **RAG & automation ready:** Stream run datasets straight into vector stores, n8n, Make, Zapier, or webhook pipelines.
### π¬ Support, Issues & Reviews
- **Bug reports & target-site changes:** Hit an unexpected empty result or selector change? [Open a ticket in the Issues tab](https://apify.com/logiover/website-link-graph-crawler/issues) with your **Run ID** or sample input β we actively monitor issues and ship fixes fast.
- **Feature & field requests:** Need an additional data attribute or filter? [Request it in the Issues tab](https://apify.com/logiover/website-link-graph-crawler/issues).
- **Enjoying this Actor?** If it saved you engineering time, please consider leaving an honest [review on Apify Store](https://apify.com/logiover/website-link-graph-crawler/reviews) or bookmarking [logiover/website-link-graph-crawler](https://apify.com/logiover/website-link-graph-crawler) β organic developer feedback helps others discover reliable tools and keeps maintenance prioritized.
### βοΈ Legal
This Actor extracts only publicly available link data and is intended for legitimate research, SEO and analytics use. You are responsible for complying with the target site's terms of service and any applicable local laws.
***
This Actor is an independent tool. It is not affiliated with, endorsed by, or sponsored by Website Link Graph & Outbound Links; Website Link Graph & Outbound Links and all other trademarks mentioned here are the property of their respective owners.
### Website Link Graph & Outbound Links scraping β frequently asked questions
#### Do I need a Website Link Graph & Outbound Links API key, account or login?
No. This Actor reads publicly available pages and feeds, so there is nothing to register for and no credentials to manage.
#### How do I export Website Link Graph & Outbound Links data to CSV, Excel, JSON or Google Sheets?
Open the **Storage** tab after a run and pick the format you want β CSV, Excel, JSON, XML or an HTML table. For Google Sheets, Airtable or a database, use the dataset URL or the Apify API so the sheet refreshes itself on every run.
#### Can I schedule this Website Link Graph & Outbound Links scraper to run daily or hourly?
Yes. Use the **Schedules** tab and pick a cadence. Daily is enough for most tracking work; pick hourly only if the rows you care about really do change that fast.
#### How many rows can I get, and what does it cost?
See the volume section above for a measured figure. Billing is per result, so you pay for the rows you actually receive and a run that finds nothing costs nothing.
#### Is scraping Website Link Graph & Outbound Links legal?
This Actor collects only information that Website Link Graph & Outbound Links publishes publicly. It does not log in, does not defeat access controls and does not touch private data. You remain responsible for how you use the output, including under GDPR and similar rules where personal data is involved.
#### What happens if Website Link Graph & Outbound Links blocks the run or changes its layout?
A refused exit IP is swapped for a fresh one within a quarter of a second and retried, and rate limiting is waited out rather than fought. If nothing gets through, the run **fails with the reason** instead of reporting success with zero rows β so an empty result always means "no matches", never "the source would not answer".
#### Can I use this with an AI agent, LangChain or an MCP client?
Yes. The Actor is callable over the Apify API and through Apify's MCP server, so an agent can run it and read the dataset back as a tool call.
### π Changelog
#### 2026-11-14
- Verified live high-volume benchmark (2026-11-14): extracted **16,689** structured records in **17.5s** on the production build.
- Runtime & reliability improvements: clean autoscaled-pool shutdown at run time limit.
#### 2026-11-11
- Added reliable terminal run status telemetry and structured Apify AI / MCP Server (`mcp.apify.com`) tool configuration.
- Added direct Issues & Reviews support links and refreshed live maintenance verification.
#### 2026-11-01
- Monthly health check (2026-11-01): a live run on the current build completed successfully and returned data. Input options, output fields and pricing are unchanged.
- README refreshed: 1 link to retired Actors removed; independence disclaimer added; documented 2 additional input fields.
#### 2026-10-15
- A run that reaches its maximum cost per run now stops cleanly with its results saved and a status message, instead of ending as aborted.
- A proxy group your account cannot use no longer stops the run at start-up; it falls back to the default Apify Proxy.
- Runs with a short timeout (API and AI-agent calls) now finish inside it with what they collected, instead of being cut off.
- Memory is managed within the run's own limit, which removes a rare out-of-memory crash on large pages.
#### 2026-09-23
- Fleet-wide quality audit. Verified end to end against live data and re-checked the input schema, the output columns and the run configuration.
- Output verified on a live run: 11 columns returned, 90.7% of cells populated.
- Run reliability reviewed: 100.0% of public runs succeeded in the last 30 days.
- Input schema, output schema and pricing configuration reviewed.
- Noted that 1 column(s) came back empty in this sample (`rel`); these are under review.
#### 2026-09-01
- Fleet-wide health check. Verified against this Actor's real run history: 30-day success rate, output row counts, per-field fill rates, and peak memory against the configured memory limit.
- Reviewed for the failure patterns that have cost this fleet runs β unguarded proxy setup, retry loops that can outlast the run's own time budget, and full-page HTML parsing that can exhaust a small container.
- No change to input, output fields or scraping logic.
#### 2026-08-11
- Maintenance release: refreshed the build and dependencies.
- Re-verified live execution, non-empty structured output and dataset field/type integrity.
- Reviewed reliability (retries, pagination) and output quality as part of a full-fleet QA pass.
#### 2026-08-01
- Completed the August 2026 full health check: verified empty/programmatic default, Console UI default, and two source-informed alternative inputs on Apify.
- Confirmed successful live execution, non-empty structured output, dataset-field/type integrity, and logical sample quality within the 5-minute quality window.
#### 2026-07-06
- β¨ README overhaul: richer output sample with a full field reference, ready-to-run example scenarios, expanded cross-promo links, and a clearer 3-step quick-start.
#### 2026-07-01
- Maintenance pass: re-verified end-to-end on live data and confirmed successful runs within the 5-minute quality window on the default input.
- Sharpened Store metadata (SEO title & description) and expanded the FAQ with high-intent, long-tail questions for easier discovery in Google and Apify Store search.
- Added ready-to-run example tasks that cover common real-world use cases.
#### 2026-06-15
- Reliability pass: re-verified end-to-end on live data with real-world inputs. Routine maintenance build.
#### 2026-06-07
- Docs: added coverage for exporting website links to CSV/JSON, scraping links without an API or browser, and internal-linking / outbound link audits.
#### 2026-06-05
- π‘οΈ Reliability fix: results are no longer dropped by strict output validation β runs now complete cleanly even at high volume (thousands of results).
- β‘ Stability & performance hardening; fresh rebuild.
#### 2026-06-04
- Verified live & refreshed build β reliability/maintenance pass.
***
\*Maintained: last verified 2026-11-14 β checked end-to-end against live run history (reliability, output fields and types, and time/memory budgets).
# Changelog
This Actor's version history is a separate document: https://apify.com/logiover/website-link-graph-crawler/changelog.md
# Actor input Schema
## `startUrls` (type: `array`):
Websites to crawl. The crawler follows internal links and records every link (edge) it finds. Leave empty to crawl the default demo site (iana.org).
## `linkScope` (type: `string`):
Export all links, only internal links, or only external/outbound links.
## `crawlScope` (type: `string`):
How far to follow links when discovering pages to visit. 'Same domain' stays on the seed hostname, 'Include subdomains' also follows links to sub-domains, and 'Single page only' fetches just the given URLs without crawling.
## `maxPagesToCrawl` (type: `integer`):
Maximum pages to crawl per run. Each page can yield many link edges, so the default is kept modest for a fast, cheap run β raise it (or set 0 for no limit) to map a whole site.
## `maxConcurrency` (type: `integer`):
Number of parallel requests. Lower this if the target site rate-limits you.
## `proxyConfiguration` (type: `object`):
Proxy used to fetch pages. Defaults to Apify Proxy (automatic). You usually don't need to change this.
## Actor input object example
```json
{
"startUrls": [
{
"url": "https://www.iana.org/"
}
],
"linkScope": "all",
"crawlScope": "same-domain",
"maxPagesToCrawl": 30,
"maxConcurrency": 10,
"proxyConfiguration": {
"useApifyProxy": true
}
}
```
# Actor output Schema
## `results` (type: `string`):
All records extracted by this run. Open the Dataset tab to browse, filter, and export as CSV, JSON, or Excel.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
{
"url": "https://www.iana.org/"
}
],
"maxPagesToCrawl": 30,
"maxConcurrency": 10
};
// Run the Actor and wait for it to finish
const run = await client.actor("logiover/website-link-graph-crawler").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`πΎ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// π Want to learn more π? Go to β https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": [{ "url": "https://www.iana.org/" }],
"maxPagesToCrawl": 30,
"maxConcurrency": 10,
}
# Run the Actor and wait for it to finish
run = client.actor("logiover/website-link-graph-crawler").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"πΎ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# π Want to learn more π? Go to β https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
{
"url": "https://www.iana.org/"
}
],
"maxPagesToCrawl": 30,
"maxConcurrency": 10
}' |
apify call logiover/website-link-graph-crawler --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,logiover/website-link-graph-crawler"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/neulzwd4OWYcXmdbW/builds/filvXZb0PPG9H3dV5/openapi.json