# Web Page Fetch & Extract (`cyberwarex/web-page-fetch-extract`) Actor

Fetch any live web page as clean LLM-ready markdown/text/HTML, extract elements by CSS selector, or capture a screenshot — JavaScript fully rendered.

- **URL**: https://apify.com/cyberwarex/web-page-fetch-extract.md
- **Developed by:** [Beatrice Chalatian](https://apify.com/cyberwarex) (community)
- **Categories:** Developer tools, AI
- **Features:** Limited permissions
- **Stats:** 8 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$6.00 / 1,000 page fetcheds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Page Fetch & Extract

Turn any live web page into **clean, LLM-ready data** — JavaScript fully rendered in a real browser, nav/ads/boilerplate stripped. Paste one or more URLs and get back page content as **Markdown, plain text, or HTML**, **elements matched by a CSS selector**, or a **full-page screenshot**. No scraping stack, no headless-browser plumbing, no proxy rotation to manage.

### What it does

For each URL you pass, pick one operation:

- **Fetch** — the fully-rendered page as clean **Markdown** (great for RAG and LLM prompts), **plain text**, or **cleaned HTML**. Handles SPAs and JS-heavy sites that a plain HTTP GET returns empty or garbled.
- **Extract** — pass a **CSS selector** (`h1`, `.price`, `table tr`) and get back just the matching elements' text and attributes — perfect for prices, links, table rows, and headings.
- **Screenshot** — a pixel-accurate **PNG** (viewport or full-page, base64) rendered in a real browser, for visual verification or as input to a vision model.

Results land in the dataset, one row per URL — export as JSON, CSV, or Excel, or pull via the API.

### Input

| Field | Type | Description |
|---|---|---|
| `urls` | array | One or more absolute `http(s)` URLs. **Required.** |
| `operation` | string | `fetch` (default), `extract`, or `screenshot`. |
| `format` | string | `markdown` (default), `text`, or `html`. Applies to `fetch`. |
| `selector` | string | CSS selector. **Required when** `operation = extract`. |
| `fullpage` | boolean | Full-page vs viewport. Applies to `screenshot`. Default `false`. |
| `width` | integer | Viewport width in px (320–4096). Applies to `screenshot`. Default `1280`. |

#### Example input

```json
{
  "urls": ["https://example.com"],
  "operation": "fetch",
  "format": "markdown"
}
```

### Example output (one dataset row)

```json
{
  "input_url": "https://example.com",
  "operation": "fetch",
  "http_status": 200,
  "format": "markdown",
  "content": "# Example Domain\n\nThis domain is for use in documentation examples..."
}
```

### Use cases

- Feed clean page content into **RAG pipelines** and LLM summarizers
- **Price / listing / table monitoring** with a targeted CSS selector
- **Visual verification** and layout capture via screenshots
- Convert JS-rendered pages to Markdown for **content archives** and drafts

### Pricing

Pay-per-result: you're charged once per URL successfully processed. No monthly minimum, no idle cost.

### Notes

- URLs are validated and internal/private/file targets are blocked — the backend only fetches public web pages.
- A URL that fails returns a row with `http_status` and an `error` field instead of content, so one bad URL never fails the run.

### Who runs this

Built and operated by **[CyberWareX](https://cyberwarex.com)** — a suite of pay-per-call data and AI
APIs for autonomous agents. Every result row carries `provider` / `provider_url` fields so the data's
origin stays traceable wherever it travels. Full service directory and machine-readable docs:
[cyberwarex.com](https://cyberwarex.com) · daily correctness proofs: [cyberwarex.com/trust](https://cyberwarex.com/trust).

# Actor input Schema

## `urls` (type: `array`):

One or more absolute http(s) URLs to process. Each URL becomes one dataset row.

## `operation` (type: `string`):

fetch = clean page content; extract = elements matching a CSS selector; screenshot = full PNG (base64).

## `format` (type: `string`):

Output format when operation = fetch. markdown (LLM-ready), text (plain), or html (cleaned).

## `selector` (type: `string`):

Required when operation = extract. A CSS selector, e.g. h1, .price, table tr.

## `fullpage` (type: `boolean`):

Capture the entire scrollable page instead of just the viewport. Applies to operation = screenshot.

## `width` (type: `integer`):

Browser viewport width for screenshots (320–4096).

## Actor input object example

```json
{
  "urls": [
    "https://example.com"
  ],
  "operation": "fetch",
  "format": "markdown",
  "fullpage": false,
  "width": 1280
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset items with page content per input URL.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("cyberwarex/web-page-fetch-extract").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://example.com"] }

# Run the Actor and wait for it to finish
run = client.actor("cyberwarex/web-page-fetch-extract").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com"
  ]
}' |
apify call cyberwarex/web-page-fetch-extract --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cyberwarex/web-page-fetch-extract"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hGpbOPr71pHnqPLLY/builds/UAqmWRAr4BTIaGoNi/openapi.json