# Beehiiv Scraper - Newsletters, Posts & Content (`scraper_guru/beehiiv-scraper`) Actor

Beehiiv scraper for newsletter posts, authors, publication data, public HTML, Markdown, and text. Filter by date, keyword, access, or reading time.

- **URL**: https://apify.com/scraper_guru/beehiiv-scraper.md
- **Developed by:** [LIAICHI MUSTAPHA](https://apify.com/scraper_guru) (community)
- **Categories:** News, Lead generation, AI
- **Features:** Limited permissions
- **Stats:** 27 total users, 4 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.40 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Beehiiv Scraper - Newsletters, Posts & Content

Extract public **Beehiiv newsletter posts** from publication homepages, custom domains, sitemaps, and direct post URLs. Get authors, publication details, dates, images, full public text, HTML, Markdown, and cited links without a Beehiiv API key.

Use this Beehiiv scraper for newsletter monitoring, competitive intelligence, content research, lead generation, archives, and AI or RAG datasets.

> Premium content is not unlocked. The Actor returns public metadata or preview content when Beehiiv marks a post as paywalled.

### What does Beehiiv Newsletter Scraper do?

The Actor turns one or more Beehiiv-powered sites into structured post records. It supports standard `*.beehiiv.com` publications and Beehiiv sites hosted on custom domains.

- Discover posts through each publication's public sitemap
- Scrape a direct `/p/<slug>` URL without crawling the full archive
- Extract titles, descriptions, authors, publication dates, images, and tags
- Return public article content as plain text, Markdown, and HTML
- Collect unique outbound links cited in each post
- Identify free and premium posts without bypassing access controls
- Filter by keyword, publication date, access level, and reading time
- Process multiple publications concurrently
- Store run totals and failures in a `STATS` record

The Actor uses lightweight HTTP extraction rather than launching a browser for every post. This keeps normal runs fast and reduces memory requirements.

### Use cases

#### Newsletter competitive intelligence

- Monitor competitor publishing cadence and editorial themes.
- Compare authors, headlines, reading time, and cited sources.
- Schedule recurring runs across a publication watchlist.

#### Content and market research

- Search public posts for products, companies, or topics.
- Build a reviewable archive of recent newsletter coverage.
- Export structured data to Sheets, a CRM, or a warehouse.

#### AI and RAG datasets

- Create a public newsletter corpus in clean Markdown or text.
- Preserve canonical URLs and timestamps for provenance.
- Use word counts to estimate embedding or model-processing costs.

#### Sponsorship and lead research

- Identify publications and authors active in a target niche.
- Review public content before sponsorship or partnership outreach.
- Analyze outbound brands and sources mentioned in newsletters.

### Data you can extract

| Category | Fields |
|---|---|
| Post | `post_id`, `slug`, `post_url`, `headline`, `subheading`, `description` |
| Publication | `beehiiv_url`, publication ID, name, domain, logo, social URLs |
| Author | `author_name`, `authors`, `author_urls`, `author_image_urls` |
| Dates and access | `published_at`, `updated_at`, `audience`, `is_paywalled`, `content_type` |
| Content | `article_text`, `article_markdown`, `article_html`, `word_count` |
| Media and taxonomy | `cover_image_url`, `tags` |
| Link intelligence | `outbound_links` |
| Provenance | `scraped_at` |

Optional values are omitted or left empty when the public page does not expose them. The Actor does not invent engagement metrics, subscriber counts, dates, or authors.

### How to scrape Beehiiv newsletters

1. Open **Beehiiv Newsletter Scraper** in Apify Console.
2. Add a publication homepage, custom domain, sitemap, or direct post URL.
3. Keep the prefilled three-post limit for the first run.
4. Optionally set keywords, dates, access, or reading-time filters.
5. Disable HTML when you only need text or Markdown.
6. Click **Start**.
7. Review results in the **Dataset** tab.
8. Export the dataset as JSON, CSV, Excel, XML, or RSS.
9. Create an Apify schedule for recurring monitoring.

Minimal input:

```json
{
  "beehiivUrls": ["https://superhuman.beehiiv.com"],
  "maxPostsPerPublication": 10,
  "maxItems": 10
}
```

Direct post extraction:

```json
{
  "beehiivUrls": [
    "https://www.superhuman.ai/p/robotics-special-unitree-lands-a-blockbuster-ipo"
  ],
  "maxItems": 1,
  "includeMarkdown": true,
  "includeHtml": false
}
```

Topic monitoring:

```json
{
  "beehiivUrls": [
    "https://superhuman.beehiiv.com",
    "https://tldrnewsletter.beehiiv.com"
  ],
  "maxPostsPerPublication": 50,
  "maxItems": 50,
  "publishedAfter": "2026-08-01",
  "keywords": ["OpenAI", "Claude", "agents"],
  "onlyFree": true,
  "includeOutboundLinks": true
}
```

### Input parameters

| Parameter | Type | Default | Description |
|---|---|---:|---|
| `beehiivUrls` | array | required | Publication, custom-domain, sitemap, or direct post URLs. |
| `maxPostsPerPublication` | integer | `50` | Maximum newest sitemap posts per publication; `0` means full sitemap. |
| `maxItems` | integer | `50` | Total matching posts saved; `0` allows up to 10,000. |
| `keywords` | string array | `[]` | Keep posts matching any keyword. |
| `publishedAfter` | date | empty | Inclusive lower publication-date bound. |
| `publishedBefore` | date | empty | Inclusive upper publication-date bound. |
| `onlyFree` | boolean | `false` | Exclude premium or paywalled posts. |
| `minReadingTime` | integer | `0` | Minimum estimated reading time in minutes. |
| `maxReadingTime` | integer | `0` | Maximum reading time; `0` means no limit. |
| `includeContent` | boolean | `true` | Return public article text and selected formats. |
| `includeMarkdown` | boolean | `true` | Include clean Markdown. |
| `includeHtml` | boolean | `true` | Include public article HTML. |
| `includeOutboundLinks` | boolean | `true` | Include external links cited in the article. |
| `postConcurrency` | integer | `10` | Parallel post requests, from 1 to 25. |
| `proxyConfiguration` | object | disabled | Optional proxy retry for protected custom domains. |

The legacy `batchSize`, `startDate`, and `endDate` fields remain accepted for existing integrations.

### Output example

```json
{
  "record_type": "post",
  "beehiiv_url": "https://www.superhuman.ai/",
  "post_url": "https://www.superhuman.ai/p/robotics-special-unitree-lands-a-blockbuster-ipo",
  "post_id": "robotics-special-unitree-lands-a-blockbuster-ipo",
  "slug": "robotics-special-unitree-lands-a-blockbuster-ipo",
  "headline": "Robotics Special: Unitree lands a blockbuster IPO",
  "published_at": "2026-08-22T14:00:00.000Z",
  "audience": "free",
  "is_paywalled": false,
  "estimated_reading_time": 5,
  "word_count": 1110,
  "author_name": "Zain Kahn",
  "authors": ["Zain Kahn"],
  "cover_image_url": "https://media.beehiiv.com/...",
  "publication": {
    "id": "faa6a747-8c1c-43c1-8155-91aa43268f01",
    "name": "Superhuman AI",
    "url": "https://www.superhuman.ai/",
    "domain": "www.superhuman.ai"
  },
  "article_text": "Public article text...",
  "article_markdown": "Public article Markdown...",
  "outbound_links": ["https://example.com/source"],
  "content_type": "full",
  "scraped_at": "2026-08-23T12:00:00+00:00"
}
```

`content_type` can be `full`, `preview_only`, `metadata_only`, or `unavailable`.

### Pricing

This Actor uses pay-per-event pricing, so platform usage is included. You pay for each post saved to the dataset, plus a tiny start charge of `$0.00005` per run.

| Plan | Per 1,000 posts |
|---|---:|
| Free | `$3.00` |
| Bronze | `$2.80` |
| Silver | `$2.60` |
| Gold, Platinum, Diamond | `$2.40` |

Example: 20 posts from each of 5 newsletters (100 posts) costs about `$0.30` on the Free plan. Apify shows the exact plan-adjusted price before each run.

### API examples

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("scraper_guru/beehiiv-scraper").call(run_input={
    "beehiivUrls": ["https://superhuman.beehiiv.com"],
    "maxPostsPerPublication": 10,
    "maxItems": 10,
    "includeHtml": False,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["headline"], item["post_url"])
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('scraper_guru/beehiiv-scraper').call({
  beehiivUrls: ['https://superhuman.beehiiv.com'],
  maxPostsPerPublication: 10,
  maxItems: 10,
  includeHtml: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/scraper_guru~beehiiv-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "beehiivUrls": ["https://superhuman.beehiiv.com"],
    "maxPostsPerPublication": 10,
    "maxItems": 10
  }'
```

Keep API tokens in environment variables or a secret manager.

### Limitations and responsible use

- The Actor extracts publicly accessible Beehiiv pages only.
- It does not log in, subscribe, or bypass paywalls.
- Protected custom domains may require Apify Proxy.
- Site owners can change templates or block automated requests.
- Public metadata varies by publication and post design.
- Respect copyright, privacy, applicable law, and each site's terms.

### FAQ

#### Do I need a Beehiiv API key?

No. The Actor reads public sitemaps and server-rendered post pages.

#### Does it support custom Beehiiv domains?

Yes. Add the publication's public custom domain. If direct requests are rejected, enable the proxy option.

#### Can it scrape a single post?

Yes. Add any public `/p/<slug>` URL. Sitemap discovery is skipped.

#### Can it scrape premium posts?

It records public metadata or preview content when available. It never unlocks protected content.

#### Why does a result have no article body?

The post may be premium, unavailable, removed, or protected by the publication's hosting configuration. Check `content_type` and the run log.

#### How can I monitor new posts?

Create an Apify Task from a working input, schedule it daily or weekly, and use a date window or dataset deduplication downstream.

#### Which export formats are available?

Apify datasets support JSON, CSV, Excel, XML, and RSS exports.

#### Is scraping Beehiiv legal?

Public web scraping rules depend on jurisdiction and use. Respect access controls, copyright, personal data, and applicable terms.

### Related scrapers

Building a newsletter or audience research dataset? These Actors from the same developer pair well with this one:

| Actor | What it collects |
|---|---|
| [Substack Scraper](https://apify.com/scraper_guru/substack-scraper) | Newsletter posts, authors, engagement and comments from any Substack |
| [Reddit Comments Deep Scraper](https://apify.com/scraper_guru/reddit-comments-deep-scraper) | Full Reddit comment threads with nested replies, using your own Reddit API credentials |
| [YouTube Comments Extractor](https://apify.com/scraper_guru/youtube-comments-extractor) | Comments and replies from YouTube videos |

***

Built by [Mustapha Liaichi](https://apify.com/scraper_guru), Automation and Web Scraping Specialist.

# Actor input Schema

## `beehiivUrls` (type: `array`):

One or more public Beehiiv-powered URLs.

## `maxPostsPerPublication` (type: `integer`):

Maximum newest posts discovered per publication. Set to 0 for the full sitemap.

## `maxItems` (type: `integer`):

Maximum matching posts saved across all publications. Set to 0 for up to 10,000 results.

## `keywords` (type: `array`):

Keep posts when any keyword matches the title, description, or public article text.

## `publishedAfter` (type: `string`):

Only include posts published on or after this date.

## `publishedBefore` (type: `string`):

Only include posts published on or before this date.

## `onlyFree` (type: `boolean`):

Exclude posts marked as premium or paywalled.

## `minReadingTime` (type: `integer`):

Only include posts with at least this estimated reading time.

## `maxReadingTime` (type: `integer`):

Only include posts at or below this reading time. Set to 0 for no limit.

## `includeContent` (type: `boolean`):

Return available article text. Disable for faster metadata-only runs.

## `includeMarkdown` (type: `boolean`):

Add clean Markdown for AI, RAG, and content pipelines.

## `includeHtml` (type: `boolean`):

Add the public article HTML to each result.

## `includeOutboundLinks` (type: `boolean`):

Extract unique external links cited in each public article.

## `postConcurrency` (type: `integer`):

Number of post requests processed in parallel.

## `proxyConfiguration` (type: `object`):

Optional proxy retry for custom domains that reject direct requests.

## Actor input object example

```json
{
  "beehiivUrls": [
    "https://superhuman.beehiiv.com"
  ],
  "maxPostsPerPublication": 3,
  "maxItems": 3,
  "keywords": [],
  "onlyFree": false,
  "minReadingTime": 0,
  "maxReadingTime": 0,
  "includeContent": true,
  "includeMarkdown": true,
  "includeHtml": true,
  "includeOutboundLinks": true,
  "postConcurrency": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `posts` (type: `string`):

Structured post records with publication, author, access, and content metadata.

## `contentExport` (type: `string`):

Plain text, Markdown, HTML, and cited links for content pipelines.

## `runStats` (type: `string`):

Sources, discovered posts, saved posts, filters, failures, and duration.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "beehiivUrls": [
        "https://superhuman.beehiiv.com"
    ],
    "maxPostsPerPublication": 3,
    "maxItems": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("scraper_guru/beehiiv-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "beehiivUrls": ["https://superhuman.beehiiv.com"],
    "maxPostsPerPublication": 3,
    "maxItems": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("scraper_guru/beehiiv-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "beehiivUrls": [
    "https://superhuman.beehiiv.com"
  ],
  "maxPostsPerPublication": 3,
  "maxItems": 3
}' |
apify call scraper_guru/beehiiv-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scraper_guru/beehiiv-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/sZyged0DjjTHhW84P/builds/eqaaUIvF4Xlzcp8Y5/openapi.json