# arXiv Papers Scraper (`maximedupre/arxiv-papers-scraper`) Actor

Search public arXiv paper records by query, category, author, title, abstract, date range, or advanced syntax. Export paper metadata, URLs, DOI values, and monitoring-ready results.

- **URL**: https://apify.com/maximedupre/arxiv-papers-scraper.md
- **Developed by:** [Maxime Dupré](https://apify.com/maximedupre) (community)
- **Categories:** Education, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.80 / 1,000 paper results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### 🔎 arXiv paper search for research feeds

arXiv Papers Scraper searches public arXiv paper records and saves one dataset row per matching paper. Use it to collect paper metadata for literature monitoring, category feeds, author tracking, date-limited searches, or advanced arXiv queries without user credentials or an API key.

- Run an [arXiv papers scraper](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-papers-scraper) to collect matching research records for a dataset or dashboard.
- Use an [arXiv paper monitor](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-paper-monitor) to watch recurring searches and return newly delivered papers.
- Build an [arXiv category feed](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-category-feed) for selected category codes such as `cs.AI` or `stat.ML`.
- Run [arXiv author search](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-author-search) workflows to track papers from specific researchers.
- Create an [arXiv metadata export](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-metadata-export) with IDs, titles, authors, categories, dates, DOI values, and URLs.
- Use [advanced arXiv search](https://apify.com/maximedupre/arxiv-papers-scraper/examples/advanced-arxiv-search) when simple keywords are not precise enough for your query.
- Collect [arXiv PDF URLs](https://apify.com/maximedupre/arxiv-papers-scraper/examples/arxiv-pdf-urls) for direct source links without downloading or hosting files.

#### 📦 What data you get

Each saved result represents one arXiv paper record. Depending on source metadata and your input settings, rows can include:

| Field | Description |
| --- | --- |
| `arxivId` | Source-native arXiv identifier. |
| `title` | Paper title as published by arXiv. |
| `authors` and `authorCount` | Authors in source order and the number of listed authors. |
| `abstract` | Paper abstract when requested and available. |
| `primaryCategory` and `categories` | Primary arXiv category and all listed categories, including cross-listings. |
| `isCrossListed` | Whether the paper appears in more than one arXiv category. |
| `submittedDate` and `updatedDate` | Initial submission date and most recent update date. |
| `abstractUrl` and `pdfUrl` | Canonical arXiv abstract page URL and direct source PDF URL. |
| `doi`, `comment`, `journalRef` | Optional DOI, author comment, and journal reference when arXiv provides them. |
| `abstractReadMinutes` | Estimated abstract reading time when the abstract is included. |
| `metadataCompleteness` | Ratio from 0 to 1 for useful metadata completeness in the emitted row. |
| `extractionTimestamp` | ISO timestamp when the Actor extracted the record. |

#### ▶️ How to run it

1. Enter a free-text search query, choose arXiv categories, add author, title, or abstract filters, or use the advanced arXiv query field.
2. Optionally filter by submitted-date range and choose sorting by relevance, newest submitted papers, or recently updated papers.
3. Set the maximum number of papers and decide whether to include abstracts.
4. Turn on **Only new papers** for scheduled monitoring when you want recurring runs to focus on newly delivered papers.
5. Start the Actor and export the dataset from Apify in your preferred format.

#### ⚙️ Input options

| Input | What it does |
| --- | --- |
| Search query | Searches public arXiv records by general free text. |
| arXiv categories | Limits results to selected arXiv category codes. |
| Advanced arXiv query | Uses arXiv advanced query syntax for precise searches. |
| Author filter | Filters by paper authors. |
| Title filter | Filters by terms in the paper title. |
| Abstract filter | Filters by terms in the abstract. |
| Submitted from / Submitted to | Filters by submitted-date range. |
| Sort results by | Orders by relevance, newest submitted papers, or recently updated papers. |
| Include abstracts | Adds abstracts and abstract read-minute estimates when available. |
| Only new papers | Returns papers not delivered by earlier runs with the same input. |
| Maximum papers | Caps the number of paper records returned. |

#### 🧾 Output example

```json
{
  "arxivId": "2401.12345",
  "title": "Example Paper Title",
  "authors": ["Jane Doe", "John Smith"],
  "authorCount": 2,
  "abstract": "This paper studies an example research problem.",
  "primaryCategory": "cs.LG",
  "categories": ["cs.LG", "stat.ML"],
  "isCrossListed": true,
  "submittedDate": "2024-01-23T00:00:00.000Z",
  "updatedDate": "2024-02-01T00:00:00.000Z",
  "abstractUrl": "https://arxiv.org/abs/2401.12345",
  "pdfUrl": "https://arxiv.org/pdf/2401.12345",
  "doi": "10.0000/example.doi",
  "comment": "12 pages, 3 figures",
  "journalRef": "Example Journal 1 (2024)",
  "abstractReadMinutes": 1,
  "metadataCompleteness": 1,
  "extractionTimestamp": "2026-07-08T12:00:00.000Z"
}
```

Optional fields such as `abstract`, `doi`, `comment`, and `journalRef` appear only when requested or available from arXiv.

#### 💳 Pricing

This actor uses pay-per-event pricing. You are charged only when the Actor saves an accepted arXiv paper record to the dataset. Empty runs, setup checks, failed runs, and searches with no matching papers are not charged as paper results.

#### 🔌 Integrations

https://www.youtube.com/watch?v=bNACk1_S_6w\&list=PLObrtcm1Kw6MUrlLNDbK9QRg8VDJg0gOW\&index=4

Use this actor with Apify integrations, API clients, webhooks, scheduled runs, and dataset exports. Send results to spreadsheets, BI tools, databases, or your own app after each run.

#### ❓ FAQ

##### 🔁 Can I use this as an arXiv paper monitor?

Yes. Use categories, queries, authors, title or abstract filters, or date ranges with scheduled Apify runs. The **Only new papers** option is designed for recurring monitoring so feeds can focus on newly delivered papers.

##### 🔍 Does it support advanced arXiv search syntax?

Yes. You can use the advanced arXiv query input when simple search terms, category filters, author filters, title terms, or abstract terms are not specific enough.

##### 📄 Does it download PDF files?

No. The Actor returns the direct source `pdfUrl` from arXiv. It does not download, mirror, host, or transform PDF files.

##### 🧠 Does it summarize or rank papers?

No. It returns source metadata and derived mechanical fields such as author count, cross-listing status, abstract read minutes, and metadata completeness. It does not create LLM summaries, recommendations, rankings, or quality judgments.

##### 🔐 Do I need an arXiv account or API key?

No. The Actor searches public arXiv paper records without requiring user credentials or an API key.

##### 📭 What happens if no papers match my query?

A no-result search is treated as a successful empty run. The pricing model charges only for accepted paper records saved to the dataset.

##### 🧩 Is this an alternative to the arXiv API?

Use it when you want arXiv paper search and monitoring inside Apify with scheduling, webhooks, API access, and dataset exports. If you only need to call arXiv directly from your own code, the source API may be enough.

### 📝 Changelog

- 0.1.0: Initial release.

### 🆘 Support

For issues, questions, or feature requests, [file a ticket](https://console.apify.com/actors/maximedupre~arxiv-papers-scraper/issues) and I'll fix or implement it in less than 24h 🫡

### 🔗 Other actors

- [URL to BibTeX Converter ↗](https://apify.com/maximedupre/url-to-bibtex-converter) - Use it for related arxiv and papers workflows.
- [ORCID Researcher Profile Search ↗](https://apify.com/maximedupre/orcid-researcher-profile-search) - Use it for related syntax and doi workflows.
- [World Bank Projects Scraper ↗](https://apify.com/maximedupre/world-bank-projects-scraper) - Use it for related abstract and education workflows.
- [CMS Open Payments Scraper ↗](https://apify.com/maximedupre/cms-open-payments-scraper) - Use it for related education workflows.
- [GitHub Security Advisories Scraper ↗](https://apify.com/maximedupre/github-security-advisories-scraper) - Use it for related feeds workflows.
  **Made with ❤️ by Maxime Dupré**

# Actor input Schema

## `searchQuery` (type: `string`):

Search arXiv paper records by general free text. Leave empty when using only field filters, categories, dates, or advanced query syntax.

## `categories` (type: `array`):

Limit results to one or more arXiv category codes. Use source category IDs such as cs.AI, stat.ML, math.NT, or physics.optics.

## `advancedQuery` (type: `string`):

Use arXiv query syntax for precise searches. Combine with the other filters only when you want a narrower result set.

## `author` (type: `string`):

Return papers matching this author name. Use when author is a required part of the search.

## `title` (type: `string`):

Return papers with matching title text. Use specific terms to keep results focused.

## `abstract` (type: `string`):

Return papers with matching abstract text. Useful for concepts that may not appear in the title.

## `submittedFrom` (type: `string`):

Include papers submitted on or after this date.

## `submittedTo` (type: `string`):

Include papers submitted on or before this date.

## `sortBy` (type: `string`):

Choose how arXiv matches are ordered before the result limit is applied.

## `includeAbstract` (type: `boolean`):

Save each paper abstract in the output. Turn off for smaller records when abstracts are not needed.

## `onlyNewPapers` (type: `boolean`):

For recurring runs, return papers not delivered by earlier runs with the same input. Leave off to collect the current matching set each run.

## `maxResults` (type: `integer`):

Maximum number of paper records to return for this run. Use a lower value for broad searches and monitoring feeds.

## Actor input object example

```json
{
  "searchQuery": "graph neural networks",
  "sortBy": "relevance",
  "includeAbstract": true,
  "onlyNewPapers": false,
  "maxResults": 100
}
```

# Actor output Schema

## `results` (type: `string`):

View arXiv paper metadata rows in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "graph neural networks"
};

// Run the Actor and wait for it to finish
const run = await client.actor("maximedupre/arxiv-papers-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQuery": "graph neural networks" }

# Run the Actor and wait for it to finish
run = client.actor("maximedupre/arxiv-papers-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "graph neural networks"
}' |
apify call maximedupre/arxiv-papers-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,maximedupre/arxiv-papers-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/34B63PBJoBqO67ZaE/builds/eZGdrziReuhXHWwR9/openapi.json