# arXiv Scraper | 12 Fields, PDF Links, No API Key (`themineworks/arxiv-preprint-search`) Actor

Scrape arXiv research papers by keyword, category (cs.AI, cs.LG, quant-ph) or author. Returns titles, abstracts, authors, dates, DOIs & PDF links as clean JSON. No API key. Use it as an MCP server in Claude, ChatGPT & AI agents for research monitoring.

- **URL**: https://apify.com/themineworks/arxiv-preprint-search.md
- **Developed by:** [The Mine Works](https://apify.com/themineworks) (community)
- **Categories:** Education, AI, MCP servers
- **Features:** Limited permissions
- **Stats:** 5 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.60 / 1,000 paper scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## arXiv Scraper: 12 Fields, PDF Links, No API Key

Pay only for results delivered. [Browse all Actors](https://apify.com/themineworks?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral).

💰 From $0.60 / 1,000 results.

Guide and FAQs: [arXiv Scraper on themineworks.com](https://themineworks.com/actors/arxiv-preprint-search/). Tutorial: [arXiv Scraper: Search Preprints With PDF Links via API](https://themineworks.com/blog/arxiv-preprint-scraper-ai-research-api-python/).

Search arXiv preprints and get **12 fields per paper**: arXiv ID, title, full abstract, authors, all categories, primary category, published and updated dates, DOI where one is registered, a **direct PDF link**, the abstract-page URL and a capture timestamp. Filter by keyword, arXiv category and a date floor, and sort by submission date, relevance or last update.

Pure HTTP against the official arXiv API. **No API key, no login, no browser**, up to 2,000 papers per run.

### Why use this arXiv scraper

- **PDF link on every row.** `pdf_url` is constructed for each paper, so a download or full-text pipeline needs no second lookup.
- **All categories, not just the primary one.** `categories[]` carries every cross-list, which is how you catch a paper that is primarily `cs.LG` but also `stat.ML`.
- **Three sort orders.** `submittedDate` for a new-paper feed, `relevance` for a topic search, `lastUpdatedDate` for revisions.
- **A date floor that stops the run early.** With `sortBy: "submittedDate"` and a `dateFrom`, the Actor stops as soon as it pages past your cutoff instead of walking the whole result set.
- **Paced for arXiv.** Requests are spaced just over three seconds apart, which is arXiv's own recommended rate and what keeps a large run from being throttled.

### Monitor new papers in an arXiv category

```json
{
  "query": "large language models",
  "category": "cs.CL",
  "sortBy": "submittedDate",
  "dateFrom": "2026-07-01",
  "maxResults": 200
}
```

Newest first with a date floor is the standard research-monitoring shape. Save it as a Task and schedule it.

### Narrow a broad topic to one arXiv category

```json
{
  "query": "reinforcement learning",
  "category": "cs.AI",
  "sortBy": "submittedDate",
  "maxResults": 500
}
```

Query and category are combined with `AND`, so this returns reinforcement-learning papers whose classification includes `cs.AI`. A query term is always required, leave it out and the Actor falls back to `machine learning`.

### Find the most relevant papers on a topic

```json
{
  "query": "retrieval augmented generation evaluation",
  "sortBy": "relevance",
  "maxResults": 300
}
```

Relevance sort is the right choice for a literature scan; submission-date sort is for a feed.

### Track revisions to existing work

```json
{
  "query": "diffusion transformer",
  "sortBy": "lastUpdatedDate",
  "maxResults": 200
}
```

`published_date` and `updated_date` are separate fields, so you can see exactly which papers were revised and when.

### What data does the arXiv scraper return

```json
{
  "arxiv_id": "2607.01234v1",
  "title": "Scaling Retrieval-Augmented Generation to Long Documents",
  "abstract": "We study retrieval-augmented generation in settings where the evidence...",
  "authors": ["Jane Doe", "John Smith", "Alice Chen"],
  "categories": ["cs.CL", "cs.AI", "cs.IR"],
  "primary_category": "cs.CL",
  "published_date": "2026-07-02T14:21:03Z",
  "updated_date": "2026-07-09T08:15:44Z",
  "doi": null,
  "pdf_url": "https://arxiv.org/pdf/2607.01234v1",
  "url": "https://arxiv.org/abs/2607.01234v1",
  "scraped_at": "2026-08-01T21:00:00.000Z"
}
```

| Field | Description |
| --- | --- |
| `arxiv_id` | arXiv identifier including version suffix |
| `title`, `abstract` | Whitespace-normalised title and abstract |
| `authors[]` | Author names in submission order |
| `categories[]`, `primary_category` | All arXiv categories and the primary one |
| `published_date`, `updated_date` | First submission and latest revision timestamps |
| `doi` | Journal DOI where the authors registered one |
| `pdf_url`, `url` | Direct PDF and abstract-page links |
| `scraped_at` | Capture timestamp |

Most preprints have no `doi`, it is only present once the work is published in a venue and the authors add it. That field comes back empty rather than fabricated.

### How the scraper works without an API key

The arXiv API is a public Atom feed. This Actor combines your query and category into an arXiv `search_query`, requests 100 entries per page, parses the Atom XML, and pauses 3.1 seconds between pages in line with arXiv's rate guidance. With a `dateFrom` and submission-date sort, it stops paging the moment it crosses your cutoff.

### Common arXiv categories

| Category | Field |
| --- | --- |
| `cs.AI` | Artificial intelligence |
| `cs.CL` | Computation and language (NLP) |
| `cs.LG` | Machine learning |
| `cs.CV` | Computer vision |
| `stat.ML` | Machine learning (statistics) |
| `quant-ph` | Quantum physics |
| `q-bio` | Quantitative biology |
| `econ.EM` | Econometrics |

### What can you build with arXiv data

**Research monitoring.** A daily feed of new preprints in your categories and keywords.

**AI and ML trend analysis.** Publication volume by category and month, or author-collaboration graphs.

**RAG over preprints.** Abstracts plus PDF links for a full-text ingestion pipeline.

**Talent and lab tracking.** Which groups are publishing in a subfield, and how often.

### How much does it cost to scrape arXiv

Pay per paper delivered: **$0.001 on the Apify Free plan, $0.0006 on Gold and above**. Searches that match nothing are never charged. Every run also carries a $0.005 start fee.

### How do I use arXiv in Claude or ChatGPT

```
https://mcp.apify.com/?tools=themineworks/arxiv-preprint-search
```

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/arxiv-preprint-search').call({
  query: 'large language models',
  category: 'cs.CL',
  sortBy: 'submittedDate',
  maxResults: 200,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

### arXiv scraper FAQ

**Do I need an API key?** No. The arXiv API is open.

**Can I search a category without a keyword?** No. A query term is always applied, if you leave `query` empty the Actor substitutes `machine learning`. Always supply a real query alongside `category`.

**Why is `doi` usually empty?** Preprints only carry a DOI once the authors register a published version.

**How fast does a large run go?** arXiv asks for roughly one request per three seconds, so a 2,000-paper run takes around a minute of paging. That pacing is deliberate.

**Does `dateFrom` filter server-side?** arXiv has no date parameter in this API, so the Actor applies the floor as it pages and stops once it crosses it. Pair it with `sortBy: "submittedDate"` for the intended behaviour.

### Complete your research intel pipeline

- **[Crossref Scraper](https://apify.com/themineworks/crossref-scholarly-metadata?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral)**, the published record and citation counts.
- **[OpenCitations Scraper](https://apify.com/themineworks/opencitations-citation-graph?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral)**, the citation graph around a DOI.
- **[PubMed Scraper](https://apify.com/themineworks/pubmed-ncbi-scraper?fpr=ymnoit\&utm_source=apify-readme\&utm_medium=referral)**, the biomedical literature with MeSH terms.

Found a bug or want a field added? Open an issue on the Actor's Apify Console page.

### Related guides

- [arXiv Scraper: Search AI, Physics, and Biology Preprints with PDF Links via API](https://themineworks.com/blog/arxiv-preprint-scraper-ai-research-api-python)

*Last verified: 2026-08*

# Actor input Schema

## `query` (type: `string`):

Keywords to search. Use prefixes: ti: title, au: author, abs: abstract. E.g. ti:transformer AND cat:cs.AI

## `category` (type: `string`):

arXiv category to filter by, e.g. cs.AI, cs.LG, cs.CL, quant-ph, q-bio.GN or math.AG. Leave blank for all.

## `sortBy` (type: `string`):

submittedDate (newest first) or relevance.

## `dateFrom` (type: `string`):

Filter papers submitted from this date, in YYYYMMDD format, e.g. 20260101.

## `maxResults` (type: `integer`):

Maximum number of papers to return in this run. Higher values take longer to fetch and cost more, since each paper returned is billed.

## Actor input object example

```json
{
  "query": "large language models",
  "category": "cs.AI",
  "sortBy": "submittedDate",
  "dateFrom": "20260101",
  "maxResults": 25
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "large language models",
    "category": "cs.AI",
    "dateFrom": "20260101",
    "maxResults": 25
};

// Run the Actor and wait for it to finish
const run = await client.actor("themineworks/arxiv-preprint-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "large language models",
    "category": "cs.AI",
    "dateFrom": "20260101",
    "maxResults": 25,
}

# Run the Actor and wait for it to finish
run = client.actor("themineworks/arxiv-preprint-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "large language models",
  "category": "cs.AI",
  "dateFrom": "20260101",
  "maxResults": 25
}' |
apify call themineworks/arxiv-preprint-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,themineworks/arxiv-preprint-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/WVm7sicWk2E5Ncb4R/builds/3RXufu5pIOjngL8M8/openapi.json