# Job Board Aggregator (`mighty_monk/job-board-aggregator`) Actor

Aggregate remote and on-site job listings from Remotive, Arbeitnow, and custom job board URLs. Extract title, company, location, salary, remote flag, description, and apply URL.

- **URL**: https://apify.com/mighty_monk/job-board-aggregator.md
- **Developed by:** [Harsh](https://apify.com/mighty_monk) (community)
- **Categories:** Automation, News
- **Features:** Limited permissions
- **Stats:** 6 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 job listings

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Job Board Aggregator

Aggregate job listings from multiple sources into a single structured dataset. Built with **TypeScript**, **Crawlee**, and **Cheerio**.

### Features

- **Remotive API** — fetches remote jobs from `https://remotive.com/api/remote-jobs`
- **Arbeitnow API** — fetches jobs from `https://www.arbeitnow.com/api/job-board-api` with automatic pagination
- **Custom job boards** — optional Cheerio scraping for any HTML job listing page
- **Structured output** — title, company, location, salary, remote flag, description, apply URL
- **Rate limiting** — configurable concurrency and requests-per-minute caps
- **Retries** — failed requests are retried up to 3 times with logging

### Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `includeRemotive` | `boolean` | `true` | Enable Remotive API source |
| `includeArbeitnow` | `boolean` | `true` | Enable Arbeitnow API source |
| `customJobBoardUrls` | `string[]` | `[]` | Optional HTML job board URLs |
| `searchQuery` | `string` | — | Keyword filter for Remotive |
| `category` | `string` | — | Remotive category slug |
| `maxJobs` | `integer` | `500` | Max jobs across all sources |
| `maxRequestsPerCrawl` | `integer` | `200` | Max HTTP requests |
| `maxConcurrency` | `integer` | `2` | Parallel request limit |
| `maxRequestsPerMinute` | `integer` | `30` | Global rate limit |

#### Example input

```json
{
  "includeRemotive": true,
  "includeArbeitnow": true,
  "customJobBoardUrls": [],
  "searchQuery": "typescript",
  "maxJobs": 100,
  "maxConcurrency": 2,
  "maxRequestsPerMinute": 30
}
```

### Output

Each dataset item contains:

| Field | Description |
|-------|-------------|
| `title` | Job title |
| `company` | Company name |
| `location` | Location or region |
| `salary` | Salary text when available |
| `remote` | Whether the job is remote |
| `description` | Plain-text job description |
| `url` | Apply or detail page URL |
| `source` | `remotive`, `arbeitnow`, or `custom` |
| `scrapedAt` | ISO timestamp |

### How it works

1. The Actor builds start requests for each enabled source.
2. **Remotive** and **Arbeitnow** responses are parsed as JSON.
3. **Arbeitnow** pagination follows `links.next` until `maxJobs` is reached.
4. **Custom boards** use Cheerio heuristics for job cards and enqueue next-page links.
5. Results are pushed to the default dataset with per-job logging.

### Local development

```bash
npm install
apify run
```

Edit `storage/key_value_stores/default/INPUT.json` for local test input.

### Deploy

```bash
apify login
apify push
```

Publish to the Apify Store with PPE pricing:

```bash
node ../../scripts/publish-factory.js job-board-aggregator '{"resultPriceUsd":0.002,"resultTitle":"Job listing","resultDescription":"Charged per job listing returned"}'
```

### Notes

- Remotive requests should be kept infrequent (their API advises max ~4 calls/day for production sync).
- Arbeitnow does not expose salary in its public API.
- Custom HTML scraping uses common CSS patterns; highly custom sites may need dedicated selectors.
- Respect each source's terms of service and attribution requirements.

# Actor input Schema

## `includeRemotive` (type: `boolean`):

Fetch jobs from the Remotive public API (remote jobs).

## `includeArbeitnow` (type: `boolean`):

Fetch jobs from the Arbeitnow public API with automatic pagination.

## `customJobBoardUrls` (type: `array`):

Optional HTML job board listing pages to scrape with Cheerio (supports pagination).

## `searchQuery` (type: `string`):

Optional keyword filter passed to the Remotive API.

## `category` (type: `string`):

Optional Remotive category slug (e.g. software-dev, customer-support).

## `maxJobs` (type: `integer`):

Maximum total jobs to save across all sources.

## `maxRequestsPerCrawl` (type: `integer`):

Maximum HTTP requests the crawler may make (pagination + custom boards).

## `maxConcurrency` (type: `integer`):

Maximum parallel requests for rate limiting.

## `maxRequestsPerMinute` (type: `integer`):

Global rate limit to avoid overloading job board APIs.

## `useApifyProxy` (type: `boolean`):

Route requests through Apify Proxy. Disabled by default for public APIs; enable for custom boards behind geo-blocks.

## Actor input object example

```json
{
  "includeRemotive": true,
  "includeArbeitnow": true,
  "customJobBoardUrls": [],
  "maxJobs": 500,
  "maxRequestsPerCrawl": 200,
  "maxConcurrency": 2,
  "maxRequestsPerMinute": 30,
  "useApifyProxy": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "customJobBoardUrls": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("mighty_monk/job-board-aggregator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "customJobBoardUrls": [] }

# Run the Actor and wait for it to finish
run = client.actor("mighty_monk/job-board-aggregator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "customJobBoardUrls": []
}' |
apify call mighty_monk/job-board-aggregator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mighty_monk/job-board-aggregator"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SO1muXSKfdmVBKaTQ/builds/3RFb6HSZwadY9UhJ2/openapi.json