# Y Combinator Scraper (`shahidirfan/y-combinator-scraper`) Actor

Discover the Y Combinator Scraper, an efficient actor for scraping the YC companies directory. Easily extract detailed profiles, founder info, and batch data. Ideal for market research, lead generation, or investment analysis. For best results and to avoid blocks, residential proxies are best.

- **URL**: https://apify.com/shahidirfan/y-combinator-scraper.md
- **Developed by:** [Shahid Irfan](https://apify.com/shahidirfan) (community)
- **Categories:** Lead generation, Developer tools, Other
- **Features:** Limited permissions
- **Stats:** 42 total users, 5 monthly users, 100.0% runs succeeded, 5 bookmarks
- **User rating**: 4.79 out of 5 stars

## Pricing

from $1.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Y Combinator Companies Scraper

Extract detailed company data from the Y Combinator directory - startup profiles, batch info, hiring signals, locations, and industry taxonomy. This scraper collects structured records from YC listings with automatic pagination, making it easy to build datasets for market research, talent sourcing, and cohort analysis.

### Features

- **Complete company profiles** - Collect name, description, batch, status, location, team size, website, and social links
- **Industry taxonomy** - Tags, industries, regions, and market category for every company
- **Hiring and growth signals** - Hiring status, nonprofit flag, top-company marker, and launch date
- **Flexible pagination** - Set a result limit, max pages, or scrape everything available
- **Batch and location filtering** - Pass a YC directory URL with query parameters to target specific cohorts or regions
- **Clean export-ready data** - JSON output with consistent field types for analysis

### Use Cases

#### Startup Market Analysis

Build a comprehensive dataset of YC-funded companies to analyze trends across batches, stages, and industries. Track which sectors are getting funded and compare cohort characteristics over time.

#### Talent Acquisition Intelligence

Identify companies that are actively hiring, review their team size and stage, and prioritize outreach targets based on growth signals and market categories.

#### Portfolio and Ecosystem Research

Map the Y Combinator ecosystem by company location, founding year, and industry. Use the taxonomy fields (tags, industries, regions) to segment companies for investment or partnership research.

#### Batch and Cohort Comparison

Compare company characteristics across YC batches. Filter by batch parameter to examine how startup profiles, funding stages, and hiring patterns have evolved over successive cohorts.

***

### Input Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `url` | String | No | `https://www.ycombinator.com/companies/industry/all` | YC directory URL to scrape. Supports path and query parameters for filtering |
| `scrape_all_companies` | Boolean | No | `false` | Ignore the result limit and collect all available companies (capped by max pages) |
| `results_wanted` | Integer | No | `20` | Maximum number of company records to save |
| `max_pages` | Integer | No | `5` | Safety limit for the number of listing pages to paginate through |
| `proxyConfiguration` | Object | No | Apify Proxy | Proxy settings for reliable extraction. Residential proxy recommended for larger runs |

***

### Output Data

Each dataset item contains the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `company_id` | String | Y Combinator company ID |
| `slug` | String | Company slug used in profile URL |
| `company_name` | String | Company name |
| `url` | String | Absolute URL to the YC profile page |
| `short_description` | String | One-line summary from the listing |
| `one_liner` | String | Tagline from the YC listing |
| `long_description` | String | Full company description |
| `batch` | String | YC batch (e.g., S25, W24) |
| `status` | String | Company status (Active, Acquired, Public, etc.) |
| `tags` | Array | Industry tags |
| `company_location` | String | Primary company location |
| `year_founded` | String | Founding year |
| `founded_raw` | String | Raw founding date text from the YC profile |
| `team_size` | String | Reported team size |
| `market` | String | Market or category tag |
| `website` | String | Company website URL |
| `company_linkedin` | String | LinkedIn profile URL |
| `company_x` | String | X / Twitter profile URL |
| `industries` | Array | Industry taxonomy labels |
| `regions` | Array | Region taxonomy labels |
| `stage` | String | Company stage (Growth, Early, etc.) |
| `top_company` | Boolean | Top-company indicator |
| `is_hiring` | Boolean | Hiring signal |
| `nonprofit` | Boolean | Nonprofit flag |
| `launched_at` | String | Launch timestamp in ISO format |
| `company_image` | String | URL of the company logo |
| `primary_partner` | String | Primary YC partner name |
| `primary_group_partner` | Object | Detailed partner info (name, title) |
| `former_names` | Array | Former company names |
| `founders` | Array | Founder profiles (enabled with detail mode) |
| `open_jobs` | Array | Open job listings (enabled with detail mode) |
| `algolia_raw` | Object | Raw Algolia record for debugging |

***

### Usage Examples

#### Basic Company Collection

Collect 20 companies from the default listing:

```json
{
  "results_wanted": 20,
  "max_pages": 1
}
```

#### Targeted Batch Extraction

Scrape 50 companies from a specific YC batch:

```json
{
  "url": "https://www.ycombinator.com/companies/industry/all?batch=S25",
  "results_wanted": 50,
  "max_pages": 3
}
```

#### Full Directory Export

Scrape all available companies with a generous page limit:

```json
{
  "scrape_all_companies": true,
  "max_pages": 50
}
```

***

### Sample Output

```json
{
  "company_id": "271",
  "slug": "airbnb",
  "company_name": "Airbnb",
  "url": "https://www.ycombinator.com/companies/airbnb",
  "short_description": "Book accommodations around the world.",
  "one_liner": "Book accommodations around the world.",
  "long_description": "Founded in August of 2008 and based in San Francisco...",
  "batch": "Winter 2009",
  "status": "Public",
  "tags": ["Marketplace", "E-commerce"],
  "company_location": "San Francisco, CA, USA",
  "website": "http://airbnb.com",
  "team_size": "6132",
  "market": "Consumer",
  "industries": ["Consumer", "Travel, Leisure and Tourism"],
  "regions": ["United States of America", "America / Canada"],
  "stage": "Growth",
  "top_company": true,
  "is_hiring": false,
  "nonprofit": false,
  "launched_at": "2012-01-17T22:34:16.000Z"
}
```

***

### Tips for Best Results

#### Start With a Small Test

Use `results_wanted: 20` and `max_pages: 1` for initial testing to verify your target URL returns expected data.

#### Use Proxy for Large Runs

Enable Apify Proxy with residential groups when scraping many pages to maintain consistent extraction.

#### Target Specific Batches

Use batch query parameters in the URL (e.g., `?batch=W24`) to narrow results to a specific YC cohort for faster, more focused runs.

***

### Integrations

Connect your scraped data with:

- **Google Sheets** - Export datasets for sorting and filtering
- **Airtable** - Build searchable company databases
- **Make / Zapier** - Trigger automated workflows on new data
- **Webhooks** - Push records into custom systems
- **Slack** - Get notified when new companies are collected

#### Export Formats

- **JSON** - For APIs and programmatic access
- **CSV** - For spreadsheet analysis and BI tools
- **Excel** - For business reporting
- **XML** - For legacy system integrations

***

### Frequently Asked Questions

#### How many companies can I collect?

You can collect all available companies by enabling `scrape_all_companies`. The only limit is how many pages you set with `max_pages`.

#### Can I filter by YC batch?

Yes. Pass a URL like `https://www.ycombinator.com/companies/industry/all?batch=S25` and the actor respects the filter parameters.

#### Why are some fields null?

Some companies do not publish every field. Partner assignments, founding dates, and team sizes may be missing for certain records.

#### Does the actor need an API key?

No. All data is collected from the public Y Combinator directory. Algolia credentials are fetched automatically on each run.

#### Can I extract founder or job data?

Currently founders and open jobs are included as empty arrays in the output. A future update will add optional detail scraping for these fields.

#### How fast is the extraction?

The actor uses API-level data access for listing and enrichment, making it significantly faster than browser-based scrapers. A typical run of 100 companies completes in under a minute.

***

### Support

For bug reports, feature requests, or questions, contact support through the Apify Console.

#### Resources

- [Apify Documentation](https://docs.apify.com/)
- [API Reference](https://docs.apify.com/api/v2)
- [Scheduling Runs](https://docs.apify.com/schedules)

***

### Legal Notice

This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with Y Combinator's terms of service and applicable laws. Use data responsibly and respect rate limits.

# Actor input Schema

## `url` (type: `string`):

Use a YC companies directory URL. Leave empty to use the default all-companies page.

## `scrape_all_companies` (type: `boolean`):

When enabled, the actor keeps scraping until pages are exhausted (or max pages limit is reached).

## `results_wanted` (type: `integer`):

Maximum number of companies to save when scrape all is disabled.

## `max_pages` (type: `integer`):

Safety limit for number of listing pages to read.

## `proxyConfiguration` (type: `object`):

Use Apify Proxy for reliable data extraction.

## Actor input object example

```json
{
  "url": "https://www.ycombinator.com/companies/industry/all",
  "scrape_all_companies": false,
  "results_wanted": 20,
  "max_pages": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://www.ycombinator.com/companies/industry/all",
    "scrape_all_companies": false,
    "results_wanted": 20,
    "max_pages": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("shahidirfan/y-combinator-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "https://www.ycombinator.com/companies/industry/all",
    "scrape_all_companies": False,
    "results_wanted": 20,
    "max_pages": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("shahidirfan/y-combinator-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://www.ycombinator.com/companies/industry/all",
  "scrape_all_companies": false,
  "results_wanted": 20,
  "max_pages": 5
}' |
apify call shahidirfan/y-combinator-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,shahidirfan/y-combinator-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zMtocDToS2uadIbu9/builds/DJKmJQf4ixW6ISPaE/openapi.json