Google Search Scraper — SERP, Qualified Leads & AI Overview

Google Search Results Scraper

Scrape Google Search result pages (SERPs) and get clean, structured JSON — organic results, related searches, People Also Ask, knowledge panels, and total result counts — for any query, country, and language.

Built for reliability: automatic retries on every page and parsing that survives Google's layout changes, so a run returns full results pages instead of empty ones.

What it extracts

• Organic results — position, title, URL, displayed URL, description, date, emphasized keywords, sitelinks, and inline product info (rating / reviews / price)
• Paid ads — the text ads above and below the results, with placement (top or bottom), title, the advertiser's landing URL, displayed URL, description and sitelinks
• Related searches — the "people also search for" / bottom-of-page queries
• People Also Ask — the expandable Q&A questions (with answer text when present)
• Knowledge panel — title, subtitle, description, fact attributes (Born, Died, Address, Height, …), and source link
• AI Overview — detection + the cited source domains for GEO/AEO tracking (the generated text itself is not available — see Limitations)
• Total results — the "About N results" count
• Contact emails (opt-in) — turn a SERP into leads: scrape each result's own site for a contact email + people. See Lead enrichment below.
• Perplexity AI answer (opt-in, BYOK) — a grounded AI answer with citations per query, via Perplexity's official API with your key. See Perplexity AI answer.

Coming from another Google search scraper?

If you're switching because another Google SERP scraper let you down, here's how this one handles the things people most often run into:

What tends to go wrong elsewhere	How this scraper handles it
Runs return 0 organic results on some queries, or sit empty for minutes	Automatic retries on every page, and parsing that survives Google layout changes. A page that does not look like a real results page is retried, not returned empty.
site: and other operators return unrelated domains	site:, relatedToSite, intitle:, intext:, inurl:, filetype:, date range and exact match are real input fields, folded into the query correctly — site:github.com returns only that domain.
Setting a results language (lr=) wipes out the organic results	searchLanguage returns a full results page — tested with German (de) returning 10 organic results.
Lead enrichment bills twice for the same contact in one run	Each business domain is scraped and billed once per run. A domain that shows up on several pages or queries reuses its cached contact for free — no second scrape, no second charge, and a page retry can't re-charge it.
AI Overview text comes back truncated or missing	You get AI Overview detection plus the cited source domains reliably (useful for GEO/AEO). Google generates the answer text in the browser after the page loads, so the output says so plainly rather than passing a truncated "Show more" preview off as the answer.

Input

Field	Type	Default	Notes
queries	string[]	—	One or more search terms. Required.
maxPagesPerQuery	integer	1	Pages to fetch per query (~10 results each).
resultsPerPage	integer	10	See the note under Limitations — Google caps this at ~10.
countryCode	string	US	Two-letter country (sets the Google domain, gl and the search location).
languageCode	string	en	Interface/results language (hl).
safeSearch	off/medium/high	off	SafeSearch level.
includeRelated	boolean	true	Extract related searches + People Also Ask.
includeKnowledgePanel	boolean	true	Extract the knowledge panel.
includeAiOverview	boolean	true	Detect AI Overview + extract its cited sources. Needs appendUdm14: false.
includeAds	boolean	true	Extract the text ads at the top and bottom of the page (see Limitations).
appendUdm14	boolean	true	Use Google's clean "Web" layout. Set false to see AI Overviews. Recommended on for max organic stability.
useAdsBotUA	boolean	false	Experimental layout option. Leave off unless you know you need it.
maxConcurrency	integer	5	Parallel requests.
maxRequestRetries	integer	5	Retries per page before giving up.
proxy	object	default	Proxy config. Leave default — see Limitations.

Advanced search filters

Google search operators exposed as structured fields (all folded into the query / URL — you can also type the operators directly inside a queries entry):

Field	Type	Operator / param	Notes
site	string	site:	Restrict to one site. Wins over relatedToSite.
relatedToSite	string	related:	Pages related to a site.
wordsInTitle	string[]	intitle:	Multi-word entries auto-quoted.
wordsInText	string[]	intext:	Words required in body text.
wordsInUrl	string[]	inurl:	Words required in the URL.
fileTypes	string[]	filetype:	OR-combined (pdf, docx, …).
forceExactMatch	boolean	"…"	Wrap the query in quotes.
searchLanguage	string	lr=lang_XX	Results language (distinct from hl).
locationUule	string	uule=	Exact location: a uule code or a place name like Berlin,Berlin,Germany. Defaults to the country.
quickDateRange	string	tbs=qdr:	Relative recency: h, d10, w, m6, y1.
afterDate / beforeDate	string	tbs=cdr:	Absolute bounds (YYYY-MM-DD or MM/DD/YYYY).
includeUnfilteredResults	boolean	filter=0	Include Google's omitted near-duplicates (off by default).
saveHtml	boolean	—	Attach raw SERP HTML to each row under html (large).
saveHtmlToKeyValueStore	boolean	—	Save HTML to the KV store; add htmlSnapshotUrl to each row.

Lead enrichment (opt-in)

Turn search results into leads. When enrichEmails: true, the top business domains in each SERP get their own site scraped for a contact email + people; the hits are attached to those organic results as contactEmail and emailEnrichment.

Field	Type	Default	Notes
enrichEmails	boolean	false	Enable contact-email enrichment.
maxEnrichedDomainsPerPage	integer	3	Cap unique business domains enriched per page.

• Best on business-intent queries ("commercial plumbers chicago", "B2B SaaS vendors").
• Non-business domains (search engines, social, Wikipedia, big marketplaces) are skipped; domains are de-duped per page.
• Best-effort and free (direct site scrape) — enrichment never fails a scrape.

Perplexity AI answer (opt-in, bring-your-own-key)

Attach a Perplexity (Sonar) AI answer to each query — grounded, with citations. This calls Perplexity's official API with your own key, so you are billed by Perplexity per request (it is not scraping). The answer lands on the page-1 row under perplexity { answer, model, citations, searchResults?, relatedQuestions? }.

Field	Type	Default	Notes
perplexitySearch	boolean	false	Enable the Perplexity answer.
perplexityApiKey	string (secret)	—	Your key from docs.perplexity.ai. Required.
perplexitySearchRecency	day/week/month/year	—	Restrict grounding to recent content.
perplexityRelatedQuestions	boolean	false	Also return follow-up questions.

Inert unless both the toggle is on and a key is supplied; a Perplexity error never breaks the SERP scrape.

Example input

{
  "queries": ["apify pricing 2026", "best web scraping tools"],
  "maxPagesPerQuery": 2,
  "countryCode": "US",
  "languageCode": "en"
}

Output

One dataset item per (query, page):

{
  "query": "apify pricing 2026",
  "page": 1,
  "searchUrl": "http://www.google.com/search?q=apify+pricing+2026&...",
  "organicResults": [
    {
      "type": "organic",
      "position": 1,
      "title": "Apify pricing - plans for data collection at any scale",
      "url": "https://apify.com/pricing",
      "displayedUrl": "https://apify.com › pricing",
      "description": "...",
      "siteLinks": []
    }
  ],
  "paidResults": [],
  "relatedQueries": [{ "title": "apify free plan", "url": "https://www.google.com/search?q=..." }],
  "peopleAlsoAsk": [{ "question": "Is Apify free to use?", "answer": "...", "url": "https://..." }],
  "knowledgePanel": null,
  "aiOverview": { "detected": true, "textAvailable": false, "sources": [{ "url": "https://ibm.com/...", "domain": "ibm.com" }] },
  "totalResults": 1230000,
  "selectorUsed": "div.tF2Cxc",
  "scrapedAt": "2026-06-25T09:00:00.000Z"
}

selectorUsed tells you which parser variant matched — handy for spotting future Google layout drift.

How it works

• Automatic retries — a results page that comes back incomplete is retried (up to maxRequestRetries times) before it counts, so a temporary error does not cost you a page.
• Clean layout — by default the scraper uses Google's "Web" results view, which leaves out the AI Overview and gives stable, easy-to-read results (appendUdm14).
• Layout-proof parsing — organic results are read with several fallback patterns (current layouts first, classic layouts as fallback), so a single Google layout change won't zero out a run.
• Complete-page check — a page is only accepted when it looks like a real results page, and a valid results page is never rejected by mistake.

Limitations & notes

• Leave proxy at its default. The default is set up for Google and applied automatically. Changing it can lead to failed pages and empty results, so leave it as it is.

• Paid ads (paidResults) appear when Google shows them. Each entry is one text ad, in page order (position), with placement: "top" (above the organic results) or "bottom" (below them). url is the advertiser's landing page rather than Google's click link: the full landing URL when the ad carries one (Google's click IDs such as gclid removed), otherwise the advertiser's site. Sitelinks get the advertiser URL when the ad carries it, otherwise an absolute google.com click link. The actor never opens an ad link, so scraping does not register clicks the advertiser pays for. Google decides per page load whether to show ads, so the same commercial query can return ads on one run and an empty array on the next. Shopping, hotel and sponsored-places units are not included.

• AI Overview text is not available. Google generates the answer text in the browser after the page loads, so only the AI Overview label and its citation cards are present. aiOverview therefore returns detected + the cited sources (useful for GEO/AEO — which domains Google cites) with textAvailable: false. Requires appendUdm14: false (the default clean layout leaves AI Overviews out entirely).

• resultsPerPage is effectively ~10. Google ignores the num parameter on most modern SERP layouts and returns ~10 results per page. To collect more, raise maxPagesPerQuery rather than resultsPerPage. Pagination always steps by the real page size, so no results are skipped if you set a larger value.

• Knowledge-panel images are lazy-loaded by Google's JS, so imageUrl may be absent even when the panel is otherwise fully populated.

• People Also Ask answers are only present when Google pre-expands the card; for most queries you'll get the question and destination URL but no answer text.

• AI Overview is intentionally excluded by appendUdm14: true. Set it to false if you want the default SERP layout (which can include the AI Overview and more ad/PAA surfaces), at the cost of slightly less stable organic results.

Tips

• Keep appendUdm14 on for the most reliable organic extraction.
• For deep coverage, set maxPagesPerQuery (e.g. 5) instead of resultsPerPage.
• Use countryCode + languageCode together for accurate localized results (e.g. DE + de, JP + ja).
• Google search operators work inside queries (site:, intitle:, "exact", OR, -exclude, filetype:).

🤖 For AI Agents & LLM Apps

Compact reference for AI agents calling this actor via the Apify MCP server or the Apify API (actor: memo23/google-search-scraper).

Purpose: Scrape structured Google SERPs for any query, country and language — organic results, related searches, People Also Ask, knowledge panel, AI-Overview cited sources and total counts — with parsing that survives Google's layout changes.

Minimal input:

{
  "queries": ["best web scraping tools"],
  "maxPagesPerQuery": 1
}

Output: one row per (query, page) — query, page, searchUrl, organicResults {type, position, title, url, displayedUrl, description, siteLinks}, paidResults {type, position, placement, title, url, displayedUrl, description, siteLinks}, relatedQueries {title, url}, peopleAlsoAsk {question, answer, url}, knowledgePanel, aiOverview {detected, textAvailable, sources {url, domain}}, totalResults, selectorUsed, scrapedAt.

Behaviors an agent should know:

• queries is the only required field; one row per query × page. To collect more results, raise maxPagesPerQuery (default 1) — Google caps resultsPerPage at ~10 regardless of the value set.
• Leave proxy at its default; changing it can lead to failed pages.
• appendUdm14 (default true) gives stable organic results but leaves out the AI Overview; set it false (with includeAiOverview) to get the AI-Overview sources — the generated text itself is not available.
• paidResults lists the page's text ads with placement "top" or "bottom"; url is the advertiser's landing page rather than Google's click link, and ad links are never opened. Google shows ads on some page loads and not others, so an empty array is normal even for commercial queries.
• enrichEmails (opt-in, best-effort, free) scrapes top business domains' own sites for contactEmail; perplexitySearch (opt-in, bring-your-own-key) attaches a Perplexity answer and is billed by Perplexity, not this actor.
• Pay-per-event billing — see the Pricing tab on the actor page.

⚠️ Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Google LLC or any of its subsidiaries. All trademarks mentioned, including Google and Google Search, are the property of their respective owners.

The scraper accesses only publicly available Google search results pages — no authenticated endpoints, paid features, or content behind any google.com login wall. Users are responsible for ensuring their use complies with google.com's Terms of Service, applicable data-protection law (GDPR, CCPA, etc.), and any contractual obligations of their own organization.

SEO Keywords

google search scraper, scrape google, google serp scraper, google search API, google.com scraper, Apify google, serp scraper, search results scraper, organic results scraper, seo rank tracking, keyword research data, competitor serp analysis, ai overview extraction, people also ask data, search engine intelligence, serp data export, marketing analytics data