> ## Documentation Index
> Fetch the complete documentation index at: https://docs.geekflare.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Extraction

> Ask AI to answer questions, extract structured data, or analyze scraped web pages using Geekflare's Web Scraping API.

Scraping a page is the first step. The next is turning that content into an answer, a structured record, or a summary. **AI Extraction** does that in the same request: pass an `aiPrompt` object alongside your `url`, and Geekflare scrapes the page, feeds the cleaned Markdown to an LLM, and returns the result under `aiResult`.

<Info>
  AI Extraction always runs against Markdown content. Any `format` you set in
  the same request is ignored — you don't need to (and shouldn't) combine
  `format` with `aiPrompt`.
</Info>

**Endpoint:** `POST https://api.geekflare.com/webscraping`

**Credits:** +7 on top of the base scrape cost (8 total).

***

## How It Works

Every `aiPrompt` object has a required `type` field that selects the extraction mode, plus mode-specific fields:

```json theme={null}
{
  "url": "https://example.com/products/wireless-headphones",
  "aiPrompt": {
    "type": "prompt | schema | listing | summary | sentiment | keywords",
    "...": "mode-specific fields below"
  }
}
```

The result is returned under `aiResult` in the response, alongside the usual `data` field (which contains the Markdown content that was analyzed) and `meta`.

***

## 1. `prompt` — Open-Ended Question

Ask any question about the page. The model answers strictly from page content and quotes the supporting sentence(s).

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/products/wireless-headphones',
    aiPrompt: {
      type: 'prompt',
      query: 'What is the return policy?'
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/products/wireless-headphones',
      ai_prompt={'type': 'prompt', 'query': 'What is the return policy?'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/products/wireless-headphones",
      "aiPrompt": {"type": "prompt", "query": "What is the return policy?"}
    }'
  ```
</CodeGroup>

| Field | Type | Required | Description |
| - | - | - | - |
| `query` | string | yes | Your question, max 1000 characters |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "prompt",
      "query": "What is the return policy?",
      "answered": true,
      "answer": "Items can be returned within 30 days of purchase for a full refund, provided they're unused and in original packaging.",
      "sourceExcerpt": "..."
    }
  }
  ```
</Accordion>

If the answer isn't present in the page content, `answered` is `false` and `answer`/`sourceExcerpt` are `null` — the model never guesses.

***

## 2. `schema` — Custom JSON Schema Extraction

Define your own field shape and let AI extract matching data from any page.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/products/wireless-headphones',
    aiPrompt: {
      type: 'schema',
      schema: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          price: { type: 'number' },
          currency: { type: 'string' },
          inStock: { type: 'boolean' }
        }
      }
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/products/wireless-headphones',
      ai_prompt={
          'type': 'schema',
          'schema': {
              'type': 'object',
              'properties': {
                  'title': {'type': 'string'},
                  'price': {'type': 'number'},
              }
          }
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/products/wireless-headphones",
      "aiPrompt": {
        "type": "schema",
        "schema": {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "price": {"type": "number"}
          }
        }
      }
    }'
  ```
</CodeGroup>

| Field | Type | Required | Description |
| - | - | - | - |
| `schema` | object | yes | JSON Schema-like object describing the fields to extract, max 4000 bytes |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "schema",
      "data": {
        "title": "AudioTech Wireless Noise-Cancelling Headphones",
        "price": 199.99,
        "currency": "USD",
        "answered": true,
        "inStock": true
      }
    }
  }
  ```
</Accordion>

<Info>
  AI extraction output is validated against your schema on a best-effort basis.
  Fields the model couldn't find are returned as `null` rather than omitted or
  invented.
</Info>

***

<Info>
  Looking for product or contact-info extraction? Those are now covered by
  the **standard extraction templates** — see [`product`](/guides/web-scraping#response-%E2%80%94-product-template)
  and [`contact`](/guides/web-scraping#response-%E2%80%94-contact-template) in the Web Scraping guide.
</Info>

***

## 3. `listing` — Category/Search Page Extraction

Extract an array of items from a category or search-results page using your own item schema.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/category/headphones',
    aiPrompt: {
      type: 'listing',
      itemSchema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          price: { type: 'number' }
        }
      },
      maxItems: 20
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/category/headphones',
      ai_prompt={
          'type': 'listing',
          'item_schema': {
              'type': 'object',
              'properties': {'name': {'type': 'string'}, 'price': {'type': 'number'}}
          },
          'max_items': 20
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/category/headphones",
      "aiPrompt": {
        "type": "listing",
        "itemSchema": {
          "type": "object",
          "properties": {"name": {"type": "string"}, "price": {"type": "number"}}
        },
        "maxItems": 20
      }
    }'
  ```
</CodeGroup>

| Field | Type | Required | Default | Description |
| - | - | - | - | - |
| `itemSchema` | object | yes | — | JSON Schema-like object for each item, max 4000 bytes |
| `maxItems` | number | no | `20` | Maximum items to extract (1–50) |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "listing",
      "answered": true,
      "itemCount": 18,
      "items": [
        { "name": "AudioTech Wireless Headphones", "price": 199.99 },
        { "name": "SoundWave Pro Earbuds", "price": 129.00 }
      ]
    }
  }
  ```
</Accordion>

Navigation links, ads, and unrelated content are excluded automatically. Items are returned in page order, capped at `maxItems`.

***

## 4. `summary` — Condensed Page Summary

Summarize the page in paragraph, unordered list, or TL;DR form, optionally focused on one topic.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/blog/state-of-web-scraping-2026',
    aiPrompt: {
      type: 'summary',
      style: 'bullets',
      focus: 'pricing'
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/blog/state-of-web-scraping-2026',
      ai_prompt={'type': 'summary', 'style': 'bullets', 'focus': 'pricing'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/blog/state-of-web-scraping-2026",
      "aiPrompt": {"type": "summary", "style": "bullets", "focus": "pricing"}
    }'
  ```
</CodeGroup>

| Field | Type | Required | Default | Description |
| - | - | - | - | - |
| `style` | string | no | `paragraph` | `paragraph`, `bullets` (unordered list), or `tldr` |
| `focus` | string | no | — | Only summarize content relevant to this topic, max 200 characters |
| `maxLength` | number | no | `5` | Sentence count for `paragraph`/`tldr`, item count for `bullets` (1–20) |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "summary",
      "style": "bullets",
      "answered": true,
      "summary": [
        "Web scraping demand grew due to AI training data needs and real-time price monitoring.",
        "Headless browser detection has become the biggest technical hurdle for scrapers in 2026.",
        "Markdown-based extraction is replacing raw HTML parsing for LLM-based pipelines."
      ]
    }
  }
  ```
</Accordion>

<Info>
  `summary` is a string for `paragraph`/`tldr` style, and an array of strings
  for `bullets` style.
</Info>

***

## 5. `sentiment` — Sentiment Analysis

Analyze overall or aspect-based sentiment across reviews, comments, or opinion text on the page.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/reviews/wireless-headphones',
    aiPrompt: {
      type: 'sentiment',
      aspects: ['sound quality', 'battery life', 'comfort', 'price']
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/reviews/wireless-headphones',
      ai_prompt={
          'type': 'sentiment',
          'aspects': ['sound quality', 'battery life', 'comfort', 'price']
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/reviews/wireless-headphones",
      "aiPrompt": {
        "type": "sentiment",
        "aspects": ["sound quality", "battery life", "comfort", "price"]
      }
    }'
  ```
</CodeGroup>

| Field | Type | Required | Description |
| - | - | - | - |
| `aspects` | string\[] | no | Up to 10 aspects to score individually. Omit to return only overall sentiment. |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "sentiment",
      "answered": true,
      "overall": {
        "label": "positive",
        "score": 0.78,
        "distribution": { "positive": 0.72, "neutral": 0.18, "negative": 0.10 }
      },
      "aspects": [
        { "aspect": "sound quality", "label": "positive", "score": 0.91 },
        { "aspect": "price", "label": "negative", "score": -0.34 }
      ],
      "summary": "Reviewers are overwhelmingly positive on sound quality and battery life, but frequently mention the price as too high."
    }
  }
  ```
</Accordion>

***

## 6. `keywords` — Keywords, Tags & Entities

Extract ranked keywords, named entities, and suggested content tags.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/blog/state-of-web-scraping-2026',
    aiPrompt: {
      type: 'keywords',
      maxKeywords: 10,
      includeEntities: true
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/blog/state-of-web-scraping-2026',
      ai_prompt={'type': 'keywords', 'max_keywords': 10, 'include_entities': True}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/blog/state-of-web-scraping-2026",
      "aiPrompt": {"type": "keywords", "maxKeywords": 10, "includeEntities": true}
    }'
  ```
</CodeGroup>

| Field | Type | Required | Default | Description |
| - | - | - | - | - |
| `maxKeywords` | number | no | `10` | Maximum keywords to return (1–30) |
| `includeEntities` | boolean | no | `false` | Also extract named entities (orgs, people, dates, laws, locations) |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "keywords",
      "answered": true,
      "keywords": [
        { "term": "web scraping", "relevance": 0.97 },
        { "term": "headless browser detection", "relevance": 0.84 }
      ],
      "entities": [
        { "text": "GDPR", "type": "law" },
        { "text": "Cloudflare", "type": "organization" }
      ],
      "suggestedTags": ["web-scraping", "ai", "data-extraction", "proxies", "compliance"]
    }
  }
  ```
</Accordion>

`entities` is only included when `includeEntities` is `true`.

***

## Content Handling

AI Extraction always uses the page's Markdown content — the same content you'd get from the `markdown` format — regardless of any `format` value you set. This keeps the model's input focused on primary content.

## Notes & Limits

* Only one `aiPrompt` mode can be used per request.
* Every mode answers strictly from the scraped page content — the model does not use outside knowledge and will not invent data that isn't present.
* Page content is treated as untrusted data: instructions embedded in scraped pages are ignored.
* Very long pages are truncated before analysis; if you're missing expected data from a long listing or product page, try narrowing the `url` to a more specific page.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.