Scrape a web page

You are wiring up the agntdata "Web" API to use this endpoint: Scrape a web page.

1. CONNECT over MCP (preferred). Add the superagnt MCP server, then let the client
   run OAuth on first tool call — no token to paste:
     claude mcp add --scope user --transport http superagnt https://mcp.superagnt.com/mcp
     claude mcp login superagnt
   MCP endpoint: https://mcp.superagnt.com/mcp
   After connecting, the tool you want is named exactly:
     data_web_scrape
   Call agnt_tools_search or agnt_tools_list_enabled to confirm it is available.

2. REST fallback (if you are not using MCP):
     POST https://api.agntdata.dev/scrape
     Header: Authorization: Bearer <YOUR_AGNTDATA_API_KEY>
   Required parameters:
    - url (string, required) — The URL to scrape (must be a publicly reachable http(s) URL).
   Optional parameters:
    - formats (array, optional) — Output formats to return. Each entry is either a string preset or an object form. String presets: 'markdown' (default), 'summary', 'html', 'rawHtml', 'links', 'images'. Object forms include { type: 'json', schema, prompt } for structured extraction, { type: 'screenshot', fullPage, quality } for an image, and { type: 'changeTracking', modes } for diffs. Defaults to ['markdown'] when omitted.
    - onlyMainContent (boolean, optional) — Return only the main content of the page, excluding navs, headers, and footers.
    - includeTags (array, optional) — Only include content within these HTML tags / CSS selectors.
    - excludeTags (array, optional) — Exclude content within these HTML tags / CSS selectors.
    - maxAge (integer, optional) — Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).
    - waitFor (integer, optional) — Milliseconds to wait for the page to render before scraping.
    - mobile (boolean, optional) — Emulate a mobile device.
    - timeout (integer, optional) — Request timeout in milliseconds (1000-300000). Default 60000.
    - parsers (array, optional) — PDF parsing configuration.
    - proxy (string, optional) — Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.
    - location (object, optional) — Geographic proxy location and language settings.

3. COST & KEY. This endpoint is priced per call in credits, and each successful response reports its own cost in its meta.costCents field. Only successful calls are charged.
   Get an agntdata API key at https://app.agntdata.dev.

4. TEST. Connect the server (or set the key), discover the tool
   (data_web_scrape), run one minimal call with just the required parameters,
   and report back the shape of the JSON response (top-level fields).

Reference (machine-readable variant): https://superagnt.com/docs/apis/web/endpoints/scrape.md

{
  "name": "scrape",
  "description": "Fetches a single URL and returns its content. Default is clean main-content `markdown` — cheapest and right for almost everything; just read the markdown. Reach for the pricier options only when the task truly needs them: the `json` format with a `schema` runs an LLM to extract structured fields (costs several credits per page — prefer scraping markdown and parsing it in code), and `proxy: enhanced` forces heavy anti-bot handling. Scrape only the specific pages you will actually use. Handles JavaScript-rendered pages.",
  "parameters": {
    "type": "object",
    "properties": {
      "url": {
        "type": "string",
        "format": "uri",
        "description": "The URL to scrape (must be a publicly reachable http(s) URL)."
      },
      "formats": {
        "type": "array",
        "description": "Output formats to return. Each entry is either a string preset or an object form. String presets: 'markdown' (default), 'summary', 'html', 'rawHtml', 'links', 'images'. Object forms include { type: 'json', schema, prompt } for structured extraction, { type: 'screenshot', fullPage, quality } for an image, and { type: 'changeTracking', modes } for diffs. Defaults to ['markdown'] when omitted.",
        "items": {
          "oneOf": [
            {
              "type": "string",
              "enum": [
                "markdown",
                "summary",
                "html",
                "rawHtml",
                "links",
                "images"
              ]
            },
            {
              "type": "object",
              "required": [
                "type"
              ],
              "properties": {
                "type": {
                  "type": "string",
                  "enum": [
                    "json",
                    "screenshot",
                    "changeTracking"
                  ]
                },
                "schema": {
                  "type": "object",
                  "description": "JSON schema describing the structured data to extract (for type 'json')."
                },
                "prompt": {
                  "type": "string",
                  "description": "Natural-language extraction prompt (for type 'json')."
                },
                "fullPage": {
                  "type": "boolean",
                  "description": "Capture the full scrollable page (for type 'screenshot')."
                },
                "modes": {
                  "type": "array",
                  "items": {
                    "type": "string",
                    "enum": [
                      "git-diff",
                      "json"
                    ]
                  },
                  "description": "Change-tracking modes (for type 'changeTracking')."
                }
              }
            }
          ]
        }
      },
      "onlyMainContent": {
        "type": "boolean",
        "description": "Return only the main content of the page, excluding navs, headers, and footers.",
        "default": true
      },
      "includeTags": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "description": "Only include content within these HTML tags / CSS selectors."
      },
      "excludeTags": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "description": "Exclude content within these HTML tags / CSS selectors."
      },
      "maxAge": {
        "type": "integer",
        "description": "Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).",
        "minimum": 0
      },
      "waitFor": {
        "type": "integer",
        "description": "Milliseconds to wait for the page to render before scraping.",
        "minimum": 0,
        "default": 0
      },
      "mobile": {
        "type": "boolean",
        "description": "Emulate a mobile device.",
        "default": false
      },
      "timeout": {
        "type": "integer",
        "description": "Request timeout in milliseconds (1000-300000). Default 60000.",
        "minimum": 1000,
        "maximum": 300000
      },
      "parsers": {
        "type": "array",
        "items": {},
        "description": "PDF parsing configuration."
      },
      "proxy": {
        "type": "string",
        "enum": [
          "basic",
          "enhanced",
          "auto"
        ],
        "description": "Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.",
        "default": "auto"
      },
      "location": {
        "type": "object",
        "description": "Geographic proxy location and language settings.",
        "properties": {
          "country": {
            "type": "string",
            "description": "ISO 3166-1 alpha-2 country code."
          },
          "languages": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
    },
    "required": [
      "url"
    ]
  }
}