Scrape a web page

You are wiring up the superagnt "Web" API to use this endpoint: Scrape a web page.

1. CONNECT over MCP (preferred). No token to paste either way.
   On Claude (claude.ai, the desktop app, or Claude Code signed in with a Claude
   account), the server is a connector: give the user this link, they click Add
   and approve:
     https://claude.ai/customize/connectors/yours?modal=add-custom-connector&connectorName=superagnt&connectorUrl=https%3A%2F%2Fmcp.superagnt.com%2Fmcp&open_in_browser=1
   Any other MCP client (or Claude Code signed in with an API key): add the server
   and let the client run the browser sign-in:
     claude mcp add --scope user --transport http superagnt https://mcp.superagnt.com/mcp
     claude mcp login superagnt
   MCP endpoint: https://mcp.superagnt.com/mcp
   After connecting, the tool you want is named exactly:
     data_web_scrape
   Call agnt_tools_search or agnt_tools_list_enabled to confirm it is available.

2. REST fallback (if you are not using MCP):
     POST https://api.superagnt.com/scrape
     Header: Authorization: Bearer <YOUR_AGNTDATA_API_KEY>
   Required parameters:
    - url (string, required) — The URL to scrape (must be a publicly reachable http(s) URL).
   Optional parameters:
    - formats (array, optional) — Output formats to return. Each entry is either a string preset or an object form. String presets: 'markdown' (default), 'summary', 'html', 'rawHtml', 'links', 'images'. Object forms include { type: 'json', schema, prompt } for structured extraction, { type: 'screenshot', fullPage, quality } for an image, and { type: 'changeTracking', modes } for diffs. Defaults to ['markdown'] when omitted.
    - onlyMainContent (boolean, optional) — Return only the main content of the page, excluding navs, headers, and footers.
    - includeTags (array, optional) — Only include content within these HTML tags / CSS selectors.
    - excludeTags (array, optional) — Exclude content within these HTML tags / CSS selectors.
    - maxAge (integer, optional) — Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).
    - waitFor (integer, optional) — Milliseconds to wait for the page to render before scraping.
    - mobile (boolean, optional) — Emulate a mobile device.
    - timeout (integer, optional) — Request timeout in milliseconds (1000-300000). Default 60000.
    - parsers (array, optional) — PDF parsing configuration.
    - proxy (string, optional) — Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.
    - location (object, optional) — Geographic proxy location and language settings.

3. COST & KEY. This endpoint is metered — pricing varies by request, from $0.001 per call. Only successful calls are charged.
   Get a superagnt API key at https://app.superagnt.com.

4. TEST. Connect the server (or set the key), discover the tool
   (data_web_scrape), run one minimal call with just the required parameters,
   and report back the shape of the JSON response (top-level fields).

Reference (machine-readable variant): https://www.superagnt.com/docs/apis/web/endpoints/scrape.md

{
  "name": "scrape",
  "description": "Fetches a single URL and returns its content. Default is clean main-content `markdown` — cheapest and right for almost everything; just read the markdown. Reach for the pricier options only when the task truly needs them: the `json` format with a `schema` runs an LLM to extract structured fields (costs several credits per page — prefer scraping markdown and parsing it in code), and `proxy: enhanced` forces heavy anti-bot handling. Scrape only the specific pages you will actually use. Handles JavaScript-rendered pages.",
  "parameters": {
    "type": "object",
    "properties": {
      "url": {
        "type": "string",
        "format": "uri",
        "description": "The URL to scrape (must be a publicly reachable http(s) URL)."
      },
      "formats": {
        "type": "array",
        "description": "Output formats to return. Each entry is either a string preset or an object form. String presets: 'markdown' (default), 'summary', 'html', 'rawHtml', 'links', 'images'. Object forms include { type: 'json', schema, prompt } for structured extraction, { type: 'screenshot', fullPage, quality } for an image, and { type: 'changeTracking', modes } for diffs. Defaults to ['markdown'] when omitted.",
        "items": {
          "oneOf": [
            {
              "type": "string",
              "enum": [
                "markdown",
                "summary",
                "html",
                "rawHtml",
                "links",
                "images"
              ]
            },
            {
              "type": "object",
              "required": [
                "type"
              ],
              "properties": {
                "type": {
                  "type": "string",
                  "enum": [
                    "json",
                    "screenshot",
                    "changeTracking"
                  ]
                },
                "schema": {
                  "type": "object",
                  "description": "JSON schema describing the structured data to extract (for type 'json')."
                },
                "prompt": {
                  "type": "string",
                  "description": "Natural-language extraction prompt (for type 'json')."
                },
                "fullPage": {
                  "type": "boolean",
                  "description": "Capture the full scrollable page (for type 'screenshot')."
                },
                "modes": {
                  "type": "array",
                  "items": {
                    "type": "string",
                    "enum": [
                      "git-diff",
                      "json"
                    ]
                  },
                  "description": "Change-tracking modes (for type 'changeTracking')."
                }
              }
            }
          ]
        }
      },
      "onlyMainContent": {
        "type": "boolean",
        "description": "Return only the main content of the page, excluding navs, headers, and footers.",
        "default": true
      },
      "includeTags": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "description": "Only include content within these HTML tags / CSS selectors."
      },
      "excludeTags": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "description": "Exclude content within these HTML tags / CSS selectors."
      },
      "maxAge": {
        "type": "integer",
        "description": "Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).",
        "minimum": 0
      },
      "waitFor": {
        "type": "integer",
        "description": "Milliseconds to wait for the page to render before scraping.",
        "minimum": 0,
        "default": 0
      },
      "mobile": {
        "type": "boolean",
        "description": "Emulate a mobile device.",
        "default": false
      },
      "timeout": {
        "type": "integer",
        "description": "Request timeout in milliseconds (1000-300000). Default 60000.",
        "minimum": 1000,
        "maximum": 300000
      },
      "parsers": {
        "type": "array",
        "items": {},
        "description": "PDF parsing configuration."
      },
      "proxy": {
        "type": "string",
        "enum": [
          "basic",
          "enhanced",
          "auto"
        ],
        "description": "Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.",
        "default": "auto"
      },
      "location": {
        "type": "object",
        "description": "Geographic proxy location and language settings.",
        "properties": {
          "country": {
            "type": "string",
            "description": "ISO 3166-1 alpha-2 country code."
          },
          "languages": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
    },
    "required": [
      "url"
    ]
  }
}