superagnt
// claude + superagnt

Can Claude scrape websites?

Yes: Claude can scrape websites into rows in a table once superagnt is connected, from a plain chat with no Python. On its own, Claude's web fetch reads a page you link, but not pages built with JavaScript, and it leaves text in the chat rather than rows you can query next week.

Claude in Chrome, on paid plans, can read JavaScript pages in your own browser while Chrome is open. superagnt does the reading on its servers instead: in the run below, Claude scraped Canva's pricing page, which answered a plain request with a bot wall, and saved its four plans to a table, for 0.144 credits.

Connect superagnt to Claude

Copy the setup prompt.
Your agent does the rest.

works with any agent+ 8 more
Install superagnt for me. It's the superagnt plugin from the github.com/superagnt/plugins marketplace, plus the superagnt MCP server (on Claude it's a connector I add in my browser; elsewhere I sign in through my browser). The setup guide for each kind of agent is at https://www.superagnt.com/install
or follow the detailed setup instructions

» the base connection already serves web scraping and the workspace database. Nothing to switch on.

01// on its own, and with superagnt

What Claude does alone, and what changes once it's connected

jobClaude on its ownwith superagnt
Read a page you linkWeb fetch reads the page into the chat. It opens only URLs already in the conversation, and doesn't support pages built with JavaScript.data_web_scrape returns the page's main content as clean markdown. JavaScript-heavy pages are rendered first, on superagnt's servers.
Pages built with JavaScriptClaude in Chrome (paid plans) reads them in your own browser, with your Chrome open.Rendered on superagnt's servers. No browser of yours, no extension.
Keep what you scrapedText in the chat, or a file you save.Rows in your workspace's Postgres database (agnt_db_write), there to query next week or compare against.
Do it again every weekClaude in Chrome runs recurring browser tasks, as long as Chrome is open.A hosted agent on a schedule (agnt_schedules_write) runs on superagnt's servers with your laptop closed.
What a page costsIncluded in your Claude plan.0.144 credits a page (a credit is a cent) in the run below. The database calls cost nothing.
02// a real run

The job, run on superagnt

Real tool calls on superagnt's staging environment, the same calls Claude makes once the server is connected. Each result is what the client received, trimmed where it says so and otherwise unchanged, and each wall time was measured around the call.

the ask

“Scrape Canva's pricing page, pull out each plan and its price, and save them to a table I can check again next week.”

  1. 01Scrape the pricing page805 ms wall time0.144 credits, $0.00144
    → calldata_web_scrape
    {
      "url": "https://www.canva.com/pricing/",
      "formats": ["markdown"],
      "onlyMainContent": true
    }
    ← result
    # Plans and pricing
    ...
    ## Free
    
    Design anything and bring your ideas to life. No cost, just creativity.
    
    US$0
    
    /year for one person
    ...
    ## Pro
    
    Unlock premium content, more powerful design tools, and AI features.
    
    US$144
    
    /year for one person
    ...
    ## Business
    
    Create content faster, market smarter, and grow your business with advanced tools.
    
    US$250
    
    /year per person
    ...
    ## Enterprise
    
    Empower your organization with end-to-end visual communication.
    
    Let’s talk

    » Trimmed: to the four plans' names, lines and prices out of a 24,311-character page, without its cookie banner, images, feature lists, comparison table and the response's metadata, which says this copy came from superagnt's cache, made at 20:50 UTC that day.

  2. 02Make a table for the snapshot3.5 s wall timenot charged
    → callagnt_db_sql
    {
      "action": "apply_migration",
      "name": "create_pricing_snapshots",
      "query": "create table if not exists public.pricing_snapshots (\n  id bigint generated always as identity primary key,\n  company text not null,\n  plan text not null,\n  price_usd numeric,\n  billed text,\n  page_url text not null,\n  scraped_at timestamptz not null default now()\n);"
    }
    ← result
    {
      "success": true
    }
  3. 03Save the plans Claude read from the page331 ms wall timenot charged
    → callagnt_db_write
    {
      "action": "insert",
      "table": "pricing_snapshots",
      "rows": [
        { "company": "Canva", "plan": "Free", "price_usd": 0, "billed": "per year, one person", "page_url": "https://www.canva.com/pricing/" },
        { "company": "Canva", "plan": "Pro", "price_usd": 144, "billed": "per year, one person", "page_url": "https://www.canva.com/pricing/" },
        { "company": "Canva", "plan": "Business", "price_usd": 250, "billed": "per year, per person", "page_url": "https://www.canva.com/pricing/" },
        { "company": "Canva", "plan": "Enterprise", "price_usd": null, "billed": "contact sales", "page_url": "https://www.canva.com/pricing/" }
      ]
    }
    ← result
    {
      "success": true,
      "affected_row_count": 4,
      "note": "Rows omitted to keep the response small. Pass `return_rows: true` if you need the inserted/updated row data."
    }
  4. 04Read them back328 ms wall timenot charged
    → callagnt_db_read
    {
      "action": "select",
      "table": "pricing_snapshots",
      "columns": ["company", "plan", "price_usd", "billed", "scraped_at"],
      "order": "id.asc"
    }
    ← result
    {
      "rows": [
        {
          "company": "Canva",
          "plan": "Free",
          "price_usd": 0,
          "billed": "per year, one person",
          "scraped_at": "2026-11-18T22:36:49.310056+00:00"
        },
        {
          "company": "Canva",
          "plan": "Pro",
          "price_usd": 144,
          "billed": "per year, one person",
          "scraped_at": "2026-11-18T22:36:49.310056+00:00"
        },
        {
          "company": "Canva",
          "plan": "Business",
          "price_usd": 250,
          "billed": "per year, per person",
          "scraped_at": "2026-11-18T22:36:49.310056+00:00"
        },
        {
          "company": "Canva",
          "plan": "Enterprise",
          "price_usd": null,
          "billed": "contact sales",
          "scraped_at": "2026-11-18T22:36:49.310056+00:00"
        }
      ]
    }

The same day, a plain HTTP request for canva.com/pricing got a 403 and Canva's "We’ll have you designing again soon" page, not the prices. Through superagnt the scrape came back in 0.8 seconds, from a copy cached at 20:50 UTC that day; fetched fresh the next day, the same page took 3.9 seconds for the same 0.144 credits. Claude read the plans out of it and the table took 4.1 seconds more. A weekly re-run is the same calls on a schedule, and each run adds dated rows next to the last ones. See Can Claude run tasks on a schedule?.

03// set it up

Set it up in your client

Claude

  1. 01Open the prefilled Add custom connector dialog on claude.ai. One connector covers Claude on the web, Claude Desktop and Cowork.
  2. 02Click Add, then approve on the superagnt consent screen. It creates your superagnt account if you don't have one.
  3. 03Ask: “Scrape <a pricing page> and save each plan and its price to a table.”

Free plans can add one custom connector. On Team and Enterprise plans, an owner adds it under Organization settings, Connectors (Anthropic).

Claude Code

  1. 01In Claude Code, run /plugin marketplace add superagnt/plugins, then /plugin install superagnt@superagnt for the superagnt skills.
  2. 02Add superagnt to Claude as a connector with the prefilled dialog and click Add. Claude Code signed in with your Claude account picks it up.
  3. 03Start a new session.
  4. 04Ask: “Scrape <a pricing page> and save each plan and its price to a table.”

ChatGPT

  1. 01In ChatGPT on the web, go to Plugins, select the plus button, then Add custom MCP server.
  2. 02Name it superagnt, enter the server URL under Connection, and set Authentication to OAuth. The server URL is https://mcp.superagnt.com/mcp.
  3. 03Tick "I understand and want to continue", then select Create as a plugin. Sign in to superagnt when asked. No token needed.
  4. 04Install the plugin, then start a new chat, type @ and select superagnt.
  5. 05Ask: “Scrape <a pricing page> and save each plan and its price to a table.”

ChatGPT takes custom MCP servers on the web, not on mobile, and only Business and Enterprise/Edu workspaces get write actions. On Pro, connected servers are read and fetch only, so saving rows to a table needs one of those plans in ChatGPT (OpenAI).

Codex

  1. 01In a terminal, run codex mcp add superagnt --url https://mcp.superagnt.com/mcp, then codex mcp login superagnt. The login opens a browser to approve.
  2. 02Start a new Codex session.
  3. 03Ask: “Scrape <a pricing page> and save each plan and its price to a table.”
04// side by side

Claude's web fetch, Claude in Chrome and superagnt

Claude web fetchClaude in Chromesuperagnt
Who can use itClaude apps with web search, and the APIPaid plans: Pro, Max, Ultra, Team and EnterpriseClaude, ChatGPT, Claude Code, Codex and Cursor, once connected
Pages built with JavaScriptNot supportedYes, in your own browserYes, rendered on superagnt's servers
Needs your computer onNoYes, with Chrome openNo
Where the results goThe chatThe chat, or a note, doc or form it fillsRows in your workspace database
Runs on a scheduleNoYes, while Chrome is openYes, as a hosted agent, laptop closed
CostIncluded (on the API, tokens only)Included in the plan0.144 credits a page in our run

Claude's columns are from Anthropic's help center and API docs, read Nov 18, 2026.

Pick by the job. To read one page while you browse, Claude in Chrome is already there. To pull the same fields from a list of pages every week, into a table you can sort, compare and hand to the next step (finding the companies, then their work emails), superagnt does it from the chat with nothing left running on your machine.

05// our numbers

What a scrape costs and returns

  • In superagnt's own test of 40 scrapes over 10 real pages, all 40 succeeded, from $0.00144 a page, and the markdown came back 25 times smaller than the raw HTML at the median: less for Claude to read, so less of your chat's room spent on page chrome (the measurements).
  • A scrape can return a copy up to two days old, which is faster (0.8 seconds against 3.9 fresh, for this page) and costs the same. Ask for a fresh read when the price matters today (maxAge: 0).
  • To find the pages first, data_web_map lists a site's URLs, and data_web_search searches the web with page content attached.
  • Pages behind a login stay out of reach: the tools read public pages only, and never fill a form, sign in or post.
07// questions

Common questions

01Can Claude scrape websites?

Yes, into rows in a table once superagnt is connected, from a plain chat with no Python. On its own, Claude's web fetch reads a page you link but doesn't support pages built with JavaScript, and Claude in Chrome reads them in your own browser while Chrome is open.

02Does Claude scrape data on its own?

It reads a page you give it when web search is on, and pulls details out of it into the chat. It opens only URLs already in the conversation, and web fetch skips pages built with JavaScript. With superagnt, it saves the fields as rows in your workspace database.

03Does Claude have a web search tool?

Yes. Claude searches the web when it helps, in the Claude apps; on the API, web search costs $10 per 1,000 searches plus tokens, and web fetch costs tokens only.

04Can Claude Code perform web searches?

Yes. Its WebSearch tool returns result titles and URLs, and WebFetch reads a page and returns a summary of what you asked for, up to 100,000 characters of the page per call.

05Can ChatGPT scrape websites?

ChatGPT searches the web on every plan, Free included, and ChatGPT Work's cloud browser, on paid plans other than Go, can read pages, click buttons and fill in forms. ChatGPT agent is no longer available. superagnt installs in ChatGPT too: Business and Enterprise workspaces can save rows, while Pro gets the read tools only.

06How do I scrape without getting banned?

Read public pages only, slowly, and back off when a site answers 429 or 503. Check robots.txt, use a site's own API when it has one, and never fight its anti-bot defenses or log in with throwaway accounts. superagnt's scrapes run on its servers and never sign in, so no account of yours is at stake.

07What are the top web scraping companies?

There is no neutral top ten: most lists are written by scraping vendors ranking themselves. The field splits into proxy networks and scraping APIs, no-code scraping apps, marketplaces of ready-made scrapers, browser services for AI agents, and open-source libraries you run yourself. Pick by the job: a one-off list, a weekly refresh, or millions of pages a day.

08What is the difference between web crawling and web scraping?

Crawling follows links to find pages, the way a search engine does; scraping pulls specific data out of the pages you found. In superagnt, data_web_map lists a site's URLs and data_web_scrape reads the ones that matter.

09What is web scraping in AI?

Usually it means letting the model do the extraction: you describe the fields you want in plain words, and Claude reads the page for meaning instead of you writing a selector for each field.

08// more answers

What else Claude can do with superagnt

Go deeper: Web scraping MCP server · Web data for AI agents · Load a CSV into a database · Run tasks on a schedule

sources
  1. Anthropic: Web fetch tool
  2. Anthropic: Enable and use web search
  3. Anthropic: Get started with Claude in Chrome
  4. Claude Code docs: tools reference
  5. OpenAI: Searching the web with ChatGPT
  6. OpenAI: Using cloud browser in ChatGPT
  7. OpenAI: ChatGPT agent
  8. Farella Braun + Martel: what the hiQ v. LinkedIn rulings say about scraping