Web Extract
v1.0.0Turn any URL into clean article text or Markdown, ready for a language model — no navigation, no scripts, no boilerplate.
- Price per call
- $0.002
- Free each month
- 200 calls
- Rate limit
- 60/min
- MCP server
- Included
Overview
Web Extract takes a URL and returns the part of the page a human would actually read — the article — as Markdown, plain text or cleaned HTML.
Feeding raw HTML to a language model is expensive and counterproductive. On a typical article page, navigation, cookie banners, comment widgets, inline scripts and footers make up the large majority of the bytes, and none of it is the content. Every one of those tokens is billed, and the noise measurably degrades retrieval quality.
Extraction identifies the article body and discards the rest, then converts what remains. The Markdown output preserves what carries meaning — headings, lists, code blocks, blockquotes, links and images — and resolves every URL to an absolute address, so the result stays usable once separated from the page it came from.
Make your first call
Every request goes through the Twotic gateway, which authenticates your key, applies the rate limit, meters the call and forwards it. The exact amount charged comes back on the X-Twotic-Cost header.
curl -X GET "https://www.twotic.dev/v1/web-extract/extract" \
-H "Authorization: Bearer $TWOTIC_KEY"const res = await fetch("https://www.twotic.dev/v1/web-extract/extract", {
method: "GET",
headers: {
Authorization: `Bearer ${process.env.TWOTIC_KEY}`,
},
})
if (!res.ok) throw new Error(await res.text())
const data = await res.json()
console.log(res.headers.get("X-Twotic-Cost"))import os, requests
res = requests.get(
"https://www.twotic.dev/v1/web-extract/extract",
headers={"Authorization": f"Bearer {os.environ['TWOTIC_KEY']}"},
)
res.raise_for_status()
data = res.json()
print(res.headers["X-Twotic-Cost"])package main
import (
"fmt"
"io"
"net/http"
"os"
)
func main() {
req, _ := http.NewRequest("GET", "https://www.twotic.dev/v1/web-extract/extract", nil)
req.Header.Set("Authorization", "Bearer "+os.Getenv("TWOTIC_KEY"))
res, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
out, _ := io.ReadAll(res.Body)
fmt.Println(string(out))
fmt.Println(res.Header.Get("X-Twotic-Cost"))
}<?php
$ch = curl_init("https://www.twotic.dev/v1/web-extract/extract");
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("TWOTIC_KEY"),
],
]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($data);require "net/http"
require "json"
uri = URI("https://www.twotic.dev/v1/web-extract/extract")
req = Net::HTTP::Get.new(uri)
req["Authorization"] = "Bearer #{ENV.fetch('TWOTIC_KEY')}"
res = Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") do |http|
http.request(req)
end
puts JSON.parse(res.body)
puts res["X-Twotic-Cost"]{
"mcpServers": {
"web-extract": {
"type": "http",
"url": "https://www.twotic.dev/api/mcp/web-extract",
"headers": { "Authorization": "Bearer tk_live_your_key" }
}
}
}Endpoints
What you can build
Build a RAG pipeline that ingests URLs
Point the extract endpoint at each source URL and store the Markdown. Because boilerplate is removed before chunking, your embeddings describe the article rather than the site navigation — usually the difference between retrieval that works and retrieval that returns the footer.
Give an agent the ability to read a page
Connected over MCP this becomes a tool an agent can call to read any URL it encounters. It receives clean Markdown instead of a wall of HTML, so far more of its context window is spent on content.
Cut token spend on summarisation
Summarising a raw HTML page routinely costs several times more than summarising its extracted text, for a worse result. Extracting first is normally cheaper than the tokens it saves on the very first call.
Audit a site's outbound links
The links endpoint returns every link on a page, deduplicated, resolved to absolute URLs and split into internal and external — the input to a broken-link check or a competitive backlink review.
Use Web Extract from an AI agent
Every endpoint above is also an MCP tool. Pick your client and the steps below fill in with this API’s real server URL, so there is nothing to substitute.
In claude.ai and the Claude desktop app
- 1Open Settings, then Connectors.
- 2Choose Add custom connector.
- 3Paste the server URL below and confirm.
- 4A Twotic sign-in page opens. Approve the connection.
- 5Start a new conversation. The tools are available there.
Server URL
https://www.twotic.dev/api/mcp/web-extractSigns you in through Twotic. Access expires hourly and renews itself, and you can revoke this one app without touching your API keys.
Once connected, the tools are named web-extract_<endpoint>. Every Web Extract endpoint becomes one. Calls are metered exactly like an HTTP request, against the same allowance and balance.
Frequently asked
Does it run JavaScript?+
No. The page is fetched and parsed as served. That covers server-rendered pages, static sites and most news and documentation, which is the large majority of what gets extracted. A single-page app that renders entirely client-side will return little content.
How is this different from scraping the page myself?+
The fetch is the easy part. What takes the week is everything around it: isolating the article reliably across wildly different page structures, resolving relative URLs, following redirect chains safely, and bounding the whole thing so one hostile page cannot take your service down. That is what you are buying.
Can it fetch internal or private URLs?+
No, deliberately. URLs that resolve to private, internal or reserved addresses are refused, and redirects cannot be used to sneak past that. This protects both our infrastructure and yours.
What is the size limit?+
Pages up to 3 MB are read, and returned content is capped at 200,000 characters. Larger pages are truncated rather than refused.
Is the content cached or stored?+
No. Each call fetches fresh and nothing is retained. Twotic does not log request or response bodies for any API in the catalogue.