Get URL Content | Supadata

Request

import {
  Supadata,
  Scrape
} from '@supadata/js';

// Initialize the client
const supadata = new Supadata({
  apiKey: 'YOUR_API_KEY',
});

const webContent: Scrape = await supadata.web.scrape('https://supadata.ai');

from supadata import Supadata, SupadataError

# Initialize the client
supadata = Supadata(api_key="YOUR_API_KEY")

web_content = supadata.web.scrape("https://supadata.ai")
print(f"Page title: {web_content.name}")
print(f"Page content: {web_content.content}")

curl 'https://api.supadata.ai/v1/web/scrape?url=https://example.com' \
-H 'x-api-key: YOUR_API_KEY'

Query Parameters

Parameter	Type	Required	Description
url	string	Yes	URL of the webpage to scrape
noLinks	boolean	No	Whether to include Markdown links in the response
lang	string	No	What language to request the content in, if the website supports multiple languages. ISO 639-1 code. Default: en

Response Format

{
  "url": "string", // The URL that was scraped
  "content": "string", // The content extracted from the URL
  "name": "string", // The name of the webpage
  "description": "string", // A description of the webpage
  "ogUrl": "string", // Open Graph URL for the webpage
  "countCharacters": "number", // The number of characters in the content
  "urls": [
    "string" // List of URLs found on the webpage
  ]
}

Error Codes

The API returns HTTP status codes and error codes. See this page for more details.

Respect robots.txt and website terms of service when scraping web content.

Pricing

1 transcript request = 1 credit