Get URL Content | Supadata

Request

import {
  Supadata,
  Scrape
} from '@supadata/js';

// Initialize the client
const supadata = new Supadata({
  apiKey: 'YOUR_API_KEY',
});

const webContent: Scrape = await supadata.web.scrape('https://supadata.ai');

from supadata import Supadata, SupadataError

# Initialize the client
supadata = Supadata(api_key="YOUR_API_KEY")

web_content = supadata.web.scrape("https://supadata.ai")
print(f"Page title: {web_content.name}")
print(f"Page content: {web_content.content}")

curl 'https://api.supadata.ai/v1/web/scrape?url=https://example.com' \
-H 'x-api-key: YOUR_API_KEY'

Query Parameters

Parameter	Type	Required	Description
url	string	Yes	URL of the webpage to scrape
noLinks	boolean	No	When true, strips Markdown links from the content. Default: false
lang	string	No	What language to request the content in, if the website supports multiple languages. ISO 639-1 code. Default: en

Response Format

{
  "url": "string", // The URL that was scraped
  "content": "string", // The content extracted from the URL
  "name": "string", // The name of the webpage
  "description": "string", // A description of the webpage
  "ogUrl": "string", // Open Graph URL for the webpage
  "countCharacters": "number", // The number of characters in the content
  "urls": [
    "string" // List of URLs found on the webpage
  ]
}

Error Codes

The API returns HTTP status codes and error codes. See this page for more details.

Respect robots.txt and website terms of service when scraping web content.

Pricing

1 scrape request = 1 credit