Web Scraper API

by oxylabs

Web Scraper API is an Automation skill for Claude Code, published by oxylabs in agent-skills.

635 stars1 forkson oxylabs/agent-skillsAdded 2026/08/24+26% in starsRepository updated 2026/08/21

View source on GitHub Download repo ZIP

agent-skillagent-skillsai-agentai-agentsclaude-codeclaude-skillsproxyvideo-dataweb-scrapingweb-unblocking

Install in seconds

Install Web Scraper API

Copy Web Scraper API into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/oxylabs/agent-skills/tree/main/skills/web-scraper-api ~/.claude/skills/web-scraper-api

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/oxylabs/agent-skills.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/web-scraper-api/SKILL.md in oxylabs/agent-skills
Installs to: ~/.claude/skills/web-scraper-api
Collection: One of 5 skills cataloged from this repository
Category: Automation — 1523 skills

What Web Scraper API does

Web Scraper API extracts structured data from web pages, search results, and supported e-commerce targets with parsing, rendering, and geo-targeting. Use it when scraping content that may block requests or needs localized results.

Web Scraper API is cataloged under Automation on DirSkills. Web Scraper API comes from a repository tagged agent-skill, agent-skills, ai-agent, ai-agents and claude-code.

Documentation

README

Excerpt

Oxylabs Web Scraper API

Authentication

Requires HTTP Basic Auth with credentials from environment variables:

curl -u "$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD" ...

Endpoint

POST https://realtime.oxylabs.io/v1/queries   # immediate response
POST https://data.oxylabs.io/v1/queries       # Push-Pull jobs, callbacks, storage
Content-Type: application/json

Core Parameters

This is the opening of the README. Read the full README on GitHub.

Frequently asked about Web Scraper API

What else does oxylabs publish alongside Web Scraper API?
Web Scraper API is one of 5 skills that DirSkills catalogs from oxylabs/agent-skills, the repository it ships in. Its siblings there include Headless Browser, Oxylabs Proxies and Video Data. Each one is a separate skill with its own page in this directory, installs the same way Web Scraper API does, and is maintained by oxylabs in that same repository. The rest of the collection is listed on the oxylabs/agent-skills page.
How does Web Scraper API compare to other Automation skills?
Web Scraper API ranks #1138 by stars among the 1523 Automation skills in this catalog. The most-starred ones next to it are Autonomous Loops, Autonomous Agent Harness and Automation Audit Ops. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Web Scraper API against them. Open each page to compare what they document and how they install.