{"items":[{"id":"cmuh0rx0i03ksqu0654grlsfw","slug":"oxylabs-agent-skills-headless-browser","name":"headless-browser","description":"Connects to Oxylabs remote headless browsers over the Chrome DevTools Protocol (CDP) with Playwright or Puppeteer. Built-in anti-detection, residential proxies, geo-targeting, persistent sessions and profiles, session recording and live VNC inspection for debugging. Use instead of WebFetch or a local browser whenever a site renders with JavaScript, blocks bots (DataDome, Cloudflare, Akamai), needs a real browser session, screenshots or PDFs. Covers connection, retries, error recovery and safe scraping of protected targets without any human help.","authorId":"gh:oxylabs","authorName":"oxylabs","version":"0.1.0","category":"Prompt","securityLevel":"Community","downloadsCount":0,"githubStars":874,"pricePerCall":0,"manifest":{"name":"headless-browser","tools":[],"category":"Prompt","entrypoint":{"type":"prompt"},"description":"Connects to Oxylabs remote headless browsers over the Chrome DevTools Protocol (CDP) with Playwright or Puppeteer. Built-in anti-detection, residential proxies, geo-targeting, persistent sessions and profiles, session recording and live VNC inspection for debugging. Use instead of WebFetch or a local browser whenever a site renders with JavaScript, blocks bots (DataDome, Cloudflare, Akamai), needs a real browser session, screenshots or PDFs. Covers connection, retries, error recovery and safe scraping of protected targets without any human help.","permissions":[],"systemPrompt":"# Oxylabs Headless Browser\n\nRemote Chrome sessions with anti-detection, proxy rotation and geo-targeting built in.\nNothing runs locally: you connect over a WebSocket, drive the browser with the CDP library you already\nuse, and close the session when done. This file holds the rules; the detail lives next to it:\n`scripts/` (copyable templates), `parameters.md`, `errors.md`, `examples.md`, `targets.md`.\n\n## 1. Connect\n\n| Item | Value |\n|------|-------|\n| Endpoint | `wss://USERNAME:PASSWORD@hb.oxylabs.io` |\n| Credentials | `OXY_UNBLOCKER_USERNAME` / `OXY_UNBLOCKER_PASSWORD` (aliases: `OXY_HB_USERNAME` / `OXY_HB_PASSWORD`) |\n| Options | URL query parameters only, e.g. `?p_cc=US&session_name=job-42` (see `parameters.md`) |\n| Libraries | Playwright `chromium.connectOverCDP` (recommended), Puppeteer `puppeteer.connect`, any CDP client |\n| Dashboard / support | `https://hb.oxylabs.io/dashboard` · `support@oxylabs.io` |\n\nRules that prevent the most common `401`:\n\n- Use `wss://`. Plain `ws://` is accepted but sends your password unencrypted.\n- Build the URL by string concatenation with the **raw** password. Do not pass the finished URL through\n  `new URL()` or `urllib.parse`: they percent-encode the password and authentication fails.\n- Use the full username exactly as shown in the dashboard, including any suffix such as `_ab12`.\n- A password containing `:` cannot be sent in the URL. Ask for a new password or send the\n  `Authorization: Basic` header yourself (see `examples.md`).\n- Authentication is checked before parameters: fix a `401` before looking at anything else.\n\n## 2. Quick start\n\nMinimal shape (Playwright, JavaScript):\n\n```javascript\nconst { chromium } = require(\"playwright\");\nconst url = `wss://${process.env.OXY_UNBLOCKER_USERNAME}:${process.env.OXY_UNBLOCKER_PASSWORD}@hb.oxylabs.io?p_cc=US`;\nconst browser = await chromium.connectOverCDP(url, { timeout: 60000 });\ntry {\n  const page = await browser.contexts()[0].newPage(); // default context: backed by fingerprint, proxy, o_profile\n  await page.goto(\"https://example.com\", { waitUntil: \"domcontentloaded\", timeout: 30000 });\n  console.log(await page.content());\n} finally {\n  await browser.close(); // always: an unclosed session keeps its concurrency slot\n}\n```\n\nFor real work copy `scripts/playwright_scrape.js` or `scripts/playwright_scrape.py` whole instead of\nreimplementing. They add the five behaviours everything else in this file assumes:\n\n- **Connect with backoff** (1 s base, 60 s cap, jitter, 6 attempts) only on retryable errors: `429`, `5xx`,\n  `CDP_SESSION_IN_USE`, `CDP_NO_BROWSERS_AVAILABLE`, `CDP_BROWSER_OVERWORKED`, `CDP_BAD_PROXY`,\n  `CDP_GENERAL_ERROR`, timeouts. `400`/`401`/`403` mean the request is wrong: fix, never retry unchanged.\n- **Redact the password** from every error message before logging; Playwright embeds the connection URL in it.\n- **Block `image`, `stylesheet`, `media`, `font`** by default; they cost time and are not needed for data extraction.\n- **Register listeners before navigating**: the `X-Error-Description` response header marks an Oxylabs-side\n  error on page traffic.\n- **`browser.close()` in `finally`**, and wrap the job in an overall deadline so a wedged session still gets there.\n\nPuppeteer, Python async, raw CDP, session hand-over, profiles, recording and fan-out: `examples.md`.\n\n## 3. Sessions and limits\n\n| Limit (account defaults) | Value | When exceeded |\n|--------------------------|-------|---------------|\n| New sessions per second | 10 | `429 CDP_SESSION_RATE_LIMIT_REACHED` (space launches >= 150 ms) |\n| Concurrent sessions | 100 | `429 CDP_MAX_CONCURRENT_SESSIONS_REACHED` |\n| Named (resumable) sessions | 5 | `429 CDP_MAX_PERSISTENT_SESSIONS_REACHED` |\n| Stored profiles (`o_profile`) | 5 | `403 profile limit reached (5 profiles maximum)` |\n| Recordings | 10 | `403 recording limit reached (10 recordings maximum)` |\n| Concurrent inspection viewers | 10 | `CDP_VNC_MAX_CONCURRENT_SESSIONS_REACHED` |\n\n- `session_name` (`^[A-Za-z0-9-]{3,36}$`) makes a session resumable for **10 minutes** after disconnect.\n  `keep_alive` is implied by it; **never send `keep_alive=true` alone** (`400 keep_alive requires session_name`).\n- Reconnecting while the old connection is still attached returns `429 CDP_SESSION_IN_USE`: close it first.\n- Any session lives at most **1 hour**. Plan long jobs as several sessions.\n- An abandoned session keeps its concurrency slot (about 20 s, or the full 10 min when named) and surfaces later\n  as an unrelated `429 CDP_MAX_CONCURRENT_SESSIONS_REACHED`. Closing the Playwright/Puppeteer object is enough.\n- `browser.close()` wipes open pages and cookies even though a named session stays resumable. To hand a session\n  over use Puppeteer `browser.disconnect()` (see `examples.md`, \"Resume a named session\"). State that must\n  outlive a session (logins, clearance cookies) belongs in `o_profile`, not keep-alive.\n- Every distinct parameter combination is provisioned separately: keep the set stable across a job.\n- Under load a connection may queue and end with `503 queue timeout` after about a minute: back off and retry.\n  Higher limits via support.\n\n## 4. Errors\n\nThree channels. **Handshake**: HTTP status plus a short body (Playwright: `WebSocket error: <URL with password>\n<status>` then the body; Puppeteer: `Unexpected server response: <status>`). **Post-connect**: the WebSocket closes\nwith code `3000` and a `CDP_*` reason that only raw clients see; Playwright/Puppeteer just report `Target closed`,\nso treat any disconnect in the first seconds of a session as retryable. **In-page**: CDP error `1337` for one\nrefused command. On page traffic, a response **with** `X-Error-Description` is an Oxylabs network error (retry);\na block page **without** it is the target's decision (change approach, do not retry).\n\n```text\nconnect failed?\n  ├─ 401 ............ fix credentials/scheme, do not retry\n  ├─ 400/403/409 .... fix the named parameter, do not retry unchanged (409: wait 30 s+ for the other session)\n  ├─ 429 ............ backoff; if MAX_CONCURRENT: hunt for unclosed sessions\n  └─ 5xx/503 ........ backoff, up to ~2 min total\nsession dropped (close 3000)?\n  └─ new session with backoff; rotate sticky id on CDP_BAD_PROXY\nnavigate failed with 1337 Invalid target?\n  └─ stop; restricted target (section 7)\npage shows block / 403 wall?\n  ├─ X-Error-Description present .... Oxylabs network issue: backoff + retry\n  └─ absent ......................... target decision: change identity, geo, device, pacing (section 5)\n```\n\nEvery message text with cause and fix: `errors.md`.\n\n## 5. Target safety (DataDome and similar)\n\n**Default parameter set for most jobs: `p_cc`, nothing else.** Every session already gets a fresh fingerprint\nand a fresh residential IP, which is what one-shot fetches and fan-outs of independent pages need. Sticky IPs\nand stored profiles are opt-in tools for a specific need, never a baseline.\n\n**Work order for a protected target.** First write a plain script and make it pass: one fresh session per page,\nthe right geo and device, human pacing, then the escalation ladder below. Only when that script still fails\nafter the ladder do you **recommend persistent profiles to the user** (the setup/consumer pattern below, with\nwhy it should help and what it costs: a setup step, the profile cap of 5) and implement them only on their\ngo-ahead. Never add a profile or sticky id on your own initiative.\n\n| Need | Add | Not for |\n|------|-----|---------|\n| Several connections must look like one visitor (login, cart, a flow that outlives one session) | `proxy_resi_ses_id` + `proxy_resi_ses_time` | one page per session |\n| Cookies or a login must survive between jobs (DataDome clearance, authenticated scraping) | `o_profile`, prepared once by a setup run, after the user agreed | a first attempt; targets that serve without a block |\n| Resume the same browser within 10 minutes | `session_name` | everything else |\n\nWhen you do use them, the combination is one identity. Keep it consistent:\n\n```text\nsetup, exactly once :  ?o_profile=acme-us-01&o_profile_save=true&p_cc=US&proxy_resi_ses_id=acmeus01&proxy_resi_ses_time=30\nconsumers, any number:  ?o_profile=acme-us-01&p_cc=US&proxy_resi_ses_id=acmeus01&proxy_resi_ses_time=30\n```\n\n- **A profile is written by one run and read by the others.** The setup run is the only connection that ever sends\n  `o_profile_save=true`: it earns the cookies (clears the entry page, logs in), verifies the page, closes. Consumer\n  runs send `o_profile=<name>` alone: read-only, no write lock, no `409`. Never \"top up\" a profile from a consumer;\n  when it stops working, run setup again under a new name. In production this is a setup service that prepares and\n  validates profiles and a consumer service that only uses them (`examples.md`, \"Profile setup and consumer runs\").\n- `proxy_resi_ses_id` + `proxy_resi_ses_time` pin the exit IP (max 1440 min). A pinned id disables automatic\n  proxy retry: on `CDP_BAD_PROXY` rotate to a new id.\n- **Never change `p_cc`/`p_city`/`p_state` for an identity** that has cookies. Start a new profile and sticky id.\n- Match interaction to `p_device`: `mobile` = taps, small scrolls, no hover; `desktop` (default) = the opposite.\n  Never set viewport or device metrics yourself; the service owns the fingerprint.\n- Pace like a person: 3 to 8 s between page loads, scroll before clicking, one page at a time per identity.\n  Run parallel identities, not parallel tabs.\n- Escalation when blocked, one rung per fresh connection: fresh session → broader geo (drop `p_city`) →\n  `p_device=mobile` → slow down → inspect (section 6) → recommend persistent profiles to the user → stop and\n  report. Repeating an identical request is never a rung.\n\nBlock signatures per vendor, do/don't table and starting values for a new protected target: `targets.md`.\n\n## 6. Operational hygiene\n\n- **Debugging.** Two tools exist, and whenever the user asks how to debug, what the browser is doing, or why a run\n  fails, tell them about both: **live inspection** (fetch the session id with the CDP command `__session_id`, open\n  `https://hb.oxylabs.io/novnc/?id=<id>` and watch the session as it runs) and **recordings** (`record=true&\n  record_name=<job>` saves a video of the session to replay later in `https://hb.oxylabs.io/dashboard`; cap 10,\n  delete old ones there). Both are off by default. Use them yourself after **3 consecutive failures on one\n  target** to confirm what the page actually shows. Snippet in `examples.md`, \"Session id, live inspection and\n  recording\".\n- **Timeouts.** Connect 60 s, navigation 30 s, plus an overall job deadline.\n- **Logging.** Never log the connection URL or a raw error message; log the parameter set and session id.\n- **Contexts.** Use `browser.contexts()[0]`. A `newContext()` is isolated from profile storage and fingerprint tuning.\n\n## 7. Restricted targets\n\nBlocked by default; access requires a short KYC via your account manager: entertainment and streaming,\nbanking and finance, government sites, gaming platforms, ticketing, webmail, ad networks, third-party IP\ncheckers. Use `https://ip.oxylabs.io/location` to verify your exit IP and geo. A blocked target fails\n`Page.navigate` with CDP error `1337 Invalid target`.\n\nSee also: `scripts/` (full Playwright templates, JS and Python), `parameters.md` (every parameter and its\nvalidation), `errors.md` (every message), `examples.md` (Puppeteer, Python async, raw CDP, reconnection,\nprofiles, recording, fan-out), `targets.md` (block detection, DataDome playbook).","schemaVersion":1},"repoUrl":"https://github.com/oxylabs/agent-skills/tree/main/skills/headless-browser","tags":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking","prompt"],"stats":{"installVelocity7d":0,"retentionRate":0,"executions":0,"rating":null},"origin":"github","source":{"repo":"agent-skills","audit":{"files":[],"binaries":[],"findings":[],"packages":0,"auditedAt":"2026-09-25T13:51:47.670Z","lockfiles":[]},"forks":2,"owner":"oxylabs","stars":874,"topics":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking"],"license":"MIT","fullName":"oxylabs/agent-skills","homepage":null,"language":"JavaScript","pushedAt":"2026-09-24T12:18:41Z","avatarUrl":"https://avatars.githubusercontent.com/u/43960873?v=4","crawledAt":"2026-09-25T13:51:45.915Z","openIssues":1,"manifestFile":"SKILL.md","manifestPath":"skills/headless-browser/SKILL.md","defaultBranch":"main"},"readme":"# Oxylabs Headless Browser\n\nRemote Chrome sessions with anti-detection, proxy rotation and geo-targeting built in.\nNothing runs locally: you connect over a WebSocket, drive the browser with the CDP library you already\nuse, and close the session when done. This file holds the rules; the detail lives next to it:\n`scripts/` (copyable templates), `parameters.md`, `errors.md`, `examples.md`, `targets.md`.\n\n## 1. Connect\n\n| Item | Value |\n|------|-------|\n| Endpoint | `wss://USERNAME:PASSWORD@hb.oxylabs.io` |\n| Credentials | `OXY_UNBLOCKER_USERNAME` / `OXY_UNBLOCKER_PASSWORD` (aliases: `OXY_HB_USERNAME` / `OXY_HB_PASSWORD`) |\n| Options | URL query parameters only, e.g. `?p_cc=US&session_name=job-42` (see `parameters.md`) |\n| Libraries | Playwright `chromium.connectOverCDP` (recommended), Puppeteer `puppeteer.connect`, any CDP client |\n| Dashboard / support | `https://hb.oxylabs.io/dashboard` · `support@oxylabs.io` |\n\nRules that prevent the most common `401`:\n\n- Use `wss://`. Plain `ws://` is accepted but sends your password unencrypted.\n- Build the URL by string concatenation with the **raw** password. Do not pass the finished URL through\n  `new URL()` or `urllib.parse`: they percent-encode the password and authentication fails.\n- Use the full username exactly as shown in the dashboard, including any suffix such as `_ab12`.\n- A password containing `:` cannot be sent in the URL. Ask for a new password or send the\n  `Authorization: Basic` header yourself (see `examples.md`).\n- Authentication is checked before parameters: fix a `401` before looking at anything else.\n\n## 2. Quick start\n\nMinimal shape (Playwright, JavaScript):\n\n```javascript\nconst { chromium } = require(\"playwright\");\nconst url = `wss://${process.env.OXY_UNBLOCKER_USERNAME}:${process.env.OXY_UNBLOCKER_PASSWORD}@hb.oxylabs.io?p_cc=US`;\nconst browser = await chromium.connectOverCDP(url, { timeout: 60000 });\ntry {\n  const page = await browser.contexts()[0].newPage(); // default context: backed by fingerprint, proxy, o_profile\n  await page.goto(\"https://example.com\", { waitUntil: \"domcontentloaded\", timeout: 30000 });\n  console.log(await page.content());\n} finally {\n  await browser.close(); // always: an unclosed session keeps its concurrency slot\n}\n```\n\nFor real work copy `scripts/playwright_scrape.js` or `scripts/playwright_scrape.py` whole instead of\nreimplementing. They add the five behaviours everything else in this file assumes:\n\n- **Connect with backoff** (1 s base, 60 s cap, jitter, 6 attempts) only on retryable errors: `429`, `5xx`,\n  `CDP_SESSION_IN_USE`, `CDP_NO_BROWSERS_AVAILABLE`, `CDP_BROWSER_OVERWORKED`, `CDP_BAD_PROXY`,\n  `CDP_GENERAL_ERROR`, timeouts. `400`/`401`/`403` mean the request is wrong: fix, never retry unchanged.\n- **Redact the password** from every error message before logging; Playwright embeds the connection URL in it.\n- **Block `image`, `stylesheet`, `media`, `font`** by default; they cost time and are not needed for data extraction.\n- **Register listeners before navigating**: the `X-Error-Description` response header marks an Oxylabs-side\n  error on page traffic.\n- **`browser.close()` in `finally`**, and wrap the job in an overall deadline so a wedged session still gets there.\n\nPuppeteer, Python async, raw CDP, session hand-over, profiles, recording and fan-out: `examples.md`.\n\n## 3. Sessions and limits\n\n| Limit (account defaults) | Value | When exceeded |\n|--------------------------|-------|---------------|\n| New sessions per second | 10 | `429 CDP_SESSION_RATE_LIMIT_REACHED` (space launches >= 150 ms) |\n| Concurrent sessions | 100 | `429 CDP_MAX_CONCURRENT_SESSIONS_REACHED` |\n| Named (resumable) sessions | 5 | `429 CDP_MAX_PERSISTENT_SESSIONS_REACHED` |\n| Stored profiles (`o_profile`) | 5 | `403 profile limit reached (5 profiles maximum)` |\n| Recordings | 10 | `403 recording limit reached (10 recordings maximum)` |\n| Concurrent inspection viewers | 10 | `CDP_VNC_MAX_CONCURRENT_SESSIONS_REACHED` |\n\n- `session_name` (`^[A-Za-z0-9-]{3,36}$`) ma","createdAt":"2026-09-25T13:51:47.682Z","updatedAt":"2026-09-25T13:51:47.682Z"},{"id":"cmuh0rx0z03kvqu06zkuqt6pk","slug":"oxylabs-agent-skills-proxies","name":"proxies","description":"Oxylabs proxy networks: Residential, Mobile, shared Datacenter/ISP, and Dedicated Datacenter/ISP proxies with geo-targeting, IP rotation, session persistence, and port-based sticky IPs. Use when routing traffic through proxies, building scrapers with proxy auth, rotating or sticky sessions, whitelisting IPs, or accessing geo-restricted content.","authorId":"gh:oxylabs","authorName":"oxylabs","version":"0.1.0","category":"Prompt","securityLevel":"Community","downloadsCount":0,"githubStars":874,"pricePerCall":0,"manifest":{"name":"proxies","tools":[],"category":"Prompt","entrypoint":{"type":"prompt"},"description":"Oxylabs proxy networks: Residential, Mobile, shared Datacenter/ISP, and Dedicated Datacenter/ISP proxies with geo-targeting, IP rotation, session persistence, and port-based sticky IPs. Use when routing traffic through proxies, building scrapers with proxy auth, rotating or sticky sessions, whitelisting IPs, or accessing geo-restricted content.","permissions":[],"systemPrompt":"# Oxylabs Proxies\n\n## Proxy Types Overview\n\n| Type | Host | Port | Best For |\n|------|------|------|----------|\n| Residential | `pr.oxylabs.io` | `7777` | High anonymity, geo-targeting |\n| Mobile | `pr.oxylabs.io` | `7777` | Mobile-specific content, highest trust |\n| Datacenter (shared) | `dc.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Speed, high volume |\n| ISP (shared) | `isp.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Speed + anonymity balance |\n| Dedicated Datacenter | `ddc.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Owned IPs, port-based access |\n| Dedicated ISP | `disp.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Owned ISP IPs, ASN locked |\n\nResidential/Mobile use `pr.oxylabs.io:7777` with username session parameters. Datacenter/ISP and Dedicated self-service products use proxy-list ports starting at `8001` for assigned/static IPs and `8000` for automatic rotation.\n\n## Environment Variables\n\nUse credentials for the specific proxy product family:\n\n| Product family | Variables | Username prefix |\n|----------------|-----------|-----------------|\n| Residential, Mobile | `OXY_RES_USERNAME`, `OXY_RES_PASSWORD` | `customer-` |\n| Datacenter, ISP, Dedicated Datacenter, Dedicated ISP | `OXY_DC_USERNAME`, `OXY_DC_PASSWORD` | `user-` for self-service/shared |\n\n## Authentication Format\n\n```\ncustomer-USERNAME:PASSWORD    # Residential, Mobile\nuser-USERNAME:PASSWORD          # Shared Datacenter, Shared ISP\n```\n\nDedicated proxy auth (Self-Service vs Enterprise) is in [dedicated-datacenter.md](dedicated-datacenter.md) and [dedicated-isp.md](dedicated-isp.md).\n\nUse separate credentials for Residential/Mobile (`OXY_RES_USERNAME`, `OXY_RES_PASSWORD`) and Datacenter/ISP (`OXY_DC_USERNAME`, `OXY_DC_PASSWORD`).\n\nWith parameters:\n```\ncustomer-USERNAME-cc-US-city-new_york-sessid-abc123:PASSWORD\n```\n\n## Quick Start\n\n**Residential/Mobile proxy:**\n```bash\ncurl -x \"pr.oxylabs.io:7777\" \\\n  -U \"customer-$OXY_RES_USERNAME:$OXY_RES_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\n**Datacenter proxy:**\n```bash\ncurl -x \"dc.oxylabs.io:8000\" \\\n  -U \"user-$OXY_DC_USERNAME:$OXY_DC_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\n**ISP proxy:**\n```bash\ncurl -x \"isp.oxylabs.io:8001\" \\\n  -U \"user-$OXY_DC_USERNAME:$OXY_DC_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\nFor Datacenter, ISP, Dedicated Datacenter, and Dedicated ISP proxies, use dashboard proxy-list ports starting at `8001` for assigned/static IPs; the first listed IP uses `8001`. Switch to port `8000` only when the task calls for automatic rotation.\n\n## Protocols (Residential)\n\n| Protocol|\tTransport|\tEntry point|\tUse when|\n|---------|----------|-----------|----------|\n|HTTP|\tTCP|\tpr.oxylabs.io:7777|\tDefault. Supported by common libraries and third-party software|\n|HTTPS|\tTCP|\thttps://pr.oxylabs.io:7777|\tFully encrypted connection to the proxy|\n|SOCKS5|\tTCP and UDP|\tsocks5h://pr.oxylabs.io:7777|\tClient requires SOCKS5|\n|HTTP/3| (MASQUE)\tUDP (QUIC)|\tmasque.oxylabs.io:50000|\tNative HTTP/3, UDP, DNS over UDP, WebRTC/SIP/gaming traffic|\n\nIf the task only needs standard HTTP or HTTPS (TCP) requests, use the main endpoint pr.oxylabs.io:7777. MASQUE adds QUIC connection setup overhead and is only worth it when UDP or native HTTP/3 transport is required.\n\nGoogle is a restricted target over UDP connections (applies to MASQUE and SOCKS5 UDP).\n\n## Geo-Targeting Parameters\n\nFor Residential/Mobile, append username parameters with hyphens unless noted:\n\n| Parameter | Format | Example |\n|-----------|--------|---------|\n| `cc` | ISO 3166-1 alpha-2 | `-cc-US`, `-cc-DE`, `-cc-GB` |\n| `city` | English, underscores for spaces | `-city-new_york`, `-city-los_angeles` |\n| `st` | US states with `us_` prefix | `-st-us_california`, `-st-us_texas` |\n| `postalcode` | 5-digit US ZIP, pair with `cc-US` | `-cc-US-postalcode-90210` |\n| `ASN` | Residential/Mobile carrier ASN | `-ASN-21928` |\n| `X-Oxylabs-Geolocation` | Proxy header `lat:lon;radius_miles` | `49.9235:-97.0811;10` |\n\nZIP targeting is US-only. Coordinate radius cannot be lower than 10 miles. If both country and ASN are used, country applies.\n\n**Example with geo-targeting:**\n```bash\ncurl -x \"pr.oxylabs.io:7777\" \\\n  -U \"customer-$OXY_RES_USERNAME-cc-US-city-new_york:$OXY_RES_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\nFor Shared Datacenter/ISP country rotation, use `-country-US` with `user-` credentials on the rotation port.\n\n## Session Control\n\n| Parameter | Description | Notes |\n|-----------|-------------|-------|\n| `sessid` | Keep the same IP across requests | Standard session is 10 minutes or up to 60s of inactivity |\n| `sessid_oneip` | Bind the session to one exact exit node | Returns `502` if that IP becomes unavailable |\n| `sesstime` | Set session duration in minutes with `sessid` or `sessid_oneip` | Residential backconnect supports up to 1440 minutes; some entry modes cap lower |\n\n**Sticky session example:**\n```bash\ncurl -x \"pr.oxylabs.io:7777\" \\\n  -U \"customer-$OXY_RES_USERNAME-cc-US-sessid-mysession123:$OXY_RES_PASSWORD\" \\\n  \"https://example.com\"\n```\n\n**Timed session (5 minutes):**\n```bash\ncurl -x \"pr.oxylabs.io:7777\" \\\n  -U \"customer-$OXY_RES_USERNAME-sessid-abc123-sesstime-5:$OXY_RES_PASSWORD\" \\\n  \"https://example.com\"\n```\n\n## Choosing the Right Proxy Type\n\n| Need | Recommended |\n|------|-------------|\n| Highest anonymity | Residential |\n| Mobile app content | Mobile |\n| Speed & volume | Datacenter |\n| Speed + anonymity | ISP |\n| Owned dedicated IPs | Dedicated Datacenter or Dedicated ISP |\n| Geo-restricted content | Residential/Mobile with `cc`/`city`/`postalcode`, or DC/ISP by country/assigned port where suitable |\n\n## Default Behavior\n\n- Without parameters: random IP for each request\n- Residential/Mobile share the same endpoint but different IP pools\n- Sessions auto-expire and get new IPs\n\n## Additional Resources\n\n- Shared proxy details (Residential, Mobile, Datacenter, ISP): [proxy-types.md](proxy-types.md)\n- Dedicated Datacenter (Self-Service + Enterprise): [dedicated-datacenter.md](dedicated-datacenter.md)\n- Dedicated ISP (Self-Service + Enterprise): [dedicated-isp.md](dedicated-isp.md)\n- Code examples (all languages): [examples.md](examples.md)","schemaVersion":1},"repoUrl":"https://github.com/oxylabs/agent-skills/tree/main/skills/proxies","tags":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking","prompt"],"stats":{"installVelocity7d":0,"retentionRate":0,"executions":0,"rating":null},"origin":"github","source":{"repo":"agent-skills","audit":{"files":[],"binaries":[],"findings":[],"packages":0,"auditedAt":"2026-09-25T13:51:47.670Z","lockfiles":[]},"forks":2,"owner":"oxylabs","stars":874,"topics":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking"],"license":"MIT","fullName":"oxylabs/agent-skills","homepage":null,"language":"JavaScript","pushedAt":"2026-09-24T12:18:41Z","avatarUrl":"https://avatars.githubusercontent.com/u/43960873?v=4","crawledAt":"2026-09-25T13:51:45.915Z","openIssues":1,"manifestFile":"SKILL.md","manifestPath":"skills/proxies/SKILL.md","defaultBranch":"main"},"readme":"# Oxylabs Proxies\n\n## Proxy Types Overview\n\n| Type | Host | Port | Best For |\n|------|------|------|----------|\n| Residential | `pr.oxylabs.io` | `7777` | High anonymity, geo-targeting |\n| Mobile | `pr.oxylabs.io` | `7777` | Mobile-specific content, highest trust |\n| Datacenter (shared) | `dc.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Speed, high volume |\n| ISP (shared) | `isp.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Speed + anonymity balance |\n| Dedicated Datacenter | `ddc.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Owned IPs, port-based access |\n| Dedicated ISP | `disp.oxylabs.io` | `8000` rotation / `8001+` assigned/static | Owned ISP IPs, ASN locked |\n\nResidential/Mobile use `pr.oxylabs.io:7777` with username session parameters. Datacenter/ISP and Dedicated self-service products use proxy-list ports starting at `8001` for assigned/static IPs and `8000` for automatic rotation.\n\n## Environment Variables\n\nUse credentials for the specific proxy product family:\n\n| Product family | Variables | Username prefix |\n|----------------|-----------|-----------------|\n| Residential, Mobile | `OXY_RES_USERNAME`, `OXY_RES_PASSWORD` | `customer-` |\n| Datacenter, ISP, Dedicated Datacenter, Dedicated ISP | `OXY_DC_USERNAME`, `OXY_DC_PASSWORD` | `user-` for self-service/shared |\n\n## Authentication Format\n\n```\ncustomer-USERNAME:PASSWORD    # Residential, Mobile\nuser-USERNAME:PASSWORD          # Shared Datacenter, Shared ISP\n```\n\nDedicated proxy auth (Self-Service vs Enterprise) is in [dedicated-datacenter.md](dedicated-datacenter.md) and [dedicated-isp.md](dedicated-isp.md).\n\nUse separate credentials for Residential/Mobile (`OXY_RES_USERNAME`, `OXY_RES_PASSWORD`) and Datacenter/ISP (`OXY_DC_USERNAME`, `OXY_DC_PASSWORD`).\n\nWith parameters:\n```\ncustomer-USERNAME-cc-US-city-new_york-sessid-abc123:PASSWORD\n```\n\n## Quick Start\n\n**Residential/Mobile proxy:**\n```bash\ncurl -x \"pr.oxylabs.io:7777\" \\\n  -U \"customer-$OXY_RES_USERNAME:$OXY_RES_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\n**Datacenter proxy:**\n```bash\ncurl -x \"dc.oxylabs.io:8000\" \\\n  -U \"user-$OXY_DC_USERNAME:$OXY_DC_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\n**ISP proxy:**\n```bash\ncurl -x \"isp.oxylabs.io:8001\" \\\n  -U \"user-$OXY_DC_USERNAME:$OXY_DC_PASSWORD\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\nFor Datacenter, ISP, Dedicated Datacenter, and Dedicated ISP proxies, use dashboard proxy-list ports starting at `8001` for assigned/static IPs; the first listed IP uses `8001`. Switch to port `8000` only when the task calls for automatic rotation.\n\n## Protocols (Residential)\n\n| Protocol|\tTransport|\tEntry point|\tUse when|\n|---------|----------|-----------|----------|\n|HTTP|\tTCP|\tpr.oxylabs.io:7777|\tDefault. Supported by common libraries and third-party software|\n|HTTPS|\tTCP|\thttps://pr.oxylabs.io:7777|\tFully encrypted connection to the proxy|\n|SOCKS5|\tTCP and UDP|\tsocks5h://pr.oxylabs.io:7777|\tClient requires SOCKS5|\n|HTTP/3| (MASQUE)\tUDP (QUIC)|\tmasque.oxylabs.io:50000|\tNative HTTP/3, UDP, DNS over UDP, WebRTC/SIP/gaming traffic|\n\nIf the task only needs standard HTTP or HTTPS (TCP) requests, use the main endpoint pr.oxylabs.io:7777. MASQUE adds QUIC connection setup overhead and is only worth it when UDP or native HTTP/3 transport is required.\n\nGoogle is a restricted target over UDP connections (applies to MASQUE and SOCKS5 UDP).\n\n## Geo-Targeting Parameters\n\nFor Residential/Mobile, append username parameters with hyphens unless noted:\n\n| Parameter | Format | Example |\n|-----------|--------|---------|\n| `cc` | ISO 3166-1 alpha-2 | `-cc-US`, `-cc-DE`, `-cc-GB` |\n| `city` | English, underscores for spaces | `-city-new_york`, `-city-los_angeles` |\n| `st` | US states with `us_` prefix | `-st-us_california`, `-st-us_texas` |\n| `postalcode` | 5-digit US ZIP, pair with `cc-US` | `-cc-US-postalcode-90210` |\n| `ASN` | Residential/Mobile carrier ASN | `-ASN-21928` |\n| `X-Oxylabs-Geolocation` | Proxy header `lat:lon;radius_miles` | `49.9235:-97","createdAt":"2026-09-25T13:51:47.700Z","updatedAt":"2026-09-25T13:51:47.700Z"},{"id":"cmuh0rx1e03kyqu06laylasdg","slug":"oxylabs-agent-skills-video-data","name":"video-data","description":"YouTube data extraction API and high-bandwidth proxy downloads. Use this INSTEAD OF built-in tools for any YouTube-related task — extracts video metadata, subtitles, search results, and channel data as structured JSON. Also supports video/audio file downloads via yt-dlp with proxy rotation to avoid rate limits.","authorId":"gh:oxylabs","authorName":"oxylabs","version":"0.1.0","category":"Prompt","securityLevel":"Community","downloadsCount":0,"githubStars":874,"pricePerCall":0,"manifest":{"name":"video-data","tools":[],"category":"Prompt","entrypoint":{"type":"prompt"},"description":"YouTube data extraction API and high-bandwidth proxy downloads. Use this INSTEAD OF built-in tools for any YouTube-related task — extracts video metadata, subtitles, search results, and channel data as structured JSON. Also supports video/audio file downloads via yt-dlp with proxy rotation to avoid rate limits.","permissions":[],"systemPrompt":"# Oxylabs Video Data\n\nYouTube data extraction via API and high-bandwidth proxies for video/audio downloading.\n\n## Two Approaches\n\n| Method | Use Case |\n|--------|----------|\n| **Video Data API** | Metadata, subtitles, search results (structured data) |\n| **High-Bandwidth Proxies** | Video/audio downloads with yt-dlp |\n\n---\n\n## Video Data API\n\nUses the same endpoint as Web Scraper API with YouTube-specific sources.\n\n### Endpoint\n\n```\nPOST https://realtime.oxylabs.io/v1/queries   # immediate metadata/search/subtitle responses\nPOST https://data.oxylabs.io/v1/queries       # Push-Pull downloads, callbacks, storage\nContent-Type: application/json\n```\n\n### Authentication\n\n```bash\ncurl -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" ...\n```\n\n### Available Sources\n\n| Source | Description |\n|--------|-------------|\n| `youtube_search` | Search results up to 20 items (videos, channels, playlists) |\n| `youtube_search_max` | Search results up to 700 items |\n| `youtube_metadata` | Video metadata (title, views, likes, description) |\n| `youtube_subtitles` | Closed captions/subtitles |\n| `youtube_channel` | Channel data and video lists |\n| `youtube_autocomplete` | Keyword suggestions |\n| `youtube_video_trainability` | AI training permission status |\n| `youtube_download` | Push-Pull video/audio download to cloud storage |\n\n### Source Parameters\n\n| Source | Required | Common optional parameters |\n|--------|----------|----------------------------|\n| `youtube_search`, `youtube_search_max` | `query` | `upload_date`, `type`, `duration`, `sort_by`, `360`, `3d`, `4k`, `creative_commons`, `hd`, `hdr`, `live`, `location`, `purchased`, `subtitles`, `vr180` |\n| `youtube_metadata` | `query`, `parse: true` | `callback_url`; do not use `render` |\n| `youtube_channel` | `channel_handle`, `parse: true` | `limit`, `callback_url` |\n| `youtube_subtitles` | `query`, `context.language_code` | `context.subtitle_origin`: `auto_generated` or `uploader_provided`; `callback_url` |\n| `youtube_autocomplete` | `query` | `location` country code, `language`, `callback_url` |\n| `youtube_video_trainability` | `video_id` | `callback_url` |\n| `youtube_download` | `query`, `storage_type`, `storage_url` | `callback_url`, `context.download_type`, `context.video_quality`, `context.start_at`, `context.end_at` |\n\nFor `youtube_download`, use Push-Pull and cloud storage. `storage_type` is `gcs`, `s3`, or `s3_compatible`; `download_type` is `audio`, `video`, or `audio_video`; `video_quality` is `best`, `worst`, or `144`, `360`, `480`, `720`, `1080`, `1440`, `2160`, `4320`.\n\nDownloads default to 720p when available and are limited to 1 hour. `start_at`/`end_at` use `hh:mm:ss`; `end_at` must be later than `start_at`. For batch downloads, use `/v1/queries/batch` with a `query` array only; keep all other parameters singular.\n\n### Quick Start\n\n**Video metadata:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_metadata\",\n    \"query\": \"dQw4w9WgXcQ\",\n    \"parse\": true\n  }'\n```\n\n**YouTube search:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_search\",\n    \"query\": \"python tutorial\"\n  }'\n```\n\n**Channel data:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_channel\",\n    \"channel_handle\": \"@channelhandle\",\n    \"parse\": true,\n    \"limit\": 10\n  }'\n```\n\n---\n\n## High-Bandwidth Proxies (Video Downloads)\n\nFor actual video/audio file downloads using yt-dlp.\n\n### Setup\n\nContact Oxylabs sales team to get a dedicated high-bandwidth endpoint.\n\n**Default configuration:**\n- Port: `60000`\n- Endpoint: Provided after purchase\n\nUse `OXY_HB_ENDPOINT`; if absent, check `OXYLABS_HB_ENDPOINT`.\n\n### Connection Test\n\n```bash\ncurl -x \"http://USERNAME-test:PASSWORD@YOUR_ENDPOINT:60000\" \\\n  \"https://ip.oxylabs.io/location\"\n```\n\n### yt-dlp Integration\n\n**With session rotation (different IP per download):**\n```bash\nyt-dlp --proxy \"http://USERNAME-Random1Session2ID:PASSWORD@YOUR_ENDPOINT:60000\" \\\n  \"https://www.youtube.com/watch?v=VIDEO_ID\"\n```\n\nChange the session ID for each download to get a fresh IP.\n\n### Python with yt-dlp\n\n```python\nimport yt_dlp\nimport os\nimport uuid\n\nusername = os.environ[\"OXY_WSA_USERNAME\"]\npassword = os.environ[\"OXY_WSA_PASSWORD\"]\nendpoint = os.environ[\"OXY_HB_ENDPOINT\"]  # Your dedicated endpoint\n\n# Random session for unique IP\nsession_id = str(uuid.uuid4()).replace(\"-\", \"\")\n\nydl_opts = {\n    \"proxy\": f\"http://{username}-{session_id}:{password}@{endpoint}:60000\",\n    \"format\": \"best\",\n    \"outtmpl\": \"%(title)s.%(ext)s\"\n}\n\nwith yt_dlp.YoutubeDL(ydl_opts) as ydl:\n    ydl.download([\"https://www.youtube.com/watch?v=VIDEO_ID\"])\n```\n\n---\n\n## Choosing the Right Method\n\n| Need | Method |\n|------|--------|\n| Video metadata (title, views, likes) | Video Data API |\n| Search results | Video Data API |\n| Subtitles | Video Data API |\n| Channel information | Video Data API |\n| Download video files | High-Bandwidth Proxies + yt-dlp |\n| Download audio files | High-Bandwidth Proxies + yt-dlp |\n\nFor more examples, see [examples.md](examples.md).","schemaVersion":1},"repoUrl":"https://github.com/oxylabs/agent-skills/tree/main/skills/video-data","tags":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking","prompt"],"stats":{"installVelocity7d":0,"retentionRate":0,"executions":0,"rating":null},"origin":"github","source":{"repo":"agent-skills","audit":{"files":[],"binaries":[],"findings":[],"packages":0,"auditedAt":"2026-09-25T13:51:47.670Z","lockfiles":[]},"forks":2,"owner":"oxylabs","stars":874,"topics":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking"],"license":"MIT","fullName":"oxylabs/agent-skills","homepage":null,"language":"JavaScript","pushedAt":"2026-09-24T12:18:41Z","avatarUrl":"https://avatars.githubusercontent.com/u/43960873?v=4","crawledAt":"2026-09-25T13:51:45.915Z","openIssues":1,"manifestFile":"SKILL.md","manifestPath":"skills/video-data/SKILL.md","defaultBranch":"main"},"readme":"# Oxylabs Video Data\n\nYouTube data extraction via API and high-bandwidth proxies for video/audio downloading.\n\n## Two Approaches\n\n| Method | Use Case |\n|--------|----------|\n| **Video Data API** | Metadata, subtitles, search results (structured data) |\n| **High-Bandwidth Proxies** | Video/audio downloads with yt-dlp |\n\n---\n\n## Video Data API\n\nUses the same endpoint as Web Scraper API with YouTube-specific sources.\n\n### Endpoint\n\n```\nPOST https://realtime.oxylabs.io/v1/queries   # immediate metadata/search/subtitle responses\nPOST https://data.oxylabs.io/v1/queries       # Push-Pull downloads, callbacks, storage\nContent-Type: application/json\n```\n\n### Authentication\n\n```bash\ncurl -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" ...\n```\n\n### Available Sources\n\n| Source | Description |\n|--------|-------------|\n| `youtube_search` | Search results up to 20 items (videos, channels, playlists) |\n| `youtube_search_max` | Search results up to 700 items |\n| `youtube_metadata` | Video metadata (title, views, likes, description) |\n| `youtube_subtitles` | Closed captions/subtitles |\n| `youtube_channel` | Channel data and video lists |\n| `youtube_autocomplete` | Keyword suggestions |\n| `youtube_video_trainability` | AI training permission status |\n| `youtube_download` | Push-Pull video/audio download to cloud storage |\n\n### Source Parameters\n\n| Source | Required | Common optional parameters |\n|--------|----------|----------------------------|\n| `youtube_search`, `youtube_search_max` | `query` | `upload_date`, `type`, `duration`, `sort_by`, `360`, `3d`, `4k`, `creative_commons`, `hd`, `hdr`, `live`, `location`, `purchased`, `subtitles`, `vr180` |\n| `youtube_metadata` | `query`, `parse: true` | `callback_url`; do not use `render` |\n| `youtube_channel` | `channel_handle`, `parse: true` | `limit`, `callback_url` |\n| `youtube_subtitles` | `query`, `context.language_code` | `context.subtitle_origin`: `auto_generated` or `uploader_provided`; `callback_url` |\n| `youtube_autocomplete` | `query` | `location` country code, `language`, `callback_url` |\n| `youtube_video_trainability` | `video_id` | `callback_url` |\n| `youtube_download` | `query`, `storage_type`, `storage_url` | `callback_url`, `context.download_type`, `context.video_quality`, `context.start_at`, `context.end_at` |\n\nFor `youtube_download`, use Push-Pull and cloud storage. `storage_type` is `gcs`, `s3`, or `s3_compatible`; `download_type` is `audio`, `video`, or `audio_video`; `video_quality` is `best`, `worst`, or `144`, `360`, `480`, `720`, `1080`, `1440`, `2160`, `4320`.\n\nDownloads default to 720p when available and are limited to 1 hour. `start_at`/`end_at` use `hh:mm:ss`; `end_at` must be later than `start_at`. For batch downloads, use `/v1/queries/batch` with a `query` array only; keep all other parameters singular.\n\n### Quick Start\n\n**Video metadata:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_metadata\",\n    \"query\": \"dQw4w9WgXcQ\",\n    \"parse\": true\n  }'\n```\n\n**YouTube search:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_search\",\n    \"query\": \"python tutorial\"\n  }'\n```\n\n**Channel data:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"source\": \"youtube_channel\",\n    \"channel_handle\": \"@channelhandle\",\n    \"parse\": true,\n    \"limit\": 10\n  }'\n```\n\n---\n\n## High-Bandwidth Proxies (Video Downloads)\n\nFor actual video/audio file downloads using yt-dlp.\n\n### Setup\n\nContact Oxylabs sales team to get a dedicated high-bandwidth endpoint.\n\n**Default configuration:**\n- Port: `60000`\n- Endpoint: Provided after purchase\n\nUse `OXY_HB_ENDPOINT`; if absent, check `OXYLABS_HB_ENDPOINT`.\n\n### Connection Test\n\n```bash\ncurl -x \"ht","createdAt":"2026-09-25T13:51:47.714Z","updatedAt":"2026-09-25T13:51:47.714Z"},{"id":"cmuh0rx1n03l1qu06152zai0j","slug":"oxylabs-agent-skills-web-scraper-api","name":"web-scraper-api","description":"Production-grade web scraping with automatic anti-bot bypass, structured JSON parsing for 40+ targets, and geo-targeting. Use when the user needs to scrape web pages, extract product data, get search results, or collect structured data from supported e-commerce and search platforms without worrying about getting blocked and when geo targeting is required.","authorId":"gh:oxylabs","authorName":"oxylabs","version":"0.1.0","category":"Prompt","securityLevel":"Community","downloadsCount":0,"githubStars":874,"pricePerCall":0,"manifest":{"name":"web-scraper-api","tools":[],"category":"Prompt","entrypoint":{"type":"prompt"},"description":"Production-grade web scraping with automatic anti-bot bypass, structured JSON parsing for 40+ targets, and geo-targeting. Use when the user needs to scrape web pages, extract product data, get search results, or collect structured data from supported e-commerce and search platforms without worrying about getting blocked and when geo targeting is required.","permissions":[],"systemPrompt":"# Oxylabs Web Scraper API\n\n## Authentication\n\nRequires HTTP Basic Auth with credentials from environment variables:\n\n```bash\ncurl -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" ...\n```\n\n## Endpoint\n\n```\nPOST https://realtime.oxylabs.io/v1/queries   # immediate response\nPOST https://data.oxylabs.io/v1/queries       # Push-Pull jobs, callbacks, storage\nContent-Type: application/json\n```\n\n## Core Parameters\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `source` | Yes | Target scraper (e.g., `universal`, `amazon_product`, `google_search`) |\n| `url` | Conditional | URL to scrape (for `universal` and `*_url` sources) |\n| `query` | Conditional | Search query or product ID (for `*_search` and `*_product` sources) |\n| `parse` | No | Enable structured data parsing (recommended for supported sources) |\n| `render` | No | JavaScript rendering: `html` or `png` |\n| `geo_location` | No | Geographic targeting: country/state/city, ZIP/postcode, coordinates, or Criteria ID where supported |\n| `session_id` | No | Reuse the same proxy IP across multiple jobs |\n| `content_encoding` | No | Set to `base64` when downloading image files via Realtime or Push-Pull |\n| `user_agent_type` | No | Device/browser preset, e.g., `desktop_chrome`, `mobile_ios`, `tablet_android` |\n| `locale` | No | Interface language / `Accept-Language`, e.g., `de-DE` |\n| `callback_url` | No | Push-Pull callback endpoint |\n| `storage_type`, `storage_url` | No | Push-Pull cloud upload target (`gcs`, `s3`, `tos`, `s3_compatible`) |\n| `markdown`, `xhr` | No | Enable markdown or captured XHR result types |\n| `browser_instructions` | No | Rendered browser actions; requires `render: \"html\"` |\n| `parsing_instructions`, `parser_preset` | No | Custom parser rules or saved preset; pair with `parse: true` |\n| `client_notes` | No | Client-side job tag saved with the job metadata |\n| `domain`, `subdomain`, `start_page`, `pages`, `limit`, `store_id`, `delivery_zip`, `fulfillment_type` | Source-specific | Marketplace/search/store localization and pagination fields |\n\n`user_agent_type` values: `desktop`, `desktop_chrome`, `desktop_edge`, `desktop_firefox`, `desktop_opera`, `desktop_safari`, `mobile`, `mobile_android`, `mobile_ios`, `tablet`, `tablet_android`, `tablet_ios`.\n\n## Context Parameters\n\nAdd these as `{ \"key\": \"...\", \"value\": ... }` objects in `context`:\n\n| Key | Use |\n|-----|-----|\n| `force_headers`, `headers` | Merge custom headers with managed headers |\n| `force_cookies`, `cookies` | Merge custom cookies with managed cookies |\n| `http_method`, `content` | Use `post` with Base64-encoded body content |\n| `follow_redirects` | Follow 3xx redirect chains |\n| `successful_status_codes` | Treat specific non-standard HTTP codes as successful |\n\nFor multi-format output, enable types in the payload (`parse`, `markdown`, `xhr`, `render: \"png\"`) and request them with `?type=raw,parsed,png,markdown,xhr`.\n\nFor batch Push-Pull jobs, use `POST /v1/queries/batch` with arrays only for `query` or `url`; keep all other parameters singular. Maximum batch size is 5,000 values.\n\n## Quick Start\n\n**Scrape any URL:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"universal\", \"url\": \"https://example.com\"}'\n```\n\n**Google search with parsing:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"google_search\", \"query\": \"best laptops\", \"parse\": true}'\n```\n\n**Amazon product by ASIN:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"amazon_product\", \"query\": \"B07FZ8S74R\", \"parse\": true}'\n```\n\n## Choosing the Right Source\n\n1. **Use specific sources when available** (`amazon_product`, `google_search`) - better parsing and reliability\n2. **Use `universal` for unsupported sites** - works with any URL\n3. **Enable `parse: true`** for structured JSON output on supported sources\n\n## Response Structure\n\n```json\n{\n  \"results\": [{\n    \"content\": \"...\",\n    \"status_code\": 200,\n    \"url\": \"https://...\"\n  }]\n}\n```\n\nWith `parse: true`, `content` contains structured data (title, price, reviews, etc.) instead of raw HTML.\n\n## Available Sources\n\nFor the complete list of 40+ supported sources organized by category, see [sources.md](sources.md).\n\n## More Examples\n\nFor detailed request/response examples including geo-location, JavaScript rendering, and custom headers, see [examples.md](examples.md).\n\n## Error Handling\n\n| Code | Meaning |\n|------|---------|\n| 200 | Success |\n| 400 | Invalid parameters |\n| 401 | Authentication failed |\n| 403 | Access denied |\n| 429 | Rate limit exceeded |\n\n## Key Guidelines\n\n- Always set `parse: true` for supported sources to get structured data\n- Use ZIP codes for US e-commerce geo-location (e.g., `\"90210\"`)\n- Use country/state format for search engines (e.g., `\"California,United States\"`)\n- Add `render: \"html\"` for JavaScript-heavy pages\n- Use `render: \"\"` only to disable automatic forced rendering for force-rendered pages; set client timeouts near 180 seconds for rendered Realtime or Proxy Endpoint requests\n- Add `content_encoding: \"base64\"` when scraping image URLs, then decode `results[0].content` before saving the file","schemaVersion":1},"repoUrl":"https://github.com/oxylabs/agent-skills/tree/main/skills/web-scraper-api","tags":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking","prompt"],"stats":{"installVelocity7d":0,"retentionRate":0,"executions":0,"rating":null},"origin":"github","source":{"repo":"agent-skills","audit":{"files":[],"binaries":[],"findings":[],"packages":0,"auditedAt":"2026-09-25T13:51:47.670Z","lockfiles":[]},"forks":2,"owner":"oxylabs","stars":874,"topics":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking"],"license":"MIT","fullName":"oxylabs/agent-skills","homepage":null,"language":"JavaScript","pushedAt":"2026-09-24T12:18:41Z","avatarUrl":"https://avatars.githubusercontent.com/u/43960873?v=4","crawledAt":"2026-09-25T13:51:45.915Z","openIssues":1,"manifestFile":"SKILL.md","manifestPath":"skills/web-scraper-api/SKILL.md","defaultBranch":"main"},"readme":"# Oxylabs Web Scraper API\n\n## Authentication\n\nRequires HTTP Basic Auth with credentials from environment variables:\n\n```bash\ncurl -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" ...\n```\n\n## Endpoint\n\n```\nPOST https://realtime.oxylabs.io/v1/queries   # immediate response\nPOST https://data.oxylabs.io/v1/queries       # Push-Pull jobs, callbacks, storage\nContent-Type: application/json\n```\n\n## Core Parameters\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `source` | Yes | Target scraper (e.g., `universal`, `amazon_product`, `google_search`) |\n| `url` | Conditional | URL to scrape (for `universal` and `*_url` sources) |\n| `query` | Conditional | Search query or product ID (for `*_search` and `*_product` sources) |\n| `parse` | No | Enable structured data parsing (recommended for supported sources) |\n| `render` | No | JavaScript rendering: `html` or `png` |\n| `geo_location` | No | Geographic targeting: country/state/city, ZIP/postcode, coordinates, or Criteria ID where supported |\n| `session_id` | No | Reuse the same proxy IP across multiple jobs |\n| `content_encoding` | No | Set to `base64` when downloading image files via Realtime or Push-Pull |\n| `user_agent_type` | No | Device/browser preset, e.g., `desktop_chrome`, `mobile_ios`, `tablet_android` |\n| `locale` | No | Interface language / `Accept-Language`, e.g., `de-DE` |\n| `callback_url` | No | Push-Pull callback endpoint |\n| `storage_type`, `storage_url` | No | Push-Pull cloud upload target (`gcs`, `s3`, `tos`, `s3_compatible`) |\n| `markdown`, `xhr` | No | Enable markdown or captured XHR result types |\n| `browser_instructions` | No | Rendered browser actions; requires `render: \"html\"` |\n| `parsing_instructions`, `parser_preset` | No | Custom parser rules or saved preset; pair with `parse: true` |\n| `client_notes` | No | Client-side job tag saved with the job metadata |\n| `domain`, `subdomain`, `start_page`, `pages`, `limit`, `store_id`, `delivery_zip`, `fulfillment_type` | Source-specific | Marketplace/search/store localization and pagination fields |\n\n`user_agent_type` values: `desktop`, `desktop_chrome`, `desktop_edge`, `desktop_firefox`, `desktop_opera`, `desktop_safari`, `mobile`, `mobile_android`, `mobile_ios`, `tablet`, `tablet_android`, `tablet_ios`.\n\n## Context Parameters\n\nAdd these as `{ \"key\": \"...\", \"value\": ... }` objects in `context`:\n\n| Key | Use |\n|-----|-----|\n| `force_headers`, `headers` | Merge custom headers with managed headers |\n| `force_cookies`, `cookies` | Merge custom cookies with managed cookies |\n| `http_method`, `content` | Use `post` with Base64-encoded body content |\n| `follow_redirects` | Follow 3xx redirect chains |\n| `successful_status_codes` | Treat specific non-standard HTTP codes as successful |\n\nFor multi-format output, enable types in the payload (`parse`, `markdown`, `xhr`, `render: \"png\"`) and request them with `?type=raw,parsed,png,markdown,xhr`.\n\nFor batch Push-Pull jobs, use `POST /v1/queries/batch` with arrays only for `query` or `url`; keep all other parameters singular. Maximum batch size is 5,000 values.\n\n## Quick Start\n\n**Scrape any URL:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"universal\", \"url\": \"https://example.com\"}'\n```\n\n**Google search with parsing:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"google_search\", \"query\": \"best laptops\", \"parse\": true}'\n```\n\n**Amazon product by ASIN:**\n```bash\ncurl -X POST 'https://realtime.oxylabs.io/v1/queries' \\\n  -u \"$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"source\": \"amazon_product\", \"query\": \"B07FZ8S74R\", \"parse\": true}'\n```\n\n## Choosing the Right Source\n\n1. **Use specific sources when available** (`amazon_product`, `google_search`) - better parsing and reliability\n2. **Use","createdAt":"2026-09-25T13:51:47.723Z","updatedAt":"2026-09-25T13:51:47.723Z"},{"id":"cmuh0rx1x03l4qu068i5m5554","slug":"oxylabs-agent-skills-web-unblocker","name":"web-unblocker","description":"Bypasses anti-bot protections using Oxylabs Web Unblocker, an AI-powered proxy that handles fingerprinting, JavaScript rendering, and retries automatically. Use when the user needs to scrape protected websites, bypass CAPTCHAs, access blocked content, or when regular proxies fail due to anti-bot measures.","authorId":"gh:oxylabs","authorName":"oxylabs","version":"0.1.0","category":"Prompt","securityLevel":"Community","downloadsCount":0,"githubStars":874,"pricePerCall":0,"manifest":{"name":"web-unblocker","tools":[],"category":"Prompt","entrypoint":{"type":"prompt"},"description":"Bypasses anti-bot protections using Oxylabs Web Unblocker, an AI-powered proxy that handles fingerprinting, JavaScript rendering, and retries automatically. Use when the user needs to scrape protected websites, bypass CAPTCHAs, access blocked content, or when regular proxies fail due to anti-bot measures.","permissions":[],"systemPrompt":"# Oxylabs Web Unblocker\n\nAI-powered proxy solution that automatically manages fingerprinting, headers, retries, and JavaScript rendering.\n\n## Endpoint\n\n```\nhttps://unblock.oxylabs.io:60000\n```\n\n## Authentication\n\nHTTP Basic Auth via proxy credentials:\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  \"https://example.com\"\n```\n\n## Quick Start\n\n**Basic request:**\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  \"https://ip.oxylabs.io/headers\"\n```\n\n**With JavaScript rendering:**\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"x-oxylabs-render: html\" \\\n  \"https://example.com/spa-page\"\n```\n\n## Headers\n\n| Header | Description |\n|--------|-------------|\n| `x-oxylabs-render` | `html` for rendered HTML, `png` for raw PNG bytes; empty value disables automatic forced rendering |\n| `X-Oxylabs-Session-Id` | Reuse same IP across requests (any random string) |\n| `X-Oxylabs-Geo-Location` | Target country, city/state, ZIP/postcode, or coordinates |\n| `x-oxylabs-force-headers: 1` | Enable custom header passthrough |\n| `x-oxylabs-force-cookies: 1` | Enable custom cookie passthrough |\n| `X-Oxylabs-Successful-Status-Codes` | Define custom success codes to prevent retries |\n| `x-oxylabs-browser-instructions` | JSON-escaped browser actions; requires `x-oxylabs-render: html` |\n\n## Session Persistence\n\nReuse the same IP across multiple requests:\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"X-Oxylabs-Session-Id: my-session-123\" \\\n  \"https://example.com/page1\"\n```\n\n## Geo-Location Targeting\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"X-Oxylabs-Geo-Location: Germany\" \\\n  \"https://example.com\"\n```\n\nUse values such as `Germany`, `90210`, `California,United States`, `New York,New York,United States`, or `lat: 40.7128, lng: -74.0060, rad: 50`.\n\nUse normal HTTP methods and request bodies through the proxy; Web Unblocker supports both GET and POST.\n\n## When to Use Web Unblocker vs Regular Proxies\n\n| Scenario | Use |\n|----------|-----|\n| Sites with anti-bot protection | Web Unblocker |\n| CAPTCHAs, fingerprint detection | Web Unblocker |\n| JavaScript-heavy SPAs | Web Unblocker with `x-oxylabs-render: html` |\n| Simple requests, no protection | Regular Proxies |\n| High volume, price sensitive | Regular Proxies |\n\n## Key Guidelines\n\n- Always use `-k` flag (or disable SSL verification) - the proxy uses its own certificates\n- Add `x-oxylabs-render: html` if experiencing empty content or low success rates; set client timeouts near 180 seconds for rendered requests\n- Check `X-Oxylabs-Final-Url` in response headers when redirects matter\n- Avoid adding custom unblocking headers that may interfere with the AI\n- Browser instruction header values must be JSON-escaped and compact; pair them with `x-oxylabs-render: html`\n\nFor code examples in Python, Node.js, PHP, Go, Java, and C#, see [examples.md](examples.md).","schemaVersion":1},"repoUrl":"https://github.com/oxylabs/agent-skills/tree/main/skills/web-unblocker","tags":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking","prompt"],"stats":{"installVelocity7d":0,"retentionRate":0,"executions":0,"rating":null},"origin":"github","source":{"repo":"agent-skills","audit":{"files":[],"binaries":[],"findings":[],"packages":0,"auditedAt":"2026-09-25T13:51:47.670Z","lockfiles":[]},"forks":2,"owner":"oxylabs","stars":874,"topics":["agent-skill","agent-skills","ai-agent","ai-agents","claude-code","claude-skills","proxy","video-data","web-scraping","web-unblocking"],"license":"MIT","fullName":"oxylabs/agent-skills","homepage":null,"language":"JavaScript","pushedAt":"2026-09-24T12:18:41Z","avatarUrl":"https://avatars.githubusercontent.com/u/43960873?v=4","crawledAt":"2026-09-25T13:51:45.915Z","openIssues":1,"manifestFile":"SKILL.md","manifestPath":"skills/web-unblocker/SKILL.md","defaultBranch":"main"},"readme":"# Oxylabs Web Unblocker\n\nAI-powered proxy solution that automatically manages fingerprinting, headers, retries, and JavaScript rendering.\n\n## Endpoint\n\n```\nhttps://unblock.oxylabs.io:60000\n```\n\n## Authentication\n\nHTTP Basic Auth via proxy credentials:\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  \"https://example.com\"\n```\n\n## Quick Start\n\n**Basic request:**\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  \"https://ip.oxylabs.io/headers\"\n```\n\n**With JavaScript rendering:**\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"x-oxylabs-render: html\" \\\n  \"https://example.com/spa-page\"\n```\n\n## Headers\n\n| Header | Description |\n|--------|-------------|\n| `x-oxylabs-render` | `html` for rendered HTML, `png` for raw PNG bytes; empty value disables automatic forced rendering |\n| `X-Oxylabs-Session-Id` | Reuse same IP across requests (any random string) |\n| `X-Oxylabs-Geo-Location` | Target country, city/state, ZIP/postcode, or coordinates |\n| `x-oxylabs-force-headers: 1` | Enable custom header passthrough |\n| `x-oxylabs-force-cookies: 1` | Enable custom cookie passthrough |\n| `X-Oxylabs-Successful-Status-Codes` | Define custom success codes to prevent retries |\n| `x-oxylabs-browser-instructions` | JSON-escaped browser actions; requires `x-oxylabs-render: html` |\n\n## Session Persistence\n\nReuse the same IP across multiple requests:\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"X-Oxylabs-Session-Id: my-session-123\" \\\n  \"https://example.com/page1\"\n```\n\n## Geo-Location Targeting\n\n```bash\ncurl -k -x \"https://unblock.oxylabs.io:60000\" \\\n  -U \"$OXYLABS_USERNAME:$OXYLABS_PASSWORD\" \\\n  -H \"X-Oxylabs-Geo-Location: Germany\" \\\n  \"https://example.com\"\n```\n\nUse values such as `Germany`, `90210`, `California,United States`, `New York,New York,United States`, or `lat: 40.7128, lng: -74.0060, rad: 50`.\n\nUse normal HTTP methods and request bodies through the proxy; Web Unblocker supports both GET and POST.\n\n## When to Use Web Unblocker vs Regular Proxies\n\n| Scenario | Use |\n|----------|-----|\n| Sites with anti-bot protection | Web Unblocker |\n| CAPTCHAs, fingerprint detection | Web Unblocker |\n| JavaScript-heavy SPAs | Web Unblocker with `x-oxylabs-render: html` |\n| Simple requests, no protection | Regular Proxies |\n| High volume, price sensitive | Regular Proxies |\n\n## Key Guidelines\n\n- Always use `-k` flag (or disable SSL verification) - the proxy uses its own certificates\n- Add `x-oxylabs-render: html` if experiencing empty content or low success rates; set client timeouts near 180 seconds for rendered requests\n- Check `X-Oxylabs-Final-Url` in response headers when redirects matter\n- Avoid adding custom unblocking headers that may interfere with the AI\n- Browser instruction header values must be JSON-escaped and compact; pair them with `x-oxylabs-render: html`\n\nFor code examples in Python, Node.js, PHP, Go, Java, and C#, see [examples.md](examples.md).","createdAt":"2026-09-25T13:51:47.733Z","updatedAt":"2026-09-25T13:51:47.733Z"}],"total":5,"limit":24,"offset":0}