Free
AI OverviewGenerated from SKILL.md
Core usage
The web-scraper-toolkit ingests a list of URLs or a sitemap.xml, extracts structured fields using CSS/XPath selectors, and exports the results to CSV or JSON. It handles pagination, retries with backoff, respects robots.txt, and supports login-walled pages via a session cookie. To use it, provide the target URLs and field selectors, run the scraper, preview sample rows, and then export the cleaned data.
Key advantages
It simplifies common scraping challenges by automating pagination, retry logic, and robots.txt compliance. The session cookie support allows access to login-walled content, and the structured CSV/JSON export makes downstream analysis easy. It also produces a run report showing success/fail per URL, which is useful for auditing scrapes.
Limitations
Users must manually define CSS/XPath selectors for each field, so setup requires some technical knowledge. The toolkit does not describe handling of JavaScript-rendered pages or advanced anti-bot measures beyond session cookies. Output is limited to CSV and JSON formats, and the run report only records per-URL success or failure without additional diagnostics.
Target audience
This is for developers, data analysts, and researchers who need to turn a known set of URLs into clean, structured data. It is especially suited for those handling paginated or login-walled content who want a straightforward pipeline to CSV/JSON without building a scraper from scratch.
Risks & notes
Scraping can still raise legal or ethical concerns even when robots.txt is respected, so users should verify site terms and permissions. Session cookies must be handled securely and not exposed in logs or shared outputs. Failed URLs are logged but may require manual review, and incorrect selectors can produce incomplete or inaccurate data if sample rows are not checked carefully.
DescriptionWritten by the seller
Use caseInput → Output
Input
URL list + field selectors -> clean CSV/JSON dataset + run report
Output
URL list + field selectors -> clean CSV/JSON dataset + run report
How it worksImport to your platform and run
1
Ingest URLsTrigger
Reads URL list or sitemap.xml
2
Scrape with retriesAI action
Fetches pages with backoff and robots check
3
Export datasetOutput
Writes data.csv and run-report.md
Item details
Use case
AI Skills
Capability
Automation
Agents
Codex, OpenCode
Version
0.9.1
Source
First seen
Aug 14, 2026
Installs
1,280
License
Non-exclusive
Last update
Aug 14, 2026
InstallCompatible with agent skill ecosystem
npx skills add https://github.com/anthropics/skills --skill web-scraper-toolkit
What you get
Verified sourceOriginal project page — inspect before install
Open the original source project
Check the upstream repository, license and changelog before install
Security ScanS+Score 100/100 · 0 findings · 2026-08-15 21:31:03
Static analysis100
Dynamic behavior100
Dependencies100
Network100
Privacy100
Threat intel100
✓ 未发现安全问题，六维检测全部通过。
Security scan generated by TopTinker automated engine from SKILL.md static analysis, for pre-install reference only. Third-party skills carry their own runtime risk.
Guarantees
Escrow paymentOn
Platform holds funds until delivery is confirmed
Install verified
Free download - install & run before you pay
Source transparency
Original GitHub repo linked & verified
No refunds after download
Digital product - disputes arbitrated on evidence only
Enhanced protection
Trusted seller - priority arbitration within 48h, backed by platform bond
Upgrade to ProRecommended
More in AI Skills