ClawHub Skill Scraper
ClawHub Skill Scraper
Scrape AI agent skill data from the ClawHub marketplace. Extract skill definitions (SKILL.md), metadata, download stats, install counts, star ratings, changelogs, and author information for 8,000+ skills in the OpenClaw ecosystem.
ClawHub Skill Scraper Features
- Extracts full SKILL.md content from each skill's ClawHub package bundle
- Collects marketplace stats: downloads, installs, stars, comments, version count
- Two modes: metadata-only (fast) or full content with SKILL.md files
- Handles 8,000+ skills with automatic pagination and rate limiting
- Merges data from the ClawHub listing API and the ClawHub package download
- Outputs structured JSON with 20+ fields per skill record
Who Uses ClawHub Skill Data?
- AI tool directories — Index and categorize AI agent skills for discovery platforms
- Developer analytics — Track skill adoption trends, download velocity, and popularity across the OpenClaw ecosystem
- Competitive research — Monitor new skill releases, author activity, and marketplace growth
- Dataset builders — Collect SKILL.md definitions to train or evaluate AI agents on real-world skill specifications
- Market research — Analyze the AI agent skill marketplace for investment or product positioning
How the ClawHub Scraper Works
- List skills — Paginates through the ClawHub listing API to collect skill metadata and marketplace stats
- Resolve authors — Owner handles are already present on each listing record
- Download SKILL.md — Pulls the skill's ZIP bundle from ClawHub's own package download endpoint and extracts the full skill definition and package metadata from it
Steps 2 and 3 only run when downloadContent is enabled. With it disabled, the scraper returns metadata only and runs significantly faster.
Input
{
"maxItems": 50,
"downloadContent": true
}
| Field | Type | Default | Description |
|---|---|---|---|
| maxItems | integer | 50 | Maximum number of skills to scrape. Set to 0 for unlimited. |
| downloadContent | boolean | true | Fetch SKILL.md content from ClawHub's package ZIP bundle. When disabled, only ClawHub API metadata is collected (faster). |
ClawHub Scraper Output Fields
{
"slug": "web-search",
"displayName": "Web Search",
"version": "1.2.0",
"summary": "Search the web and return structured results",
"author": "openclaw",
"tags": { "latest": "1.2.0", "stable": "1.1.5" },
"downloads": 12450,
"installsAllTime": 8320,
"installsCurrent": 1540,
"stars": 89,
"comments": 12,
"versions": 8,
"changelog": "Added timeout configuration and retry logic",
"skillMd": "# Web Search\n\nSearch the web using...",
"supportingFiles": [],
"ownerId": "usr_abc123",
"createdAt": "2024-06-15T10:30:00Z",
"updatedAt": "2025-02-20T14:15:00Z",
"publishedAt": "2025-02-20T14:15:00Z"
}
| Field | Type | Description |
|---|---|---|
| slug | string | Unique skill identifier (URL slug) |
| displayName | string | Human-readable skill name |
| version | string | Latest published version |
| summary | string | Short skill description |
| author | string | Skill author from skill.yaml metadata |
| tags | object | Tag-to-version mapping (e.g. latest, stable) |
| downloads | number | Total download count |
| installsAllTime | number | Lifetime install count |
| installsCurrent | number | Current active installs |
| stars | number | Star/favorite count |
| comments | number | Comment count |
| versions | number | Total published version count |
| changelog | string | Latest version changelog text |
| skillMd | string | Full SKILL.md file content |
| supportingFiles | array | Additional files in the skill package |
| ownerId | string | Internal owner identifier |
| createdAt | string | Skill creation timestamp (ISO 8601) |
| updatedAt | string | Last update timestamp (ISO 8601) |
| publishedAt | string | Latest version publish timestamp (ISO 8601) |
FAQ
How do I scrape all skills from ClawHub?
Set maxItems to 0 and downloadContent to true. The ClawHub Skill Scraper will paginate through the full marketplace and download SKILL.md content for every skill.
How much does the ClawHub Skill Scraper cost to run?
A full run with content download for all 8,000+ skills uses approximately 256 MB of memory and takes under an hour. Metadata-only mode is significantly faster and cheaper.
What is a SKILL.md file?
SKILL.md is the standard definition file for AI agent skills in the OpenClaw ecosystem. It describes what the skill does, its inputs, outputs, and configuration. The ClawHub Skill Scraper extracts these files directly from each skill's ClawHub package bundle.
Can I scrape ClawHub without downloading SKILL.md content?
Yes. Set downloadContent to false to collect only marketplace metadata (names, stats, tags) without fetching the package bundle. This mode is much faster and skips the download phase entirely.
Need More Features?
Need custom fields, filters, or a scraper for a different AI marketplace? File an issue or get in touch.
Why Use the ClawHub Skill Scraper?
- Complete — Combines ClawHub marketplace data with full SKILL.md content from the package bundle in a single run
- Flexible — Metadata-only mode for fast runs, full mode for complete skill definitions
- Reliable — Rate limiting, retry logic, and graceful error handling for 8,000+ skill records
Further reading: Track Your Competitors Across 7 App Stores From One Toolkit
