KIDS COUNT Child Welfare Data Scraper

Extract child well-being indicator data from the Annie E. Casey Foundation KIDS COUNT Data Center. Every state, hundreds of indicators, going back decades.

KIDS COUNT is the canonical US child-wellbeing benchmark — cited by state agencies, foundations, journalists, and researchers. The database covers indicators like child poverty, children in foster care, single-parent families, and teen birth rates across all 50 states, DC, and the national level. No login required. No paywall.

What You Get

Each record is one value for one indicator, one location, and one year.

Field	Description
indicator_id	Numeric indicator ID (e.g. 106 = Children in single-parent families)
indicator_name	Full indicator name
location	Location name (United States, Alabama, etc.)
location_type	national or state
year	Data year
data_format	Number or Percent
value	Numeric value
value_display	Formatted display string (e.g. 34% or 20,748,000)
data_source	Source citation from KIDS COUNT
table_url	URL of the indicator table page

Input

Parameter	Type	Default	Description
indicatorIds	string[]	(all)	Specific indicator IDs to scrape. Leave empty for all 4,000+ indicators from the sitemap.
locationTypes	string[]	["national", "state"]	Which location types to include: national, state, or both.
maxItems	integer	15	Maximum number of records to collect.

Example inputs

Scrape all years for the single-parent families indicator across all US states:

{
  "indicatorIds": ["106"],
  "locationTypes": ["national", "state"],
  "maxItems": 1000
}

Scrape the top five child welfare indicators:

{
  "indicatorIds": ["106", "12", "7", "8", "9"],
  "locationTypes": ["state"],
  "maxItems": 5000
}

Leave indicatorIds empty to crawl the full database. With ~4,000 indicators x ~52 locations x ~25 years of data, that is a lot of records.

How It Works

The site has a JSON API endpoint (/api/reports/detailedtable) that is not protected by Cloudflare. The scraper:

1. Reads the sitemap to enumerate all indicator URLs (4,000+), or uses the indicatorIds you provide.
2. Fetches each indicator's HTML page to extract available time-frames and data sources.
3. Calls the API with all state IDs in a single batch request per indicator to get the full location x year matrix.
4. Parses the returned HTML table and saves each data point as a flat record.

No proxy needed. The API responds cleanly to datacenter requests.

Use Cases

• Child welfare research — track foster care rates, child poverty, and family stability across states and years.
• Policy analysis — benchmark a state against national averages or compare regions.
• Journalism — pull the data behind the KIDS COUNT Data Book in machine-readable form.
• Grant reporting — support program evaluation against KIDS COUNT baseline indicators.
• Data science — train models on child well-being trends using a clean, authoritative dataset.

Notes

• The rawdata.axd download endpoint is Cloudflare-protected and is not used. This scraper reads the same data through the undocumented JSON API instead.
• Some indicators have county-level sub-state data; the location_type field is set to state for all loct=2 API responses regardless of geographic granularity.
• Values labeled N.A. (not available) or S (suppressed for small sample size) are skipped.
• Data for some indicators only covers a subset of US states.