Threat Intel IOC Feed Aggregator

Normalizes and cross-feed-dedupes six free, published threat-intelligence IOC feeds into one schema, with a cross-feed corroboration count and a persisted delta cursor for hourly-poll blocklist/enrichment pipelines.

Built for SOC/blue-team, detection-engineering, and threat-intel teams that currently pull each free feed separately in its own raw format and stitch them together by hand.

Passive by design. This actor reads the published feed endpoints only (urlhaus.abuse.ch, feodotracker.abuse.ch, threatfox.abuse.ch, bazaar.abuse.ch, openphish.com, data.phishtank.com). It never connects to, resolves, or scans any of the malicious infrastructure listed inside those feeds — the URLs/IPs/hashes being cataloged are parsed as inert text only.

---

Covered Feeds

Source	Feed	IOC Type	Auth Required
urlhaus	abuse.ch URLhaus — recent malware-distribution URLs	url	None
feodo	abuse.ch Feodo Tracker — active botnet C2 IPs	ip_port	None
threatfox	abuse.ch ThreatFox — recent multi-type IOCs	url / domain / ip_port / hashes	None
malwarebazaar	abuse.ch MalwareBazaar — recent malware sample hashes	sha256_hash	None
openphish	OpenPhish — public phishing URL feed	url	None
phishtank	PhishTank — verified online phishing URLs	url	None

All six feeds are free, published downloads. None require an account or API key for the endpoints this actor uses.

---

Input

Field	Type	Default	Description
sources	array	all six	Subset of feeds to fetch and merge: urlhaus, feodo, threatfox, malwarebazaar, openphish, phishtank
maxItems	integer	15	Maximum normalized IOC records to return this run, across all selected feeds combined
resetState	boolean	false	Force this run to treat all currently-listed IOCs as new, ignoring the persisted delta cursor. Use once for a full baseline pull

Delta mode (default behavior)

Every run persists the set of IOC keys it has already emitted (in the actor's key-value store, pruned after 30 days). On each subsequent run, only IOCs not previously emitted are returned — so a scheduled hourly run naturally becomes a "what's new since last time" feed instead of re-delivering the same records. If a run is capped by maxItems before covering all new IOCs, the remainder carries forward and surfaces on the next run — nothing is dropped, just paginated across runs.

Set resetState: true for a one-time full snapshot of everything currently in the selected feeds (e.g. for an initial bulk load), then leave it false for ongoing polling.

---

Output

Each record contains:

Field	Type	Description
ioc_value	string	The indicator itself — URL, domain, ip:port, or file hash, depending on ioc_type
ioc_type	string	url | domain | ip_port | md5_hash | sha256_hash | sha1_hash
threat_type	string	malware_download | botnet_c2 | phishing | payload_delivery
malware_family	string	Malware family / signature name, when known
source_feed	string	Primary reporting feed for this IOC
confidence	integer	Normalized 0-100 confidence score
first_seen	string	Timestamp first reported (earliest across corroborating feeds)
last_seen	string	Timestamp last confirmed active (latest across corroborating feeds)
reference_url	string	Link to the source feed's report page for this indicator
tags	array	Merged tags/labels from all corroborating feeds
reporter	string	Reporting analyst/organization, when the source feed provides one
is_online	boolean	Whether the indicator was reported as currently active/online
dedup_source_count	integer	Number of distinct feeds that independently reported this same indicator
scraped_at	string	ISO timestamp this run fetched the source feeds

A dedup_source_count of 2+ means multiple independent feeds corroborated the same indicator — useful as a confidence signal on top of each feed's own scoring.

---

Example Inputs

Hourly-poll delta (recommended for scheduled runs):

{
  "maxItems": 500,
  "sp_intended_usage": "Hourly blocklist enrichment feed",
  "sp_improvement_suggestions": "None"
}

One-time full baseline of just the malware-hash feeds:

{
  "sources": ["malwarebazaar", "threatfox"],
  "maxItems": 5000,
  "resetState": true,
  "sp_intended_usage": "Initial IOC database seed",
  "sp_improvement_suggestions": "None"
}

Phishing-only feed for a browser-extension blocklist:

{
  "sources": ["openphish", "phishtank"],
  "maxItems": 1000,
  "sp_intended_usage": "Phishing URL blocklist",
  "sp_improvement_suggestions": "None"
}

---

Data Sources

All data is sourced from free, published threat-intelligence feeds:

• abuse.ch — URLhaus, Feodo Tracker, ThreatFox, MalwareBazaar (CC0 / free public feeds)
• OpenPhish — public phishing feed
• PhishTank — Cisco Talos community phishing verification

This actor performs no scanning, resolution, or connection to any indicator it catalogs — it only downloads and normalizes the published feed files.