OpenAlex Scholarly Works, Authors & Citations Crawler

Extract structured scholarly data from OpenAlex, the open successor to Microsoft Academic Graph. The API covers 309M+ works, 100M+ authors, 120K+ institutions, and 65K+ concepts — titles, DOIs, authors with ORCIDs, citation counts, open-access status, grants, and full abstracts reconstructed from the inverted index most scrapers quietly skip.

OpenAlex Crawler Features

• Queries six entity types in one actor: Works, Authors, Institutions, Concepts, Sources (journals), and Publishers
• Filters works by publication year range, work type, concept ID, institution country, venue ISSN, and open-access status — combine any of them
• Reconstructs full-text abstracts from OpenAlex's inverted word-position index, a step most competing actors leave out
• Fetches 200 records per API call with cursor pagination, so pulling 10,000 records takes 50 requests, not 500
• Extracts 30+ fields per work including authors with ORCIDs, all contributing institutions, top concepts with scores, grants, referenced works, and SDG tags
• Qualifies for the OpenAlex polite pool via the politeEmail input — higher rate limits and better latency
• Pure JSON API, no HTML parsing, no proxy, no authentication required

Who Uses OpenAlex Data and Why?

• AI/ML teams training research assistants — bulk-fetch works with abstracts to build citation recommenders, literature review agents, and domain-specific retrieval corpora
• Bibliometric analysts — map citation networks by concept, institution, or country and track publication output over time
• Research intelligence products — feed competitive analysis dashboards that track which labs publish what and who funds it
• Academic librarians — build institutional publication lists filtered by open-access status and funding source
• Science policy researchers — measure research alignment with the UN Sustainable Development Goals across countries and years

How OpenAlex Crawler Works

1. Pick an entity type — Works is the default, but Authors, Institutions, Concepts, Sources, and Publishers all work the same way.
2. Add filters. For Works that means year range, concept, country, venue ISSN, and the open-access flag. Other entities accept the filters that apply to them.
3. The crawler hits the OpenAlex API with cursor pagination, pulling up to 200 records per request until it hits your maxItems limit or exhausts the query.
4. Each record is flattened into a consistent schema — arrays stay as arrays of primitives, concepts and grants get formatted into readable strings, abstracts are rebuilt from the inverted index when you want them.

Input

Basic: recent machine learning papers

{
    "entityType": "works",
    "query": "machine learning",
    "yearFrom": 2023,
    "yearTo": 2024,
    "maxItems": 500
}

Open-access climate science, concept-filtered

{
    "entityType": "works",
    "concept": "C132651083",
    "openAccessOnly": true,
    "institutionCountry": "US",
    "maxItems": 1000,
    "politeEmail": "you@example.com"
}

Authors search

{
    "entityType": "authors",
    "query": "yoshua bengio",
    "maxItems": 20
}

Institutions in a country

{
    "entityType": "institutions",
    "institutionCountry": "DE",
    "maxItems": 100
}

Input Parameters

Field	Type	Default	Description
entityType	string	works	One of works, authors, institutions, concepts, sources, publishers.
query	string	machine learning	Full-text search across titles and abstracts (works) or display names (other entities). Leave empty to browse all.
yearFrom	integer	0	Earliest publication year (Works only). 0 means no lower bound.
yearTo	integer	0	Latest publication year (Works only). 0 means no upper bound.
openAccessOnly	boolean	false	Restrict Works to open-access publications.
workType	string	""	Filter by work type: article, preprint, book, book-chapter, dataset, dissertation, review, report, standard, other.
concept	string	""	OpenAlex concept ID (e.g. C41008148 for Computer Science). Applies to Works and Authors.
institutionCountry	string	""	Two-letter ISO country code. Applies to Works (via authorships), Authors, and Institutions.
venueIssn	string	""	Filter Works by host venue ISSN (e.g. 0028-0836 for Nature).
reconstructAbstract	boolean	true	Rebuild full abstracts from the OpenAlex inverted index. Adds a small amount of per-record work.
politeEmail	string	""	Your email to qualify for the OpenAlex polite pool. Recommended for any real run.
maxItems	integer	100	Maximum records to return. Set to 0 for unlimited — requires at least one filter or search query.
proxyConfiguration	object	disabled	Proxy settings. OpenAlex does not require proxies.

OpenAlex Crawler Output Fields

All entity types share a common output schema. Fields that don't apply to a given entity type are left empty. The examples below show Works and Authors — Institutions, Concepts, Sources, and Publishers use the same table.

Works output example

{
    "openalex_id": "W2101234009",
    "doi": "https://doi.org/10.5555/1953048.2078195",
    "title": "Scikit-learn: Machine Learning in Python",
    "abstract": "Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems...",
    "entity_type": "work",
    "publication_year": 2011,
    "publication_date": "2011-10-01",
    "work_type": "article",
    "language": "en",
    "open_access_is_oa": true,
    "open_access_status": "green",
    "open_access_oa_url": "https://hal.inria.fr/hal-00650905v2/document",
    "venue_name": "Journal of Machine Learning Research",
    "venue_issn": "1532-4435, 1533-7928",
    "venue_publisher": "JMLR",
    "author_names": ["Fabian Pedregosa", "Gaël Varoquaux", "Alexandre Gramfort"],
    "author_ids": ["A5014316393", "A5047509574", "A5001829085"],
    "author_orcids": ["https://orcid.org/0000-0003-4025-383X"],
    "corresponding_author_name": "Fabian Pedregosa",
    "first_institution_name": "Inria",
    "first_institution_country": "FR",
    "institution_names": ["Inria", "CEA", "ENS Paris"],
    "institution_ids": ["I1294671590", "I4210121523"],
    "concepts": [
        "Computer science (L0, 0.820)",
        "Machine learning (L1, 0.750)",
        "Python (programming language) (L3, 0.610)"
    ],
    "cited_by_count": 58421,
    "referenced_works_count": 18,
    "referenced_works": ["W2109394935", "W2131462250"],
    "related_works": ["W2963773651"],
    "counts_by_year": ["2023: 8421", "2022: 7310", "2021: 6122"],
    "grants": ["French National Research Agency — ANR-12-BS01-0009"],
    "sustainable_development_goals": [],
    "openalex_url": "https://openalex.org/W2101234009"
}

Authors output example

{
    "openalex_id": "A5083138872",
    "title": "Albert Einstein",
    "entity_type": "author",
    "author_orcids": [],
    "institution_names": ["Princeton University", "Institute for Advanced Study"],
    "concepts": [
        "Physics (L0, 0.910)",
        "Quantum mechanics (L1, 0.720)"
    ],
    "cited_by_count": 184521,
    "works_count": 412,
    "counts_by_year": ["2024: 1502", "2023: 1610"],
    "openalex_url": "https://openalex.org/A5083138872"
}

Field	Type	Description
openalex_id	string	Short OpenAlex identifier (W*, A*, I*, C*, S*, P*).
entity_type	string	work, author, institution, concept, source, or publisher.
title	string	Work title or entity display name.
doi	string	DOI URL (Works only).
abstract	string	Full abstract reconstructed from OpenAlex's inverted index (Works only, when reconstructAbstract is enabled).
publication_year	number	Publication year (Works only).
publication_date	string	Publication date in YYYY-MM-DD (Works only).
work_type	string	Work type: article, preprint, book, dataset, etc. (Works only).
language	string	Language code (Works only).
open_access_is_oa	boolean	Open-access flag (Works and Sources).
open_access_status	string	gold, green, hybrid, bronze, diamond, closed (Works only).
open_access_oa_url	string	URL to the open-access version (Works only).
venue_name	string	Host venue or journal name (Works only).
venue_issn	string	Host venue ISSN, comma-separated (Works and Sources).
venue_publisher	string	Host venue publisher name (Works and Sources).
author_names	string[]	Author display names.
author_ids	string[]	OpenAlex author IDs.
author_orcids	string[]	Author ORCID URLs when available.
corresponding_author_name	string	Name of the corresponding author (Works only).
first_institution_name	string	Primary institution of the first author (Works only).
first_institution_country	string	Country code of the first author's institution (Works only).
institution_names	string[]	All institutions across all authors (Works, Authors).
institution_ids	string[]	All OpenAlex institution IDs (Works only).
concepts	string[]	Top 5 OpenAlex concepts, formatted as Name (Level, Score).
cited_by_count	number	Total citation count.
referenced_works_count	number	Number of works this work cites (Works only).
referenced_works	string[]	OpenAlex IDs of referenced works (Works only).
related_works	string[]	OpenAlex IDs of related works (Works only).
counts_by_year	string[]	Citations per year, formatted as YYYY: N.
grants	string[]	Funders and award IDs, formatted as Funder — Award.
sustainable_development_goals	string[]	UN SDG names matched to the work.
works_count	number	Number of works (Authors, Institutions, Concepts, Sources, Publishers).
country_code	string	Country code (Institutions, Publishers).
homepage_url	string	Homepage URL (Institutions, Publishers).
concept_level	number	Concept hierarchy level, 0 = most general (Concepts only).
concept_description	string	Concept description text (Concepts only).
openalex_url	string	Full OpenAlex URL for the entity.

FAQ

How many records does OpenAlex Crawler cover? OpenAlex Crawler reads the full OpenAlex index — 309M+ works, 100M+ authors, 120K+ institutions, and 65K+ concepts. If a paper, researcher, or organization is in OpenAlex, the crawler can reach it.

Do I need an API key or proxies? OpenAlex Crawler runs without either. OpenAlex is free and open. Setting politeEmail is optional but recommended — it qualifies runs for the polite pool, which has higher rate limits and more predictable latency than the shared anonymous pool.

What is abstract reconstruction and why does it matter? OpenAlex stores abstracts as an inverted index — a map of words to their positions — to comply with publisher terms. The raw field is not human-readable. OpenAlex Crawler rebuilds the original text from that index, so you get a normal paragraph instead of a JSON object. Most competing actors skip this step.

Can I run a bulk export without filters? Not with maxItems set to 0. Pulling the entire 309M-work index through the API would take a while and would not be what most people actually want. Provide at least one filter or a search query when running unlimited. With filters, unlimited runs are fine — the cursor pagination scales.

How do I filter by a specific concept? OpenAlex Crawler accepts any OpenAlex concept ID in the concept field. Find IDs by browsing openalex.org/concepts or querying the Concepts endpoint in this actor. Common examples: C41008148 (Computer Science), C86803240 (Biology), C185592680 (Chemistry), C121332964 (Physics).

How current is the data? OpenAlex Crawler reads the live OpenAlex API. OpenAlex ingests from Crossref, PubMed, institutional repositories, and other sources on a continuous basis; individual records include publication_date and update timestamps on the underlying records.

Need More Features?

Need extra fields, a different filter, or a scheduled run? Get in touch.

Why Use OpenAlex Crawler?

• Full scholarly graph, one actor — Works, Authors, Institutions, Concepts, Sources, and Publishers all share the same output schema, so you can build pipelines across entity types without juggling separate tools
• Reconstructed abstracts — OpenAlex returns abstracts as an inverted word-position index, which is not text anyone can use directly; this crawler rebuilds them into readable paragraphs, which is usually what the data is for
• Clean, flat output — 30+ fields per work, arrays of primitives rather than nested blobs, so downstream CSV exports, pandas DataFrames, and database loads work without preprocessing