Slashdot Scraper
NEWSDEVELOPER TOOLS
Independent tool. Trademarks referenced on this page are the property of their respective owners and are used only to identify the public website this tool reads. No affiliation or endorsement.
Slashdot Scraper
Extract technology news stories from Slashdot.org — the long-running "News for nerds, stuff that matters" aggregator. Captures all story fields available on the main feed and section pages: headline, Slashdot URL, external source link, author, publish time, comment count, department tag, and topic section.
What you get
Each result record contains:
| Field | Description |
|---|---|
storyId |
Slashdot internal story ID |
title |
Story headline |
url |
Slashdot story page URL |
sourceUrl |
External source article URL |
sourceDomain |
Domain of the external source (e.g. reuters.com) |
commentCount |
Number of community comments |
author |
Slashdot editor who posted the story |
publishedAt |
Publication datetime |
department |
"from the X dept." tagline |
section |
Topic section (Hardware, Science, YRO, etc.) |
summary |
Story summary text (first paragraph of the post) |
scrapedAt |
ISO-8601 timestamp of when the record was scraped |
Input
| Parameter | Type | Default | Description |
|---|---|---|---|
maxItems |
integer | 10 | Maximum number of stories to return (0 = unlimited) |
startPage |
integer | 1 | Page to start from (1-based) |
maxPages |
integer | 3 | Number of listing pages to crawl (~15 stories/page) |
Usage
The default run fetches the first 10 stories from Slashdot's homepage. Increase maxItems and maxPages to collect more. Each page yields approximately 15 stories.
Technical notes
- Quick and cheap per story
- The site's bot-protection layer is handled cleanly
- Protocol-relative URLs (
//subdomain.slashdot.org/...) are normalised tohttps://in output - Memory usage: 256 MB
