Open Source Scraping Benchmark

Benchmark Your Scraper Against Real-World Challenges

ScrapeMe provides 18 test scenarios simulating real websites — e-commerce, auctions, GraphQL APIs, CSRF protection, robots.txt compliance, and more. Every visit is tracked, fingerprinted, and scored.

18
Test Scenarios
4,100+
Seeded Data Points
1,800
Max Score
9
Bot Signals Tracked

10 Test Scenarios

Each test simulates a real-world scraping challenge with difficulty levels, verifiable data, and clear passing criteria.

How It Works

Four steps to benchmark your scraper. Max score: 1,000 points.

1
Create a Test Run
POST to /api/test-run to register your scraper and get an API key.
POST /api/test-run
2
Scrape the Tests
Visit each test page with your scraper. Extract the data.
GET /tests/ecommerce, /tests/auction, ...
3
Submit Results
Send your extracted data for verification against the database.
POST /api/test-run/{id}/submit
4
Get Your Score
Receive a detailed report with per-test scores and bot analysis.
GET /api/test-run/{id}/report
Compete & Compare

Public Leaderboard

Register your scraper, run the benchmark, and see how you rank against others. Top performers earn the “Certified” badge for scoring above 80%.

Every Visit Is Tracked & Scored

The admin dashboard reveals exactly what your scraper did — and whether it was detected.

Visitor Fingerprinting

Canvas hash, WebGL renderer, font metrics, navigator properties — all collected silently via JavaScript.

Bot Score (0-100)

9 detection signals analyzed in real-time: webdriver flag, headless markers, timing anomalies, honeypot hits, and more.

Session Replay

See the exact sequence of pages your scraper visited, with millisecond-precision timing between each request.

Honeypot Detection

8 hidden trap links using CSS tricks. Bots that follow invisible links are immediately flagged.

Visit Reports

Per-session analysis with bot score, human score, classification, detected signals, and a natural language verdict.

Test Run Reports

Programmatic API returns scored results with bot analysis. Compare scraper runs over time.

Quick Start

Get up and running in under 2 minutes.

# Start PostgreSQL
docker run -d --name scrapeme-postgres \
  -e POSTGRES_USER=scrapeme \
  -e POSTGRES_PASSWORD=scrapeme \
  -e POSTGRES_DB=scrapeme \
  -p 5432:5432 postgres:16-alpine

# Install & seed
npm install
cd apps/web
cp .env.example .env
npx prisma db push && npx prisma db seed

# Run
cd ../..
npm run dev

Admin dashboard: admin@scrapeme.dev / admin123

Test accounts: testuser1 through testuser5 / password123