Resident Advisor Historical + Ongoing Scraper
Large-scale scraper + incremental updater + PostgreSQL pipeline for RA.co events.
R
RA.co Event ScraperPlaywrightScrapyPostgreSQL
The Challenge
Large-scale scraper + incremental updater + PostgreSQL pipeline for RA.co events.
Key Highlights
- Incremental ID logic
- Retry handling
- Rate-limit safety
- Playwright/Scrapy hybrid design
- Railway PostgreSQL integration
Outcome
Continuous production ingestion with full documentation.
Technologies Used
Playwright
Scrapy
PostgreSQL