Back to feed
Dev.to
Dev.to
7/22/2026
Open-source Python pipeline for automated tech job scraping and LLM-enriched matching

Open-source Python pipeline for automated tech job scraping and LLM-enriched matching

Original: I Built a Job-Scraping Bot to Escape Cloud Consulting (And It's Open Source)

Short summary

A developer burned out from manual job hunting built 'job-digest,' an open-source Python pipeline that scrapes LinkedIn, Indeed, Glassdoor and others, enriches listings with GPT-4o-mini (~$0.02/100 jobs), ranks by stack match, and delivers a daily Telegram digest. The architecture uses async httpx, SQLite for dedup, adaptive rate limiting, and cron on a Raspberry Pi. The author seeks collaborators and shares full architecture decisions.

  • Open-source Python pipeline scrapes multiple job boards, enriches with GPT-4o-mini at ~$0.02/100 jobs, and delivers daily Telegram digest
  • Architecture: async scrapers → LLM enricher → scoring ranker → Telegram notifier, backed by SQLite
  • Author applied to 73 jobs in 4 weeks with 4 phone screens before automating the process

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more