Gig
$20 per day (4 hour per day)
TBD
Apr 4, 2025
Overview:
I'm looking for a Python developer to help streamline and expand a set of existing job scraper scripts. These scrapers currently work for a subset of target companies, and I need someone who can improve their reliability and add support for new sources. You’ll be working with my existing Python codebase and helping me build toward a more scalable solution.
Responsibilities:
Review and refactor existing job scrapers for clarity, consistency, and performance.
Implement a more unified architecture for scraping across multiple sources.
Add scraping support for additional job boards and company career pages.
Handle common scraping challenges (e.g., JavaScript rendering, CAPTCHAs, IP blocks).
Clean and structure the scraped job data for storage and future use (e.g., CSV, JSON, database).
Recommend methods to set up automation/scheduling to run scrapers regularly.
Requirements:
Strong Python skills with experience in web scraping using requests, BeautifulSoup, Selenium, or Playwright.
Familiarity with data structuring and cleaning using pandas.
Ability to troubleshoot and improve existing code.
Knowledge of good coding practices, error handling, and logging.
Self-motivated and comfortable working independently with some guidance.
Nice-to-Haves:
Experience scraping JavaScript-heavy sites or bypassing anti-scraping measures.
Experience with databases (PostgreSQL, MongoDB, etc.).
To Apply:
Please send your portfolio or GitHub profile and a quick note about relevant scraping projects you’ve worked on. Bonus points if you can share an example of a multi-site scraping tool you’ve built or helped optimize.
Pay:
Salary will be $20 per day (with the expectation that you work ~4 hours per day). I expect this to take 5 days with the opportunity to extend / take on future projects based on performance