Scrape job listings into a career tracker

To scrape job listings into a career tracker, you’ll need a tool or script that can:

1. Scrape listings from job boards or company websites

2. Extract relevant data (job title, company, location, salary, posting date, URL, etc.)

3. Store the data in a structured format like a spreadsheet, database, or web dashboard for tracking.

Here’s a high-level approach using Python:

1. Choose Target Job Boards

Popular options include:

• Indeed

• LinkedIn (requires workarounds due to anti-scraping measures)

• Glassdoor

• Remote OK

• Stack Overflow Jobs

2. Set Up Scraping Script (e.g., with BeautifulSoup & Requests)

python
CopyEdit
import requests
from bs4 import BeautifulSoup
import pandas as pd

def scrape_indeed_jobs(query, location, num_pages=1):
    base_url = "https://www.indeed.com/jobs"
    job_listings = []

    for page in range(num_pages):
        params = {"q": query, "l": location, "start": page * 10}
        response = requests.get(base_url, params=params)
        soup = BeautifulSoup(response.text, "html.parser")

        for card in soup.select(".result"):
            title = card.select_one("h2.jobTitle span").text if card.select_one("h2.jobTitle span") else ""
            company = card.select_one(".companyName").text if card.select_one(".companyName") else ""
            location = card.select_one(".companyLocation").text if card.select_one(".companyLocation") else ""
            summary = card.select_one(".job-snippet").text.strip() if card.select_one(".job-snippet") else ""
            link = "https://www.indeed.com" + card.a['href'] if card.a else ""

            job_listings.append({
                "Job Title": title,
                "Company": company,
                "Location": location,
                "Summary": summary,
                "Link": link
            })

    return pd.DataFrame(job_listings)

3. Store in a Career Tracker (Spreadsheet/CSV)

python
CopyEdit
df = scrape_indeed_jobs("Data Analyst", "Remote", num_pages=3)
df.to_csv("career_tracker.csv", index=False)

4. Optional: Use Google Sheets API for Live Tracker

You can push data to a live Google Sheet:

• Authenticate via gspread and oauth2client

• Update your sheet with fresh job listings daily or weekly

5. Automate with Cron/Task Scheduler

Run your script on a regular schedule using:

• Linux/macOS: cron

• Windows: Task Scheduler

• Or use cloud automation (e.g., GitHub Actions, Zapier with Webhooks)

Would you like a version for LinkedIn, Google Sheets integration, or a ready-to-run web dashboard?