Scrape apartment listings by location

To scrape apartment listings by location, you’ll typically use a combination of tools and libraries like Python, requests, BeautifulSoup, or Selenium for dynamic websites. Here’s a step-by-step guide and sample code for scraping apartment listings (e.g., from Craigslist, Zillow, or similar sites) by location:

⚠️ Important: Always check the website’s robots.txt file and terms of service before scraping. Many real estate sites restrict or prohibit scraping.

✅ Example: Scrape Craigslist Apartment Listings by Location (using Python)

1. Install Required Libraries:

bash
CopyEdit
pip install requests beautifulsoup4

2. Sample Code:

python
CopyEdit
import requests
from bs4 import BeautifulSoup

def scrape_apartments(location='newyork', max_results=10):
    base_url = f'https://{location}.craigslist.org/search/apa'
    params = {'sort': 'date'}
    response = requests.get(base_url, params=params)
    
    if response.status_code != 200:
        print("Failed to retrieve listings")
        return []

    soup = BeautifulSoup(response.text, 'html.parser')
    listings = soup.find_all('li', class_='result-row')[:max_results]

    results = []
    for listing in listings:
        title = listing.find('a', class_='result-title').text
        price = listing.find('span', class_='result-price').text
        date = listing.find('time')['datetime']
        link = listing.find('a', class_='result-title')['href']
        results.append({'title': title, 'price': price, 'date': date, 'link': link})
    
    return results

# Example usage
apartments = scrape_apartments(location='sfbay', max_results=5)
for apt in apartments:
    print(apt)

🔁 For Other Sites (e.g., Zillow, Apartments.com)

These often require Selenium due to dynamic content.

Example with Selenium (Zillow):

python
CopyEdit
from selenium import webdriver
from selenium.webdriver.common.by import By
import time

def scrape_zillow(location_query):
    driver = webdriver.Chrome()
    search_url = f"https://www.zillow.com/homes/for_rent/{location_query}_rb/"
    driver.get(search_url)
    time.sleep(5)  # wait for JavaScript to load

    listings = driver.find_elements(By.CLASS_NAME, 'list-card-info')
    results = []
    for listing in listings[:5]:
        title = listing.find_element(By.CLASS_NAME, 'list-card-heading').text
        price = listing.find_element(By.CLASS_NAME, 'list-card-price').text
        link = listing.find_element(By.TAG_NAME, 'a').get_attribute('href')
        results.append({'title': title, 'price': price, 'link': link})

    driver.quit()
    return results

# Usage:
# scrape_zillow("San-Francisco-CA")

📌 Tips for Scraping by Location:

• Use location-specific subdomains or query parameters.

• Add delay between requests (time.sleep) to avoid being blocked.

• Use headers to mimic a real browser (especially on modern websites).

• Use proxies for large-scale scraping.

Would you like a version that saves to CSV, scrapes multiple cities at once, or works with a specific site?