Scrape recipe ingredients from cooking blogs

To scrape recipe ingredients from cooking blogs, you can use Python with libraries like requests, BeautifulSoup, and optionally Selenium for JavaScript-heavy sites. Below is a sample script that shows how to scrape recipe ingredients from a cooking blog post:

Python Script to Scrape Ingredients

python
CopyEdit
import requests
from bs4 import BeautifulSoup

def get_ingredients(url):
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)'
    }

    response = requests.get(url, headers=headers)
    soup = BeautifulSoup(response.content, 'html.parser')

    ingredients = []

    # Common class names used by cooking blogs (adjust based on site)
    possible_classes = [
        'ingredients', 
        'ingredient', 
        'recipe-ingredients', 
        'ingredients-list', 
        'wprm-recipe-ingredient'
    ]

    for class_name in possible_classes:
        ingredient_sections = soup.find_all(class_=class_name)
        for section in ingredient_sections:
            for li in section.find_all(['li', 'span']):
                text = li.get_text(strip=True)
                if text and text not in ingredients:
                    ingredients.append(text)

    return ingredients

# Example usage
url = 'https://example.com/your-recipe-page'
ingredients = get_ingredients(url)

for idx, item in enumerate(ingredients, 1):
    print(f"{idx}. {item}")

Notes:

• Dynamic content: If the blog loads content with JavaScript (e.g., using React), use Selenium or Playwright to load the full page.

• Structured data: Many recipe blogs use schema.org’s Recipe markup in JSON-LD. You can also parse the embedded JSON to extract ingredients more reliably.

• Robots.txt: Always check the website’s robots.txt and terms of service before scraping.

Would you like a version that extracts structured recipe data (title, steps, time, etc.) as well?