Scrape and organize YouTube video descriptions

To scrape and organize YouTube video descriptions, you typically follow these steps using code, as YouTube does not allow scraping via HTML due to dynamic content and policy constraints. The best method is using the YouTube Data API v3, which allows authorized access to video metadata including descriptions.

Here’s how you can do it using Python:

✅ Prerequisites:

1. Google Developer account

2. YouTube Data API v3 enabled

3. API key created

✅ Python Script to Scrape and Organize YouTube Video Descriptions

python
CopyEdit
import requests
import pandas as pd

# Replace with your actual API key
API_KEY = 'YOUR_YOUTUBE_API_KEY'

# Example channel ID or list of video IDs
channel_id = 'UC_x5XG1OV2P6uZZ5FSM9Ttw'  # Google Developers Channel

# Step 1: Get uploads playlist ID
def get_uploads_playlist_id(channel_id):
    url = f"https://www.googleapis.com/youtube/v3/channels?part=contentDetails&id={channel_id}&key={API_KEY}"
    response = requests.get(url).json()
    playlist_id = response['items'][0]['contentDetails']['relatedPlaylists']['uploads']
    return playlist_id

# Step 2: Get videos from playlist
def get_video_ids_from_playlist(playlist_id, max_results=50):
    url = f"https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults={max_results}&playlistId={playlist_id}&key={API_KEY}"
    response = requests.get(url).json()
    video_data = []
    for item in response['items']:
        title = item['snippet']['title']
        video_id = item['snippet']['resourceId']['videoId']
        video_data.append({'title': title, 'video_id': video_id})
    return video_data

# Step 3: Get video descriptions
def get_video_descriptions(video_data):
    descriptions = []
    for video in video_data:
        video_id = video['video_id']
        url = f"https://www.googleapis.com/youtube/v3/videos?part=snippet&id={video_id}&key={API_KEY}"
        response = requests.get(url).json()
        snippet = response['items'][0]['snippet']
        descriptions.append({
            'title': snippet['title'],
            'video_id': video_id,
            'description': snippet['description']
        })
    return descriptions

# Main execution
playlist_id = get_uploads_playlist_id(channel_id)
video_data = get_video_ids_from_playlist(playlist_id, max_results=50)
descriptions = get_video_descriptions(video_data)

# Optional: Save to DataFrame
df = pd.DataFrame(descriptions)
print(df.head())  # Preview data

# Save to CSV if needed
df.to_csv('youtube_video_descriptions.csv', index=False)

📁 Output Structure in CSV or DataFrame:

🔒 Notes:

• YouTube API has quota limits (default: 10,000 units/day).

• Each videos API call costs 1 unit per ID.

• Use pagination (nextPageToken) for more than 50 results.

Would you like a version that scrapes based on a search term or from a playlist instead of a channel?