Archive livestream sessions with Python

Livestreaming has become a mainstream way of sharing content in real time, whether it’s on platforms like YouTube, Twitch, or custom RTMP-based setups. However, archiving these livestreams — i.e., saving them for later viewing — is often essential for content repurposing, record-keeping, or analytics. Python, with its robust ecosystem of libraries, provides a flexible and powerful way to automate and customize livestream archiving workflows. Below is a detailed guide on how to archive livestream sessions with Python, covering multiple platforms and protocols.

Understanding Livestreaming Protocols

Before diving into the code, it’s essential to understand the protocols involved in livestreaming:

• RTMP (Real-Time Messaging Protocol): Used by many platforms including Twitch and YouTube for pushing livestream content.

• HLS (HTTP Live Streaming): Apple’s streaming protocol used for playback. Platforms like YouTube use HLS to serve livestreams to viewers.

• DASH (Dynamic Adaptive Streaming over HTTP): Used by platforms such as Facebook Live and YouTube.

To archive a livestream, you can either:

1. Pull from the playback URL (HLS/DASH).

2. Capture from a stream pushed to an RTMP server.

Tools and Libraries

Key Python tools and libraries involved in livestream archiving:

• ffmpeg-python: A Python wrapper for FFmpeg, used for media manipulation.

• streamlink: A command-line utility with Python bindings for extracting streams from various services.

• requests: For making HTTP requests (useful for APIs).

• schedule or APScheduler: For automating archiving jobs.

• subprocess: When direct FFmpeg shell commands are required.

Method 1: Archive HLS Streams Using FFmpeg

You can archive a livestream by recording its HLS playlist (.m3u8) URL using FFmpeg. Python can orchestrate this.

python
CopyEdit
import subprocess
from datetime import datetime

def record_hls_stream(hls_url, output_path):
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    filename = f"{output_path}/livestream_{timestamp}.mp4"
    
    command = [
        'ffmpeg',
        '-i', hls_url,
        '-c', 'copy',
        '-t', '01:00:00',  # Record for 1 hour
        filename
    ]
    
    subprocess.run(command)

Example Usage:

python
CopyEdit
hls_url = 'https://example.com/live/stream.m3u8'
output_path = '/path/to/archive'
record_hls_stream(hls_url, output_path)

Method 2: Use Streamlink to Archive from Supported Services

Streamlink supports platforms like YouTube, Twitch, and more. It extracts the stream and passes it to a media player or saves it to a file.

python
CopyEdit
import subprocess
from datetime import datetime

def download_with_streamlink(stream_url, output_path):
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    filename = f"{output_path}/archive_{timestamp}.mp4"
    
    command = [
        'streamlink',
        stream_url,
        'best',
        '-o',
        filename
    ]
    
    subprocess.run(command)

Example:

python
CopyEdit
download_with_streamlink("https://www.twitch.tv/example_channel", "/videos")

Method 3: Capture RTMP Streams

If you control the stream source and are pushing to an RTMP server, you can also record from the RTMP endpoint.

python
CopyEdit
def record_rtmp_stream(rtmp_url, output_path, duration="00:30:00"):
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    filename = f"{output_path}/rtmp_recording_{timestamp}.mp4"
    
    command = [
        'ffmpeg',
        '-i', rtmp_url,
        '-t', duration,
        '-c', 'copy',
        filename
    ]
    
    subprocess.run(command)

Automating Stream Archiving

Using schedule, you can automate the archiving process to run at specific times (e.g., daily or on a weekly schedule).

python
CopyEdit
import schedule
import time

def job():
    download_with_streamlink("https://www.youtube.com/watch?v=live_stream_id", "/archives")

schedule.every().day.at("10:00").do(job)

while True:
    schedule.run_pending()
    time.sleep(60)

Extracting Stream URLs from APIs

For platforms with official APIs (YouTube, Twitch), you can dynamically fetch livestream URLs:

YouTube Data API Example:

python
CopyEdit
import requests

def get_live_video_id(channel_id, api_key):
    url = f"https://www.googleapis.com/youtube/v3/search?part=snippet&channelId={channel_id}&eventType=live&type=video&key={api_key}"
    response = requests.get(url)
    data = response.json()
    if data['items']:
        return data['items'][0]['id']['videoId']
    return None

Combine with Archiving:

python
CopyEdit
video_id = get_live_video_id('UCabcxyz1234', 'YOUR_API_KEY')
if video_id:
    stream_url = f"https://www.youtube.com/watch?v={video_id}"
    download_with_streamlink(stream_url, "/yt_archive")

Monitoring and Notifications

To monitor archiving status or failures:

• Use logging to log successes or errors.

• Send email or Telegram notifications when a stream ends or fails.

• Combine with a database or dashboard (like using SQLite + Flask) to track archives.

Security and Best Practices

• Ensure you have permission to archive content.

• Respect copyright and platform TOS.

• Use robust exception handling and retry logic.

• Store archives on external or cloud storage (e.g., S3, Google Drive) to avoid disk overflow.

Advanced Archival Enhancements

• Transcode on the fly: Convert to different formats (e.g., .webm or .avi) while recording.

• Segment recordings: Use FFmpeg to split long recordings into 30-minute files.

• Metadata tagging: Add timestamps, channel name, or topics into the filename or embedded metadata.

• Webhook integrations: Trigger Slack or Discord alerts when a new recording starts/stops.

Conclusion

Python simplifies the automation and customization of livestream archiving through tools like FFmpeg, Streamlink, and APIs. Whether you’re archiving your own sessions or capturing public broadcasts for permitted uses, Python gives you precise control over the recording process, scheduling, file management, and notifications. With a well-built system, you can ensure you never miss a livestream again — and always have access to high-quality archives for future use.