Schedule backups with Python

Scheduling backups with Python can be achieved using several approaches depending on your environment and needs. Here’s a detailed guide on how to automate backups using Python with built-in modules and scheduling techniques.

Why Schedule Backups with Python?

• Automate regular data backup tasks.

• Avoid human error or forgetting to backup.

• Customize backup logic (select files, compress, upload).

• Integrate with other Python scripts or tools.

Basic Python Backup Script

A simple backup script copies files or directories from a source to a destination, optionally compressing them.

python
CopyEdit
import shutil
import os
import datetime

def backup_folder(source_folder, backup_folder):
    if not os.path.exists(backup_folder):
        os.makedirs(backup_folder)
    
    date_str = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    backup_path = os.path.join(backup_folder, f"backup_{date_str}")
    
    shutil.copytree(source_folder, backup_path)
    print(f"Backup created at: {backup_path}")

if __name__ == "__main__":
    source = "/path/to/source"
    backup = "/path/to/backup"
    backup_folder(source, backup)

This script copies the entire source directory into a timestamped backup directory.

Adding Compression

Compressing backups saves space and simplifies storage.

python
CopyEdit
import shutil
import os
import datetime

def backup_and_compress(source_folder, backup_folder):
    if not os.path.exists(backup_folder):
        os.makedirs(backup_folder)
    
    date_str = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    archive_name = os.path.join(backup_folder, f"backup_{date_str}")
    
    shutil.make_archive(archive_name, 'zip', source_folder)
    print(f"Backup compressed at: {archive_name}.zip")

if __name__ == "__main__":
    source = "/path/to/source"
    backup = "/path/to/backup"
    backup_and_compress(source, backup)

This creates a zip archive of the source folder with a timestamp.

Scheduling Backups

Option 1: Using schedule Python Module

schedule is a lightweight library to run Python functions at scheduled intervals.

1. Install it:

bash
CopyEdit
pip install schedule

2. Create a scheduler script:

python
CopyEdit
import schedule
import time
import shutil
import os
import datetime

def backup_and_compress(source_folder, backup_folder):
    if not os.path.exists(backup_folder):
        os.makedirs(backup_folder)
    
    date_str = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    archive_name = os.path.join(backup_folder, f"backup_{date_str}")
    
    shutil.make_archive(archive_name, 'zip', source_folder)
    print(f"Backup compressed at: {archive_name}.zip")

source = "/path/to/source"
backup = "/path/to/backup"

schedule.every().day.at("02:00").do(backup_and_compress, source, backup)

while True:
    schedule.run_pending()
    time.sleep(60)

This runs the backup daily at 2 AM. Run this script continuously on your server or local machine.

Option 2: Using OS-level Schedulers

On Linux/macOS:

Use cron jobs to schedule the Python script.

• Write the backup script and save it as backup.py.

• Open crontab:

bash
CopyEdit
crontab -e

• Add a cron job (example: daily at 2 AM):

cron
CopyEdit
0 2 * * * /usr/bin/python3 /path/to/backup.py

On Windows:

Use Task Scheduler:

• Open Task Scheduler.

• Create a Basic Task.

• Choose “Daily” and set the time.

• For the action, select “Start a program”.

• Program/script: python

• Add arguments: C:pathtobackup.py

• Finish the setup.

Advanced: Backup to Cloud Storage

You can extend the script to upload backups to cloud services like AWS S3, Google Drive, or Dropbox using their SDKs or APIs.

Example snippet for AWS S3 using boto3:

python
CopyEdit
import boto3

def upload_to_s3(file_path, bucket_name, s3_key):
    s3 = boto3.client('s3')
    s3.upload_file(file_path, bucket_name, s3_key)
    print(f"Uploaded {file_path} to s3://{bucket_name}/{s3_key}")

Integrate this after creating your backup archive.

Best Practices

• Use timestamps to avoid overwriting backups.

• Keep backups organized by date.

• Regularly test restore process.

• Handle exceptions and log errors.

• Secure backup storage location.

• Automate cleanup of old backups to save space.

Scheduling backups with Python lets you customize, automate, and integrate backup processes seamlessly for personal or business use.