Schedule backup tasks with Python

Scheduling backup tasks with Python can be efficiently done using various methods. Below is an approach that uses the schedule library for time-based task scheduling and shutil for performing the backup.

Prerequisites:

1. Install the schedule library:

   bash
   CopyEdit
   pip install schedule
   

2. Python’s built-in shutil library is great for copying files and directories, which can be used to perform the actual backup.

Example: Scheduling Regular Backups

This example demonstrates how to backup files from one directory to another at scheduled intervals.

python
CopyEdit
import schedule
import time
import shutil
import os
from datetime import datetime

# Define the backup function
def backup_files():
    # Define source and destination
    source_dir = "/path/to/source/directory"
    backup_dir = "/path/to/backup/directory"
    
    # Get the current date to create a unique backup folder
    current_date = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    backup_folder = os.path.join(backup_dir, f"backup_{current_date}")

    # Create the backup folder if it doesn't exist
    if not os.path.exists(backup_folder):
        os.makedirs(backup_folder)
    
    # Perform the backup using shutil.copytree for directories
    try:
        shutil.copytree(source_dir, backup_folder)
        print(f"Backup completed successfully: {backup_folder}")
    except Exception as e:
        print(f"Error during backup: {e}")

# Schedule the backup task
schedule.every().day.at("02:00").do(backup_files)  # Backups at 2 AM daily

# Run the scheduler
while True:
    schedule.run_pending()
    time.sleep(60)  # Wait for a minute before checking again

Explanation:

• Source Directory: The directory where your files are located.

• Backup Directory: The directory where backups will be saved.

• Date-based Folder Naming: The backup folder name includes a timestamp to ensure that backups don’t overwrite each other.

• shutil.copytree(): This is used to copy the entire contents of the source directory to the backup folder.

• Scheduling: The schedule library is used to run the backup_files function daily at 2 AM.

Additional Considerations:

1. Error Handling: Ensure that backup tasks include proper error handling to prevent data loss.

2. Multiple Backup Intervals: You can schedule backups at multiple intervals using schedule.every() for times such as weekly, monthly, or even hourly backups.

3. Logging: Consider adding logging functionality for better tracking and reporting.

4. File Changes: For incremental backups, you might need to track changed files. This can be achieved using tools like rsync or by maintaining a log of modified files.

Advanced Example: Incremental Backups

If you need incremental backups (only backing up new or changed files), you could use os.path.getmtime() to check file modification times or use a more advanced library like rsync.

Here’s a quick example that compares modification times:

python
CopyEdit
import os
import shutil
from datetime import datetime

def incremental_backup(source_dir, backup_dir):
    # Ensure the backup directory exists
    if not os.path.exists(backup_dir):
        os.makedirs(backup_dir)

    # Get current time for logging purposes
    current_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")

    for foldername, subfolders, filenames in os.walk(source_dir):
        for filename in filenames:
            source_filepath = os.path.join(foldername, filename)
            relative_path = os.path.relpath(source_filepath, source_dir)
            backup_filepath = os.path.join(backup_dir, relative_path)

            # Check if file exists and if it's modified
            if not os.path.exists(backup_filepath) or os.path.getmtime(source_filepath) > os.path.getmtime(backup_filepath):
                # Ensure the backup folder structure exists
                os.makedirs(os.path.dirname(backup_filepath), exist_ok=True)

                # Copy the file
                shutil.copy2(source_filepath, backup_filepath)
                print(f"Backed up: {source_filepath} -> {backup_filepath}")
            else:
                print(f"Skipped: {source_filepath} (No changes)")

# Example usage
source_directory = "/path/to/source/directory"
backup_directory = "/path/to/backup/directory"
incremental_backup(source_directory, backup_directory)

In this approach:

• We walk through the source directory.

• We check the modification times of the files, and only copy those that are new or changed.

Conclusion

By combining Python’s powerful libraries such as schedule, shutil, and os, you can easily create scheduled backup tasks. For regular backups, you can schedule tasks at fixed intervals, and for incremental backups, you can track file modifications.