How to Automate Disk Cleanup with Python

Automating disk cleanup using Python is an effective way to manage storage space, remove unnecessary files, and improve system performance without manual intervention. This process can be particularly useful for system administrators, developers, or any user looking to maintain a tidy and efficient file system. This article outlines a practical guide on how to automate disk cleanup with Python, focusing on identifying unnecessary files, deleting temporary or old files, and scheduling the script to run at regular intervals.

Understanding Disk Cleanup Objectives

Before diving into code, it’s important to define what “cleanup” means in the context of your system:

• Temporary files: Files stored temporarily by applications or the operating system.

• Old log files: Logs generated by applications that are no longer relevant.

• Unused cache files: Cached data not accessed for a long time.

• Large obsolete files: Files that are unnecessarily consuming large disk space.

By identifying what you want to delete or manage, you can design your script accordingly.

Setting Up the Environment

Ensure you have Python installed. Most systems come with Python pre-installed, but you can verify it with:

bash
CopyEdit
python --version

Install any required libraries using pip:

bash
CopyEdit
pip install schedule

You may also need psutil for monitoring disk usage:

bash
CopyEdit
pip install psutil

Scanning for Files to Delete

The script needs to locate files based on specific criteria—such as file type, age, or size.

python
CopyEdit
import os
import time

def find_files_to_delete(directory, days_old=30):
    current_time = time.time()
    files_to_delete = []
    for root, dirs, files in os.walk(directory):
        for name in files:
            file_path = os.path.join(root, name)
            if os.path.isfile(file_path):
                file_age = current_time - os.path.getmtime(file_path)
                if file_age > days_old * 86400:  # Convert days to seconds
                    files_to_delete.append(file_path)
    return files_to_delete

This function walks through a directory, identifying files older than a specified number of days.

Deleting Unnecessary Files

Once files are identified, you can proceed to delete them safely.

python
CopyEdit
def delete_files(file_list):
    for file_path in file_list:
        try:
            os.remove(file_path)
            print(f"Deleted: {file_path}")
        except Exception as e:
            print(f"Error deleting {file_path}: {e}")

Use this function with caution to avoid deleting critical system or application files.

Automating the Cleanup Process

Python’s schedule module helps to automate this cleanup process.

python
CopyEdit
import schedule
import time

def cleanup_job():
    target_dir = "/path/to/cleanup"
    files = find_files_to_delete(target_dir, days_old=30)
    delete_files(files)

schedule.every().day.at("02:00").do(cleanup_job)

while True:
    schedule.run_pending()
    time.sleep(60)

This script schedules the cleanup to run daily at 2 AM. Modify the path and timing as needed.

Monitoring Disk Usage with psutil

You can also incorporate disk monitoring to trigger cleanup only when disk usage exceeds a certain threshold.

python
CopyEdit
import psutil

def should_cleanup(threshold=80):
    disk_usage = psutil.disk_usage('/')
    return disk_usage.percent > threshold

Integrate this check into your job:

python
CopyEdit
def cleanup_job_with_check():
    if should_cleanup(80):
        print("Disk usage high. Starting cleanup...")
        files = find_files_to_delete("/path/to/cleanup", days_old=30)
        delete_files(files)
    else:
        print("Disk usage under control. No cleanup needed.")

Adding Logging for Auditing

To track what’s being deleted and when, implement logging:

python
CopyEdit
import logging

logging.basicConfig(filename='cleanup.log', level=logging.INFO,
                    format='%(asctime)s - %(levelname)s - %(message)s')

def delete_files_with_logging(file_list):
    for file_path in file_list:
        try:
            os.remove(file_path)
            logging.info(f"Deleted: {file_path}")
        except Exception as e:
            logging.error(f"Error deleting {file_path}: {e}")

Use this in place of the regular delete function to maintain a permanent audit trail.

Creating a Configuration File

For greater flexibility, use a JSON configuration file to specify cleanup directories, file age, and other parameters.

json
CopyEdit
{
  "directories": ["/tmp", "/var/log"],
  "days_old": 30,
  "disk_usage_threshold": 80
}

Load this configuration in your script:

python
CopyEdit
import json

def load_config():
    with open("config.json", "r") as f:
        return json.load(f)

config = load_config()

Then pass the parameters accordingly in your cleanup logic.

Packaging for Deployment

You can package your script with pyinstaller to create an executable, making it easier to deploy on systems without Python.

bash
CopyEdit
pip install pyinstaller
pyinstaller --onefile cleanup_script.py

This generates a standalone executable in the dist/ folder.

Scheduling with Cron or Task Scheduler

For persistent execution without an open terminal:

On Linux (using Cron):

bash
CopyEdit
crontab -e

Add an entry like:

bash
CopyEdit
0 2 * * * /usr/bin/python3 /path/to/cleanup_script.py

On Windows (using Task Scheduler):

1. Open Task Scheduler.

2. Create a new task.

3. Set trigger to daily at a specific time.

4. Set action to run python.exe with argument as your script path.

Handling Special File Types

You can expand your script to include filters for file types or specific naming patterns:

python
CopyEdit
def find_files_by_extension(directory, extension=".log"):
    files_to_delete = []
    for root, dirs, files in os.walk(directory):
        for name in files:
            if name.endswith(extension):
                files_to_delete.append(os.path.join(root, name))
    return files_to_delete

Combine this with age checks for more targeted cleanup.

Error Handling and Safety Measures

To ensure safe operation:

• Include a dry-run mode to preview deletions.

• Use whitelist/blacklist folders.

• Never operate on system-critical directories like /bin, /etc, or C:Windows.

Example of dry-run mode:

python
CopyEdit
def dry_run(file_list):
    for file in file_list:
        print(f"Would delete: {file}")

Activate with a command-line flag or a config setting.

Conclusion

Automating disk cleanup with Python provides a customizable and powerful way to manage file clutter and disk usage. Whether targeting temporary files, aging logs, or oversized directories, Python allows for a flexible and scalable approach. With scheduled execution, logging, and disk monitoring, your cleanup process can become an intelligent background task that maintains your system without manual intervention.