Filter spam from a CSV list of emails

Filtering spam from a CSV list of emails typically involves identifying and removing emails that are either clearly spam traps, suspicious, or known to be from disposable or low-quality domains. Here’s a detailed approach you can follow to filter spam emails from your CSV list:

1. Load and Parse the CSV

• Read the CSV file.

• Extract the email column.

Example in Python:

python
CopyEdit
import csv

emails = []
with open('emails.csv', newline='') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        emails.append(row['email'].strip().lower())

2. Basic Cleaning

• Remove duplicates.

• Remove invalid email formats.

Use regex for email validation:

python
CopyEdit
import re

def is_valid_email(email):
    pattern = r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+.[a-zA-Z0-9-.]+$"
    return re.match(pattern, email) is not None

clean_emails = list(set(filter(is_valid_email, emails)))

3. Identify Disposable or Temporary Emails

Disposable email providers offer temporary emails often used for spam or throwaway accounts.

• Use a list of disposable email domains (e.g., mailinator.com, tempmail.com, 10minutemail.com).

• Remove emails with those domains.

Example:

python
CopyEdit
disposable_domains = set(['mailinator.com', 'tempmail.com', '10minutemail.com'])  # Add more to your list

def is_disposable(email):
    domain = email.split('@')[1]
    return domain in disposable_domains

filtered_emails = [email for email in clean_emails if not is_disposable(email)]

4. Check Against Known Spam Traps or Blacklists

• Use APIs or datasets that list known spam traps or blacklisted domains/emails.

• Remove those from your list.

Example services:

• Spamhaus

• AbuseIPDB

• Email validation APIs like ZeroBounce, NeverBounce, etc.

5. Heuristic Checks

• Remove emails with suspicious patterns (e.g., random strings, very short usernames).

• Flag emails that contain suspicious keywords like spam, test, fake.

Example:

python
CopyEdit
def is_suspicious(email):
    suspicious_keywords = ['spam', 'test', 'fake']
    username = email.split('@')[0]
    if any(keyword in username for keyword in suspicious_keywords):
        return True
    if len(username) < 3:  # too short usernames
        return True
    return False

final_emails = [email for email in filtered_emails if not is_suspicious(email)]

6. Optional: Verify Emails

• Use SMTP verification tools to check if the email server exists and accepts mail.

• This is more advanced and can be done with specialized libraries.

Summary of Filtering Steps in Code:

python
CopyEdit
import csv
import re

def is_valid_email(email):
    pattern = r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+.[a-zA-Z0-9-.]+$"
    return re.match(pattern, email) is not None

disposable_domains = set(['mailinator.com', 'tempmail.com', '10minutemail.com'])  # Expand this list

def is_disposable(email):
    domain = email.split('@')[1]
    return domain in disposable_domains

def is_suspicious(email):
    suspicious_keywords = ['spam', 'test', 'fake']
    username = email.split('@')[0]
    if any(keyword in username for keyword in suspicious_keywords):
        return True
    if len(username) < 3:
        return True
    return False

emails = []
with open('emails.csv', newline='') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        emails.append(row['email'].strip().lower())

clean_emails = list(set(filter(is_valid_email, emails)))
filtered_emails = [email for email in clean_emails if not is_disposable(email)]
final_emails = [email for email in filtered_emails if not is_suspicious(email)]

# final_emails now contains the filtered list without obvious spam or disposable emails

This approach will help clean and filter spam or low-quality emails from your CSV list. For the most accurate filtering, integrating professional email validation services is recommended.