Build a personal finance anomaly detector

A personal finance anomaly detector identifies unusual or suspicious financial transactions by analyzing patterns in spending behavior. Here’s a step-by-step guide to building a simple anomaly detector using Python. The approach uses unsupervised machine learning, as we often lack labeled anomalous data in personal finance.

---

1. Define the Problem

Detect anomalies in personal finance data such as:

• Unusually large transactions

• Spending in rarely used categories

• Duplicate transactions

• Transactions outside of expected timeframes

---

2. Gather and Prepare Data

Use a sample dataset with the following fields:

• Date

• Description

• Category

• Amount

• Account

You can generate a CSV or use data from apps like Mint, YNAB, or exported bank statements.

Example CSV:

csv
CopyEdit
Date,Description,Category,Amount,Account
2025-05-01,Coffee Shop,Food & Drink,3.50,Checking
2025-05-02,Online Shopping,Shopping,350.00,Checking
...

---

3. Preprocess Data

• Parse dates

• Normalize amounts

• Encode categorical data

• Extract features (e.g., day of week, transaction hour)

python
CopyEdit
import pandas as pd
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.ensemble import IsolationForest

# Load data
df = pd.read_csv('transactions.csv', parse_dates=['Date'])

# Feature engineering
df['DayOfWeek'] = df['Date'].dt.dayofweek
df['Hour'] = df['Date'].dt.hour

# Drop unused columns
X = df[['Amount', 'Category', 'DayOfWeek']]

# Preprocessing
preprocessor = ColumnTransformer(transformers=[
    ('num', StandardScaler(), ['Amount', 'DayOfWeek']),
    ('cat', OneHotEncoder(), ['Category'])
])

---

4. Train Anomaly Detection Model

Using Isolation Forest, a popular anomaly detection algorithm.

python
CopyEdit
# Pipeline with preprocessing + model
pipeline = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('model', IsolationForest(contamination=0.02, random_state=42))
])

pipeline.fit(X)

# Predict anomalies (-1 is anomaly, 1 is normal)
df['Anomaly'] = pipeline.predict(X)
df['Anomaly'] = df['Anomaly'].map({1: 0, -1: 1})

---

5. Review Anomalies

python
CopyEdit
anomalies = df[df['Anomaly'] == 1]
print(anomalies[['Date', 'Description', 'Amount', 'Category']])

---

6. Optional Enhancements

Add more features:

• Transaction frequency per category

• Rolling average spend

• Merchant name vectorization (e.g., TF-IDF)

Use other models:

• One-Class SVM

• Autoencoders (deep learning)

• DBSCAN (density-based clustering)

Visualization:

python
CopyEdit
import matplotlib.pyplot as plt

plt.scatter(df.index, df['Amount'], c=df['Anomaly'], cmap='coolwarm')
plt.title("Anomaly Detection in Transactions")
plt.xlabel("Transaction Index")
plt.ylabel("Amount")
plt.show()

---

7. Deployment Tips

• Run as a scheduled task (daily/weekly)

• Integrate with Google Sheets, email alerts, or dashboards

• Store model output logs with timestamps

---

Conclusion

A personal finance anomaly detector can help flag suspicious transactions early, enabling better budget control and fraud detection. With basic data, a simple machine learning pipeline like Isolation Forest is effective. For more sophisticated solutions, consider incorporating user feedback loops and continuously training the model with new data.