personal-finance

Local-first personal finance tracker. Imports bank transactions from CSV exports, categorizes them with substring rules, detects internal transfers, tracks net worth via nightly balance snapshots, and serves a dashboard + review UI at /personal-finance over Tailscale. Launchd-scheduled nightly rollup with an iMessage ping when uncategorized transactions pile up.

Status

Phase 1 (MVP) shipped. CSV import, transfer detection, rule-based categorization, nightly rollup, dashboard, and /personal-finance* routes on webpage-server.py all work end-to-end with the test suite green (30 pytest cases in tests/). Phases 1.5 (LLM categorization), 2 (Plaid), and 3 (budgets + tax export + anomaly detection) are still in PLAN.md and unimplemented.

Scope

personal-finance answers one question: "where did my money go this month, and what still needs categorizing?"

At the end of every day, launchd kicks rollup, which: 1. Re-applies all active category rules. 2. Writes one balance snapshot per account. 3. Recomputes month-to-date spend, income, and uncategorized count. 4. If there are uncategorized rows and ≥24h since the last ping, sends an iMessage pointing at /personal-finance/review.

Ingestion is manual CSV-in-the-loop by design: Mark downloads a CSV from each bank's website, runs personal-finance.py import --account <id> <file.csv>, and the dedup layer handles idempotency so re-running never creates duplicates.

Out of scope (Phase 1): - Multi-user, multi-currency. - Automated scraping or brokerage API sync — CSV is the interface. - Budgets / budget alerts (Phase 3). - LLM-based auto-categorization (Phase 1.5). - Plaid / live sync (Phase 2). - Tax export, anomaly detection, Zestimate real-estate tracking (Phase 3).

Investment / brokerage PnL lives in ../day-trading/, not here.

Architecture

Pipeline (one rollup run)

flock .lock           ← single-run enforcement; second invocation exits
  ↓
migrate               ← db._user_version + SCHEMA_V1 idempotent apply
  ↓
ensure_accounts       ← UPSERT every [[accounts]] row from config.toml
  ↓
apply_rules           ← categorize.apply_rules over NULL-category rows
  ↓
snapshot_balances     ← one balance_snapshots row per account per day
  ↓
month_summary         ← spend_total, income_total, uncat, category_breakdown
  ↓
should_ping?          ← ≥24h since rollup_state.last_ping_ts AND uncat>0
  ↓ yes
ping_text             ← "Finance: $X spent this month. N txns need categorizing: <url>"
  ↓
deliver.send_imessage ← bb-send.sh +phone "<text>"; record_ping on success

Manual CSV flow (personal-finance.py import):

src.csv
  ↓
_archive_csv          ← copy to imports/<sha1>.csv (content-addressed)
  ↓
_check_header         ← KeyError fast if mapper columns missing
  ↓
for row in DictReader ← mapper.date_col + amount_col + description_col
  ↓
parse_date + parse_amount + apply_sign
  ↓
content_hash          ← sha1(acct + date + amt + desc + balance_after)
  ↓
INSERT OR IGNORE      ← UNIQUE(acct, bank_txn_id) + UNIQUE(acct, content_hash)
  ↓
detect_transfers      ← opposite-sign, ±$0.01, ±3 days, different accounts

Components

DB schema (SQLite, WAL)

One file: ~/Library/Application Support/personal-finance/finance.db. Migrations tracked via PRAGMA user_version; current version = 1.

Interface

CLI (personal-finance.py)

All subcommands read config from ~/.config/personal-finance/config.toml unless FINANCE_CONFIG or FINANCE_CONFIG_DIR is set. Entry point uses uv run --script (PEP 723 inline deps); no global pip install.

personal-finance.py import --account <id> <file.csv>
    Ingest a bank CSV export. Requires an [[accounts]] row matching <id>
    and a [csv_mappers.<institution>] block in config.toml. Archives the
    raw CSV under imports/<sha1>.csv. Dedups by (account_id, bank_txn_id)
    first, then (account_id, content_hash) — so two legitimate same-day
    duplicates both land when the bank exports a running balance column.
    Auto-runs detect_transfers after insert. Idempotent.

personal-finance.py review
    Apply all active category rules to NULL-category rows, then print the
    most-recent 20 uncategorized transactions to stdout + a total count.
    Safe to run repeatedly; first-write-wins semantics mean re-running does
    not re-categorize.

personal-finance.py rollup [--dry-run]
    Full nightly pipeline: apply_rules → snapshot_balances → month_summary
    → iMessage ping (if uncat>0 and ≥24h since last ping). Acquires a
    flock — concurrent runs are rejected. Launchd entry point.
    --dry-run prints the iMessage preview instead of sending AND does NOT
    record_ping (so a real run immediately after still fires).

personal-finance.py snapshot
    One-off balance snapshot without the rest of the pipeline. Writes one
    balance_snapshots row per configured account for today.

personal-finance.py doctor
    Print config path, DB path, imports dir, log dir, resolved [[accounts]]
    with mapper binding status, csv_mappers, db user_version, transaction
    /account/rule/snapshot counts, uncategorized count, bb-send.sh presence.

personal-finance.py unpair <id>
    Clear is_transfer/transfer_pair_id on a transaction + its pair partner.
    Use when the transfer detector falsely paired two unrelated rows.

Environment variables:

HTTP endpoints (served by webpage-server.py on :8899)

personal-finance does not run its own HTTP server. The sibling ../webpage-server/ project serves ~/www/ and hosts the /personal-finance* routes, loading the finance package modules on first request via a namespace-isolated _finance_load_all() that avoids colliding with newsfeed's own config.py / db.py / render.py / deliver.py.

The X-Finance-Token shared secret is reused from newsfeed's ~/Library/Application Support/newsfeed/feedback_token.txt — same file, different header, which keeps the secrets surface area small.

iMessage (via bb-send.sh to cfg.phone)

• Nightly rollup: Finance: $X spent this month. N transactions need categorizing: <review_url>. Sent only when uncategorized>0 AND ≥24h since last ping. Review URL resolves to the Tailscale Funnel (https://marks-mac-mini.tail20af9f.ts.net/personal-finance/review) unless [delivery] public_url_base is overridden to an explicit URL.

Dependencies

Python (PEP 723 inline, resolved by uv run)

Declared in personal-finance.py's script header. ./personal-finance.py <cmd> resolves everything via uv automatically — no global pip install needed.

jinja2>=3.1
tomli>=2.0 ; python_version < '3.11'

Stdlib-only for the core: sqlite3, csv, hashlib, dataclasses, argparse, logging, fcntl, subprocess, tomllib (3.11+).

External services

• BlueBubbles on localhost:1234 — iMessage delivery via /Users/mark/bin/bb-send.sh (shared with newsfeed).
• Tailscale Funnel — public HTTPS hostname used for the iMessage deep link. Neither daemon nor CLI is required by personal-finance itself; webpage-server.py binds the port.

External services (separate repos)

• webpage-server.py — lives in the ~/agents monorepo, required in prod. Serves ~/www/ and hosts the /personal-finance* routes. It loads THIS package by path (FINANCE_PKG=~/personal-finance) and dispatches every computation into it — the dashboard/review/api handlers there are thin HTTP plumbing, not finance logic. No cross-repo coordination beyond keeping that path pointed at ~/personal-finance.

Data sources

Phase 1: manual CSV exports from the bank's website. No scraping, no brokerage API, no Plaid. Mark downloads, runs personal-finance.py import --account <id> <file.csv>, dedup handles idempotency.

Phase 2+ (deferred — not built): - Plaid /transactions/sync, /investments/holdings/get. - Anthropic API (Claude Haiku) for LLM-assisted categorization of stubborn rows.

Configuration

All configuration lives under these paths. Secrets never commit.

Example — minimal config.toml

phone            = "+17076556006"
timezone         = "America/New_York"
default_currency = "USD"

[delivery]
public_url_base = "auto"     # "auto" | "lan" | explicit URL
webpage_port    = 8899

[[accounts]]
id          = "chase_checking"
name        = "Chase Checking"
kind        = "checking"        # checking|savings|credit|investment|realestate
institution = "chase"

[csv_mappers.chase]
date_col        = "Posting Date"
amount_col      = "Amount"
description_col = "Description"
txn_id_col      = "Transaction ID"   # optional; enables strong dedup
balance_col     = "Balance"          # optional; hardens content_hash
sign            = "debit_negative"   # or "debit_positive" for Amex-style

sign semantics: - debit_negative — most banks (Chase checking etc.): -50.00 in the CSV for a $50 charge. Pass through unchanged. - debit_positive — Amex and a few other credit cards: 50.00 for a $50 charge. Flip the sign so charges become negative.

See examples/config.toml for the authoritative template.

Category set is hard-coded in config.CATEGORIES: groceries, dining, transport, utilities, rent, entertainment, shopping, health, travel, income, tax, savings, other. create_rule validates against this list; extending it requires a code change + a fresh rule-engine pass.

Deployment

install.sh at this repo's root is the deploy entry point:

cd ~/personal-finance
./install.sh

It: 1. Symlinks ~/personal-finance/personal-finance.py into ~/bin/. 2. Copies examples/com.mark.personal-finance.plist to ~/Library/LaunchAgents/ (plists copied, not linked — launchd distrusts symlinked plists). 3. Seeds ~/.config/personal-finance/config.toml from examples/ only when the live copy is missing (hand-edits never clobbered). 4. Leaves ~/Library/Application Support/personal-finance/ alone — DB and imports are user data.

The dashboard is served by the webpage-server.py in the ~/agents monorepo, which loads this package by path (FINANCE_PKG=~/personal-finance). That path is the only coupling — keep it pointed here and the /personal-finance* routes keep working.

Launchd schedule

com.mark.personal-finance.plist runs personal-finance.py rollup at 22:00 local daily (StartCalendarInterval). RunAtLoad=false, KeepAlive=false, ProcessType=Background, Nice=5. FINANCE_CONFIG_DIR=/Users/mark/.config/personal-finance in EnvironmentVariables.

Uninstall

launchctl unload ~/Library/LaunchAgents/com.mark.personal-finance.plist
rm ~/Library/LaunchAgents/com.mark.personal-finance.plist
# Data survives — remove manually if desired:
# rm -rf ~/Library/Application\ Support/personal-finance

Operations

Run once, immediately

launchctl start com.mark.personal-finance          # via launchd
./personal-finance.py rollup                        # direct
./personal-finance.py rollup --dry-run              # preview without sending

Logs

tail -f ~/Library/Logs/personal-finance/finance-$(date +%Y%m%d).log
tail -f ~/Library/Logs/personal-finance/launchd.log

Healthcheck

./personal-finance.py doctor
# config path / db path / accounts / mappers / db user_version
# / counts (txn/accounts/rules/snapshots/uncategorized) / bb-send.sh status

Import a CSV

./personal-finance.py import --account chase_checking ~/Downloads/chase.CSV
# "12 inserted, 0 skipped (dup), 12 rows parsed from chase.CSV"

Create a categorization rule

# terminal
python3 -c 'import sys; sys.path.insert(0,"/Users/mark/personal-finance"); \
  import db, categorize; c=db.connect(); db.migrate(c); \
  print(categorize.create_rule(c,"STARBUCKS","dining"))'

# or via HTTP from the /review UI
#   POST /personal-finance/api/rule pattern=STARBUCKS category=dining

Unpair a false-positive transfer

./personal-finance.py unpair 42
# "unpaired 42 + 43"

Recent SQL snippets

sqlite3 ~/Library/Application\ Support/personal-finance/finance.db \
  "SELECT category, COUNT(*), ROUND(SUM(-amount),2) AS spent
   FROM transactions
   WHERE amount<0 AND is_transfer=0
     AND posted_date >= date('now','start of month')
   GROUP BY category
   ORDER BY spent DESC;"

Re-running after a bank column rename

Raw CSVs live under imports/<sha1>.csv. If a bank renames Amount to Transaction Amount, update the mapper in config.toml and re-import — dedup via content_hash stops the rows re-inserting.

Running the test suite

cd ~/personal-finance
uv run --python 3.11 --with jinja2 --with pytest python3 -m pytest tests/ -v
# 30 passed

Open questions

Resolved to ship Phase 1:

1. Which accounts first? — Chase checking + Chase savings. Add Amex by dropping a [[accounts]] block + a [csv_mappers.amex] block with sign = "debit_positive".
2. Category taxonomy. — Locked to the 13 values in config.CATEGORIES. Extending requires a code change; document it in the README if you add one.
3. Dashboard auth on LAN-only GETs. — Skipped per PLAN. GET HTML routes are unauthenticated (Tailscale trust boundary); writes and JSON reads require X-Finance-Token.

Still open (deferred to later phases):

1. Phase 1.5 LLM model — Claude Haiku 4.5 assumed; re-confirm merchant accuracy on real data before wiring.
2. Plaid yes/no — Non-trivial (dev approval, privacy policy, webhook verification). Defer until CSV-forever clearly insufficient.
3. Tax-export target — TurboTax CSV vs QuickBooks IIF vs plain Schedule-C mapped CSV. Decide before Phase 3.
4. Real-estate tracking — Manual set-balance only, or a Zillow scraper behind --zillow? Scrape is fragile; decide up-front if/when Phase 3 starts.