webpage-server

HTTP server that hosts ~/www/ on port 8899 and exposes the newsfeed JSON + TTS API. Threaded (ThreadingMixIn), standard-library only for the server itself, launched by launchd, and published to the internet via Tailscale Funnel.

The server is tightly coupled to the newsfeed project (its own standalone repo at ~/newsfeed): it reads the newsfeed SQLite DB, and imports newsfeed.config, newsfeed.tts, newsfeed.db, newsfeed.render, newsfeed.gnews_resolve, and newsfeed.url_resolve via sys.path injection of ~/newsfeed/.

Interface

All responses include: - Access-Control-Allow-Origin: * - Access-Control-Allow-Headers: Content-Type, X-Newsfeed-Token, X-Trade-Token, X-Finance-Token, X-Feedback-Token - Access-Control-Allow-Methods: GET, POST, OPTIONS - Cache-Control: no-cache (except /article/<id> → private, max-age=3600, and the Feller study-guide HTML at /ac54e507-...html → no-store, must-revalidate) - X-Robots-Tag: noindex, nofollow, noarchive

Directory listings are disabled (403 — 404 under the car library, see below). OPTIONS always returns 204.

Static files

GET /<path> — serves ~/www/<path>. Used for newsfeed.html, images, audio MP3s (/audio/<id>.mp3), and anything else dropped into ~/www/.

The car library (`~/www/280de112-a164-4fcf-9e42-861119cf47f7/`)

Study leaves published for the Tesla's in-dash browser — the same rendered bytes the iOS app bundles, served over the Funnel because the car can only reach a URL. No route is needed: the static fallthrough serves the leaves and their sibling mathjax/ and design/fonts/ asset dirs with correct MIME.

Two hygiene properties are load-bearing, because the UUID directory name is the access control (Funnel serves ~/www unauthenticated):

X-Robots-Tag: noindex, nofollow, noarchive — global, so it already covers the library; the leaves carry the matching <meta name="robots"> too.
A directory request under the prefix (…/, …/mathjax/) returns 404, not a generated listing and not the global 403. A listing would turn one leaked bearer URL into a table of contents of the whole library, and a 403 would still confirm the directory exists — the one bit a prober who guessed the prefix doesn't already have. The guard is a prefix test on the request path in Handler.list_directory, so nothing else under ~/www changes.

Pinned by tests/test_car_probe.py.

Private pages index (tailnet-only)

https://marks-mac-mini.tail20af9f.ts.net:8443/pages — a navigable index of every top-level ~/www/*.html page and every one-level-deep ~/www/<dir>/index.html page (title from <title>, name fallback; newest first, the two kinds merged into one ordering). Served by a SECOND listener inside the same process, bound to 127.0.0.1:$PAGES_INDEX_PORT and proxied via tailscale serve --https=8443 --set-path /pages (mounted by install.sh). It must NEVER be reachable through the public Funnel: UUID filenames are the only access control on ~/www pages, and a public index would enumerate them all. That's why it's a separate loopback port rather than a route on 8899 (the Funnel proxies all of 8899). Links are absolute to https://<host>/<file> (host from the request's Host header, port stripped) so they land on the Funnel origin that actually serves the pages.

The directory form exists because a library — one page with siblings it must sit beside — can't be a loose top-level file. The Truth Seeker car chapter library is exactly that: <uuid>/index.html with 41 chapter files and the mathjax/ + design/fonts/ symlinks next to it. Its row links to <dir>/ (trailing slash, so the static handler serves the index.html) and is labelled by the directory, not by a bare index.html that would read identically for every such page.

The walk stops at one level, and a directory page sorts on its index.html's own mtime, never the directory's. Both are load-bearing: ~/www is full of asset and payload trees (vendor/mathjax, mathjax, design/fonts, <slug>-ota) that a recursive walk would crawl slowly to list nothing a human navigates to; and a directory's mtime changes whenever any sibling is rewritten, so sorting on it would jerk the library to the top of the index on every unrelated re-render. A child directory with no index.html is simply skipped. Guard test: tests/test_pages_index.py.

`GET /<slug>-ota/<file>.ipa` and `GET /<slug>-ota/manifest.plist`

Backs the OTA-install skills (over-the-air install of dev builds to Mark's own phone, off-LAN over cellular — /riff-ota, /truth-seeker-ota). The handlers are generic: they serve any ~/www/<slug>-ota/ directory — currently riff-ota (via /riff-ota, written by riff-ota.sh) and truth-seeker-ota (via /truth-seeker-ota, written by truth-seeker-ota.sh) — deriving the .ipa filename and its Content-Disposition from the request path. (A regex restricts the slug to <name>-ota and the filename to a safe character class, so neither can traverse outside ~/www.) They need dedicated handlers because the stdlib static serve gets the manifest content-type wrong and has no Range support:

GET /<slug>-ota/<file>.ipa — the ad-hoc-signed app archive. Served as Content-Type: application/octet-stream with Accept-Ranges: bytes and 206 Partial Content on a Range request (resumable on flaky cellular — same byte-range logic as /audio/<id>.mp3). HEAD is also handled so iOS's install daemon sees the right Content-Length/Accept-Ranges before it downloads. 404 if the file is absent.
GET /<slug>-ota/manifest.plist — the itms-services manifest. Served as Content-Type: text/xml; charset=utf-8 (load-bearing: itms-services refuses an application/octet-stream manifest, which is what the stdlib default would send). 404 if absent.

install.html (the one-tap landing page) is a plain static file in the same dir — it falls through to the static handler, which serves .html as text/html correctly, so it needs no dedicated route.

`GET /api/items`

Paginated newsfeed items for a given run.

Auth: header X-Newsfeed-Token: <token> — must match the contents of ~/Library/Application Support/newsfeed/feedback_token.txt.

Query params: - run_id (int, required) — delivery run to page through - offset (int, default 0) — zero-based offset - limit (int, default 40, max 100)

Response: 200 application/json

{
  "items": [
    {
      "id": 12345,
      "title": "…",
      "summary": "…",
      "canonical_url": "…",
      "raw_url": "…",
      "resolved_url": "…",
      "source": "…",
      "bucket": "ai-research",
      "rank": 0,
      "short_host": "nytimes.com",
      "age": "3h",
      "topic_label": "AI Research",
      "thumbnail_url": "…",
      "source_initial": "N",
      "source_color": "hsl(217, 40%, 45%)",
      "kind": "article"
    }
  ],
  "total": 120,
  "has_more": true
}

For kind == "tweet" items, additional fields are passed through: tweet_author_handle, tweet_author_display_name, tweet_author_avatar_url, tweet_media, tweet_in_reply_to, tweet_retweet_of, tweet_body.

Errors: 400 (bad/missing run_id), 401 (bad token).

`POST /api/feedback`

Records a swipe gesture (block/boost) against a newsfeed item.

Auth: header X-Newsfeed-Token: <token>.

Body (application/json, max 4096 bytes):

{
  "direction": "block" | "boost",
  "canonical_url": "https://…",
  "source": "nytimes",
  "bucket": "world",
  "title": "…",
  "rank_position": 3,
  "run_id": 42,
  "item_id": 12345
}

Row inserted into swipe_feedback table in ~/Library/Application Support/newsfeed/newsfeed.db.

Response: 204 No Content on success.

Errors: 400 (missing direction/canonical_url/source, or invalid JSON), 401 (bad token), 413 (body too large / missing length), 500 (DB error).

`POST /api/tts`

Synthesizes (or returns cached) speech audio for a newsfeed item via ElevenLabs. MP3 is written to ~/www/audio/<item_id>.mp3 and served through the static handler.

Auth: header X-Newsfeed-Token: <token>.

Body: {"item_id": <int>} — must resolve to an existing row in items.

Response: - 200 {"url": "/audio/<id>.mp3", "cached": true|false} — synthesized or cache hit - 202 {"status": "pending"} — another request is synthesizing this item; client should poll - 400 {"error": "tts_not_available_for_tweets"} — tweets do not synthesize - 401 — bad token - 404 — item_id not in DB - 429 {"error": "rate limit"} — exceeded rate_limit_per_hour (default 50) - 429 {"error": "tts_budget_exceeded", "resets_at": "<iso8601>"} — daily character budget exhausted - 502 {"error": "tts_upstream_failed"} — ElevenLabs error - 503 {"error": "tts_unavailable" | "tts_disabled"} — no API key, module missing, or config disables TTS

Concurrency: O_EXCL lockfile at ~/Library/Application Support/newsfeed/tts-locks/<id>.lock prevents duplicate synthesis. Stale locks (>120s old) are reclaimed.

Rate limits: 1. Per-hour: [tts].rate_limit_per_hour (default 50) tracked in ~/Library/Application Support/newsfeed/tts_rate.json. 2. Per-day: [tts].daily_char_budget (default 50000) tracked in the tts_log table via newsfeed.db.

`GET /api/tts/status` (also accepts POST for symmetry)

Cheap probe for the front-end play button. Never calls ElevenLabs.

Response: 200 application/json

{
  "enabled": true,
  "budget": 50000,
  "used_today": 1284,
  "remaining": 48716
}

enabled is true iff TTS_ENABLED=1 is in the server's environment.

`GET /api/papers`

Daily arxiv picks from daily-paper-digest's history.json, newest-first. No auth (public arxiv links). Consumed by the Truth Seeker iOS Papers tab.

Response: 200 application/json

{"papers": [{"date": "...", "arxiv_id": "...", "title": "...",
             "url": "https://arxiv.org/abs/...", "hook": "..."}, ...]}

url is derived from arxiv_id; rows with a malformed id are dropped and duplicate ids are de-duped (newest wins). hook is omitted when absent. Missing/empty/dirty history → 200 {"papers": []} (never 500).

`GET /api/reports`

Per-topic evolving technical reports from daily-paper-digest's reports.json (a separate file from history.json). A report is a living synthesis of the state of the art for one narrow topic, rewritten by the daily job only when a finding genuinely advances understanding. No auth (public — it synthesizes already-public research). Consumed by the Truth Seeker iOS Papers tab → Reports view.

Response: 200 application/json

{"reports": [{"topic": "...", "title": "...", "updated": "YYYY-MM-DD",
              "markdown": "...", "sources": ["https://...", ...]}, ...]}

topic is the stable kebab id; title is the display string; updated is the ISO date of the last change; markdown is the full synthesis (may contain $…$ /$$…$$ math, rendered client-side) and is not capped; sources are the URLs that informed the current markdown. Rows with an empty topic are dropped and duplicate topics are de-duped (first wins). Order is producer-controlled (display order) — never re-sorted. Missing/empty/dirty store → 200 {"reports": []} (never 500).

`GET /api/podcasts/search?term=…`

Proxies the public iTunes Search API (media=podcast, limit=25) so the Podcasts page's search box never calls Apple directly — keeps the public Funnel from being an open iTunes proxy. Auth: header X-Newsfeed-Token: <token>.

Response: 200 application/json

{"results": [{"name": "...", "feed_url": "https://...",
              "artwork": "https://...", "artist": "..."}, ...]}

name is collectionName (falls back to trackName); artwork prefers the 600px art over 100px. Shows with no usable RSS feedUrl are dropped — you can't subscribe to a feed we can't fetch. Missing term → 400; no/bad token → 401; iTunes unreachable or garbled → 502 (never a partial/200).

`POST /api/podcasts/subscribe`

Appends one show to ~/.config/newsfeed/podcasts.yaml so Mark can subscribe from the page without hand-editing the file. Auth: header X-Newsfeed-Token: <token> — gated because the Funnel is public and this mutates his subscription list.

Body: {"name": str, "feed_url": "http(s)://…", "category"?: str} (category defaults to "podcast"; new shows get per_run_cap: 5).

Response: 200 application/json

{"ok": true, "added": true}                     # appended
{"ok": true, "added": false, "already": true}   # dup — no write

Dedupes by name (case-insensitive) or feed_url. Write is read→append→atomic-rename, mirroring resolve_podcast_feeds.py's safe_dump. A config that won't parse is never overwritten — the handler bails 500 ("podcasts config unreadable") and leaves the file byte-for-byte intact, so a corrupt/locked file can't cost Mark his shows. Malformed/missing name or non-http feed_url → 400; no/bad token → 401; body > 4 KiB → 413.

`POST /api/podcasts/unsubscribe`

Removes one show from ~/.config/newsfeed/podcasts.yaml — the Unfollow control in the podcast page's show-detail header. Auth: header X-Newsfeed-Token: <token> — gated for the same reason subscribe is.

Body: {"name": str} and/or {"feed_url": "http(s)://…"} — at least one required (400 if neither). An entry is dropped when its name matches case-insensitively or its feed_url matches.

Response: 200 application/json

{"ok": true, "removed": true}    # one+ entries dropped, file rewritten
{"ok": true, "removed": false}   # no match — file left untouched, no write

Same safety contract as subscribe: the rewrite is atomic temp-write→rename via yaml.safe_dump, and a config that won't parse is never overwritten — the handler bails 500 ("podcasts config unreadable") and leaves the file byte-for-byte intact. Never truncates: a no-match request performs no write at all. No/bad token → 401; body > 4 KiB → 413. Tests: tests/test_podcasts_endpoints.py (the unsubscribe suite pins byte-for-byte preservation on no-match / unparseable / unauthorized).

`GET /article/<item_id>`

Renders a reader view for a single item. HTML output. Used by the swipe UI as the "open" target.

Pipeline: 1. Load the item from newsfeed.db. 2. If the URL is a Google News redirect, resolve it via gnews_resolve or search by title via url_resolve.search_article_url. 3. Fetch/extract full article markdown via tts.fetch_full_article_md with on-disk cache at ~/.cache/newsfeed/articles/<sha1>.md. 4. Render to HTML via newsfeed.render.render_article and return.

Response: 200 text/html; charset=utf-8 with Cache-Control: private, max-age=3600. 400 on bad id, 404 if item not found, 500 on DB/render error.

`POST /api/study-feedback`

Highlight-to-update feedback loop for the Feller Vol. II study guide. Mark highlights a passage in the rendered HTML at /ac54e507-...html, the in-page modal sends the passage + question here, the server drops an event for the study-feedback poll session, which uses the Edit tool to weave an inline answer into ~/reading/feller-vol-2/ch01/study-guide.md; the server then re-renders the HTML and the page reloads. Synchronous from the caller's view — it polls a done-file up to 240s (~30-90s typical). flock single-flight at ~/.cache/feller-feedback.lock. History at ~/reading/feller-vol-2/feedback.log (JSON lines).

Auth: header X-Feedback-Token: <token> (same token file as X-Newsfeed-Token, distinct header).

Body: application/json

{"passage": "<verbatim selection from the rendered page>",
 "question": "<what to weave in>"}

Responses: - 200 {"ok": true, "url": "<funnel URL>"} — markdown edited, HTML re-rendered. - 400 {"ok": false, "error": "bad request: ..."} — missing/oversize fields. - 401 {"error": "unauthorized"} — bad/missing token. - 409 {"ok": false, "error": "another update in progress, retry shortly"} — flock collision. - 413 {"error": "bad length"} — body empty or > 16 KB. - 422 {"ok": false, "error": "Couldn't locate that passage..."} — Claude couldn't find the passage; user adjusts selection and resubmits. - 500 {"ok": false, "error": "Claude error: ..."} / "Render failed; rolled back" — claude exit ≠ 0 or renderer failure (markdown rolled back from /tmp/study-guide-<ts>.bak). - 504 {"ok": false, "error": "Update timed out"} — claude exceeded 240s.

The study-guide HTML path overrides the global Cache-Control: no-cache to no-store, must-revalidate so iOS Safari bfcache can't serve a stale page after a successful edit.

Worker code: ~/webpage-server/study_feedback.py. Renderer: ~/reading/feller-vol-2/render.py. Verbatim chapter at ~/reading/feller-vol-2/ch01/chapter.md is read-only; the prompt template forbids editing it.

Verbatim comment threads (reply-only)

The Feller "Verbatim" leaf supports the same highlight→ask gesture as the study guide, but Claude's answer surfaces as a comment thread anchored to the passage — the verbatim transcription is never mutated. Thread data lives only in an isolated store at ~/Library/Application Support/feller-feedback/verbatim-threads.json; no route here edits content/feller-vol-2/ch01/chapter.md or the rendered HTML.

The thread model is cherry-picked from the prose plugin (op model + locate() + CSS Custom Highlight + suggestion fence). Two endpoints mirror prose's /submit (ask) vs /threads (pure op) split:

Sidecar schema (additive over prose): {schema_version, threads:[{id:"t<n>", anchor:{scope:"text"|"file"|"document", file, section, quote, prefix?, suffix?}, status:"open"|"resolved", book, chapter, leaf, created_ts, updated_ts, messages:[{author:"reviewer"|"claude", body, html, ts}]}]}.
Op model: {op:"add"|"reply"|"edit"|"status"|"delete", id?, anchor?, body?, author?, index?, status?, book?, chapter?, leaf?}.
Auth: header X-Feedback-Token: <verbatim token>, a dedicated token at ~/Library/Application Support/feller-feedback/verbatim-token.txt (minted by render.py, distinct from the study token so the two leaves' lifecycles don't disturb each other). GET also accepts ?token= as a fallback.

`POST /api/verbatim-feedback`

The ask path (prose /submit, fused with add). Does an add (new thread) or reply (follow-up via thread_id) carrying the reviewer's question, persists the reviewer turn to the store before dispatch, then runs the reply-only worker; appends Claude's answer as a claude message. Synchronous; polls a done-file up to 240s. flock at ~/.cache/verbatim-feedback.lock.

Body: {book, chapter, leaf, anchor:{scope,file,section,quote,prefix?, suffix?}, question, thread_id?, context?} (≤ 64 KB). context (optional, ≤ 40000 chars) is a document's markdown used as the worker's READ-ONLY reference instead of the Feller chapter — see "Document-keyed reuse for reports" below. A verbatim ask omits it.

Responses: 200 {"ok":true,"thread":{...}} (thread with the claude message appended), 400 bad request, 401 bad/missing token, 409 flock collision, 413 body empty/oversize, 500 Claude/render error, 504 {"error":"Mac isn't answering — is the verbatim-feedback poll session running?"} on 240s timeout (the reviewer turn is persisted first, so the thread is recoverable). Worker: ~/webpage-server/verbatim_feedback.py; poll session: verbatim-feedback.

`POST /api/verbatim-threads`

The pure-op path (prose /threads). Applies ONE op (status resolve/reopen, delete, edit, or a bare reply reviewer comment) with NO Claude dispatch — so lifecycle actions never pay the 240s round-trip or need a live poll session.

Body: one op object (≤ 16 KB).

Responses: 204 for ops returning nothing (delete/status/edit/reply, mirror prose returning None), 200 {"ok":true,"thread":{...}} for add, 400 bad op / validation, 401 bad/missing token, 404 unknown thread id, 413 body empty/oversize. Store module: ~/webpage-server/verbatim_threads.py.

`GET /api/verbatim-threads?book&chapter&leaf`

Read the sidecar threads for a leaf (newest-first by updated_ts), filtered by any provided coords. Auth via header or ?token=. Read-only.

Responses: 200 {"ok":true,"threads":[...]}, 401 bad/missing token.

CORS for all three routes rides the existing global headers (Access-Control-Allow-Origin: *, X-Feedback-Token allowlisted, do_OPTIONS→204), so the bundled file:// app leaf (null origin) can fetch them. A regression test in tests/test_verbatim_endpoints.py pins this.

Document-keyed reuse for reports

These same three endpoints are document-keyed by (book, chapter, leaf) — nothing is Feller-specific in the store, the handlers, or the token. The daily Reports leaf reuses them directly with coords book="reports", chapter=<topic-id>, leaf="report" (no parallel /api/report-threads, no coord allowlist — the token gates writes). A report ask sends the report's own markdown in the context field; the worker writes it to a temp file and uses THAT as the prompt's READ-ONLY reference (and points the poll session's add_dir at the temp dir) instead of the Feller chapter.md. context is treated as UNTRUSTED data (same posture as the passage/question) and capped at 40000 chars (larger → 4xx). Reports share the same verbatim-token.txt. tests/test_verbatim_endpoints.py + tests/test_verbatim_feedback.py mirror the verbatim suites for report coords and pin the context-vs-chapter reference selection.

`POST /api/car-probe`

Intake for the Tesla browser's capability report. The car has no devtools, no console, and no way to copy text out, so the probe page's only channel back is to POST what it found — that's the whole reason this endpoint exists. Used to settle what the in-dash Chromium actually supports (notably contenteditable="plaintext-only", which the composer depends on) instead of guessing at its version.

Auth: X-Feedback-Token header or ?token= — the same verbatim-token.txt the leaves use. Rate-limited at 30/hour via its own state file (feller-feedback/car-probe-rate.json) so probe spam can't eat the ask budget.

Body: the report, any JSON object, ≤ 8 KB. Stored verbatim (the submitted bytes, unparsed) at ~/Library/Application Support/truth-seeker/car-probe/<utc>.json. The filename is colon-free basic ISO-8601 with microseconds — a : reads as / in Finder, and microseconds keep two probes in the same second from colliding.

Responses: 200 {"ok":true,"saved":"<filename>"}, 400 unparseable or non-object body, 401 bad/missing token, 413 empty/oversize, 429 over the cap, 500 the store is unwritable. Tests: tests/test_car_probe.py.

Test-mode endpoints (gated by `NF_TEST_MODE=1`)

Used by the playwright harness in ~/newsfeed/tests/.

GET /api/test-report — read JSONL records from /tmp/newsfeed_test_report.jsonl; supports scenario and since query params.
POST /api/test-report — append a record to the same file.
GET /api/test-bus — pop the next pending command for the scenario.
POST /api/test-bus — enqueue a command.
GET /tests/test_harness.js — serves ~/newsfeed/tests/test_harness.js.
GET /test-ruler — 10×10 red square calibration page.
GET / and GET /newsfeed.html — when ?test=1&scenario=<name>, injects the harness script + a <meta name="nf-test-scenario"> tag into the page.

These endpoints 404 when NF_TEST_MODE is unset.

`GET /projects`

HTML index of the projects named in the PROJECT_SLUGS allowlist in webpage-server.py — each rendered from ~/<slug>/README.md. No auth. Sorted by README mtime descending. The allowlist (not a glob of $HOME) is the source, so adding a project means appending its slug there. Template: ~/webpage-server/templates/projects_index.html.j2. Slugs are still validated against ^[a-z0-9_-]+$ as defense-in-depth.

`GET /projects/<slug>`

Renders ~/<slug>/README.md to HTML with a sticky TOC sidebar on desktop, monokai pygments syntax highlighting, and a mermaid loader for mermaid fenced blocks. No auth. Slugs must be in PROJECT_SLUGS and match ^[a-z0-9_-]+$; non-allowlisted slugs and missing READMEs both return 404 (never 500). Template: ~/webpage-server/templates/project_spec.html.j2.

Config

Environment variables

Var	Default	Effect
`WEBPAGE_PORT`	`8899`	Bind port.
`PAGES_INDEX_PORT`	`8898`	Loopback bind port for the private pages index (see Interface).
`TTS_ENABLED`	unset	Reported by `/api/tts/status` as `enabled=true` iff `"1"`. Does not gate `/api/tts` directly — that's handled inside `newsfeed.tts`.
`NF_TEST_MODE`	unset	When `"1"`, exposes `/api/test-*`, `/tests/test_harness.js`, `/test-ruler`, and injects the harness into `/newsfeed.html`.

Config file

Loaded lazily per request via newsfeed.config.load().tts. The TOML block lives in the newsfeed project's config file and supports at minimum:

rate_limit_per_hour (int, default 50)
min_chars (int, default 120)
daily_char_budget (int, default 50000)
daily_char_budget_testing (int, default 5000)
testing (bool) — when true, use daily_char_budget_testing

Token file

~/Library/Application Support/newsfeed/feedback_token.txt holds the stable token for X-Newsfeed-Token. Written by newsfeed.deliver.ensure_feedback_token() at publish time — minted once, reused thereafter (never rotated per run; rotation 401'd pages the Truth Seeker app keeps loaded). The server re-reads on every auth check (no cache). If the file is missing, all token-protected endpoints return 401.

Dependencies

Python

Server itself: standard library only (http.server, socketserver, sqlite3, json, secrets, zoneinfo).

The newsfeed package (standalone repo at ~/newsfeed, imported via sys.path insertion of ~/newsfeed/): - config — loads [tts] block - tts — clean_for_tts, synthesize, fetch_full_article, fetch_full_article_md, render_article_html, TTSDisabled, TTSError, BudgetExceeded - db — connect, tts_chars_used_today - render — render_article - gnews_resolve — is_google_news, resolve - url_resolve — search_article_url

External services

Tailscale Funnel on port 8899 for public HTTPS: tailscale funnel 8899. Public URL: https://marks-mac-mini.tail20af9f.ts.net/.
ElevenLabs API for TTS synthesis (key + voice configured inside newsfeed.tts).

Filesystem

~/www/ — served as document root (content written by other processes; newsfeed.html by the newsfeed digest job, audio MP3s by this server's /api/tts).
~/www/<slug>-ota/ — <App>.ipa + manifest.plist + install.html for the OTA-install skills, served by the generic handlers above:
~/www/riff-ota/ — written by ~/riff/skills/riff-ota/riff-ota.sh.
~/www/truth-seeker-ota/ — written by ~/truth-seeker/skills/truth-seeker-ota/truth-seeker-ota.sh (the truth-seeker app is a standalone repo; this server only serves its OTA dir).
~/www/280de112-a164-4fcf-9e42-861119cf47f7/ — the car library (Study leaves for the Tesla browser, written by the truth-seeker repo's render pipeline).
~/Library/Application Support/truth-seeker/car-probe/<utc>.json — capability reports posted by the car probe page (POST /api/car-probe).
~/Library/Application Support/feller-feedback/car-probe-rate.json — the probe's own hourly rate window.
~/Library/Application Support/newsfeed/newsfeed.db — items, swipe_feedback, tts_log.
~/Library/Application Support/newsfeed/feedback_token.txt — auth token.
~/Library/Application Support/newsfeed/tts-locks/ — per-item lockfiles.
~/Library/Application Support/newsfeed/tts_rate.json — hourly rate window.
~/.cache/newsfeed/articles/<sha1>.{md,txt} — article extraction cache.
~/Library/Logs/webpage-server.log — launchd stdout+stderr.

Testing

Two lanes — a hermetic ship-gate (default) and a live integration smoke (opt-in):

`./verify.sh` — hermetic ship-gate (run before every build)

Runs the full tests/ suite (pip-free, /usr/bin/python3 -m pytest), a py_compile syntax check, a de-leak (no stray ~/agents path literals), and a repoint assertion (import roots resolve outside ~/agents). Offline and zero-cost — every test monkeypatches or stubs its externals; no server, no network, no Claude. This is the gate that must be green before shipping.

The live e2e test below is skipped in this lane (its module-level skipif keys on TS_E2E, unset here), so verify.sh reports it as 1 skipped and never burns API or depends on a running server.

`./e2e.sh` — live ask→answer smoke (opt-in, costs API)

The regression net for "poll session wedged" — the failure that took the Truth Seeker core feature dark for 13.5h on 2026-06-06 (a verbatim-feedback session hit API Error: 529, dropped its watch loop to an idle prompt while the process stayed alive, and every ask timed out at 504). The hermetic test_verbatim_feedback.py monkeypatches _run_claude to a canned reply, so it structurally cannot catch a wedged session; the in-app UITests drove a synthetic seam and never touched the network. e2e.sh closes that gap: it does a real authenticated POST /api/verbatim-feedback against the live server, which dispatches to the live poll session, and asserts a real non-empty answer about Feller eq (1.1) comes back inside the 260s deadline. It then deletes the thread it created (the delete pure-op) to keep verbatim-threads.json clean.

Preconditions and failure shapes are deliberately distinct:

token file or server absent → SKIP (environmental, not a failure).
server + session up but timeout / 504 / empty answer → FAIL — this is the regression. A wedged poll session turns the test RED on assert status == 200 (the 504 from verbatim_feedback.process's timeout branch). The red path is validated-by-construction: that timeout branch returns 504, which fails the assert; we do not take the live service down to demo it.

Requires the server on :8899 (WEBPAGE_PORT) and a healthy verbatim-feedback poll session (see ~/poll-bringup — the watchdog there auto-recovers a wedged session). Token read at runtime from ~/Library/Application Support/feller-feedback/verbatim-token.txt; it is never printed. The test creates a fresh thread each run (no thread_id) so reruns are independent.

Deployment

install.sh

From the repo root ~/webpage-server/:

./install.sh

This: 1. Symlinks ~/webpage-server/webpage-server.py into ~/bin/webpage-server.py (the launchd plist runs /usr/bin/python3 /Users/mark/bin/webpage-server.py, so repointing the symlink is the cutover). 2. Copies LaunchAgents/com.mark.webpage-server.plist into ~/Library/LaunchAgents/. (Copy, not symlink — launchd distrusts symlinked plists.)

launchd plist

~/Library/LaunchAgents/com.mark.webpage-server.plist:

Label: com.mark.webpage-server
ProgramArguments: /usr/bin/python3 /Users/mark/bin/webpage-server.py
RunAtLoad: true
KeepAlive: true
Logs: ~/Library/Logs/webpage-server.log (stdout + stderr merged)

To (re)load after editing the plist:

launchctl bootout  gui/$UID ~/Library/LaunchAgents/com.mark.webpage-server.plist
launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.mark.webpage-server.plist

Editing webpage-server.py alone is live immediately via the symlink — but because KeepAlive=true the process won't pick up the change until it's restarted. Kick it with:

launchctl kickstart -k gui/$UID/com.mark.webpage-server

Tailscale Funnel

Public exposure on port 8899 (once per machine, persists across reboots):

tailscale funnel --bg 8899

Verify with tailscale funnel status.

Usage examples

Read the current token

TOKEN="$(cat "$HOME/Library/Application Support/newsfeed/feedback_token.txt")"

Page through today's items

curl -s -H "X-Newsfeed-Token: $TOKEN" \
  "http://localhost:8899/api/items?run_id=42&offset=0&limit=20" | jq .

Submit swipe feedback

curl -s -X POST \
  -H "X-Newsfeed-Token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"direction":"boost","canonical_url":"https://example.com/a","source":"example","bucket":"tech","rank_position":0,"run_id":42}' \
  http://localhost:8899/api/feedback -o /dev/null -w "%{http_code}\n"
# 204

Trigger TTS for an item

curl -s -X POST \
  -H "X-Newsfeed-Token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"item_id": 12345}' \
  http://localhost:8899/api/tts | jq .
# {"url":"/audio/12345.mp3","cached":false}

Then fetch the MP3:

curl -s http://localhost:8899/audio/12345.mp3 -o /tmp/12345.mp3

Check TTS budget

curl -s http://localhost:8899/api/tts/status | jq .

Render an article

open "http://localhost:8899/article/12345"

Watch logs

tail -f ~/Library/Logs/webpage-server.log

Files

webpage-server.py — the server (symlinked to ~/bin/webpage-server.py).
../LaunchAgents/com.mark.webpage-server.plist — launchd unit.
../newsfeed/ — required sibling package.

webpage-server

Interface

Static files

The car library (~/www/280de112-a164-4fcf-9e42-861119cf47f7/)

Private pages index (tailnet-only)

GET /<slug>-ota/<file>.ipa and GET /<slug>-ota/manifest.plist

GET /api/items

POST /api/feedback

POST /api/tts

GET /api/tts/status (also accepts POST for symmetry)

GET /api/papers

GET /api/reports

GET /api/podcasts/search?term=…

POST /api/podcasts/subscribe

POST /api/podcasts/unsubscribe

GET /article/<item_id>

POST /api/study-feedback

Verbatim comment threads (reply-only)

POST /api/verbatim-feedback

POST /api/verbatim-threads

GET /api/verbatim-threads?book&chapter&leaf

Document-keyed reuse for reports

POST /api/car-probe

Test-mode endpoints (gated by NF_TEST_MODE=1)

GET /projects

GET /projects/<slug>

Config

Environment variables

Config file

Token file

Dependencies

Python

External services

Filesystem

Testing

./verify.sh — hermetic ship-gate (run before every build)

./e2e.sh — live ask→answer smoke (opt-in, costs API)

Deployment

install.sh

launchd plist

Tailscale Funnel

Usage examples

Read the current token

Page through today's items

Submit swipe feedback

Trigger TTS for an item

Check TTS budget

Render an article

Watch logs

Files

The car library (`~/www/280de112-a164-4fcf-9e42-861119cf47f7/`)

`GET /<slug>-ota/<file>.ipa` and `GET /<slug>-ota/manifest.plist`

`GET /api/items`

`POST /api/feedback`

`POST /api/tts`

`GET /api/tts/status` (also accepts POST for symmetry)

`GET /api/papers`

`GET /api/reports`

`GET /api/podcasts/search?term=…`

`POST /api/podcasts/subscribe`

`POST /api/podcasts/unsubscribe`

`GET /article/<item_id>`

`POST /api/study-feedback`

`POST /api/verbatim-feedback`

`POST /api/verbatim-threads`

`GET /api/verbatim-threads?book&chapter&leaf`

`POST /api/car-probe`

Test-mode endpoints (gated by `NF_TEST_MODE=1`)

`GET /projects`

`GET /projects/<slug>`

`./verify.sh` — hermetic ship-gate (run before every build)

`./e2e.sh` — live ask→answer smoke (opt-in, costs API)