riff

← Home · ~/riff · updated 5 days ago

riff

Dictation on your own ElevenLabs subscription. Two apps, one job: hold your voice and put the text at the cursor.

iOSRiff + RiffKeyboard A system keyboard you enable in any app. Tap the mic, talk, the transcript types itself at the cursor. Wispr-Flow parity: the mic stays warm, no per-dictation app switch.
macOSRiffDictate A menu-bar daemon. ⌥Space starts, ⌥Space again stops, the transcript is pasted into the frontmost app. See macos/README.md.

Both transcribe with ElevenLabs Scribe v2. macOS calls ElevenLabs directly. iOS goes through a small server on your own Mac (server/riff_server.py), because a distributed .ipa would ship the API key to every tester.

Riff used to be an SSH terminal with a chat client, a share extension and a weather widget bolted on. All of that was subtracted on 2026-07-29; what is left is the two keyboards.

The iOS keyboard (RiffKeyboard)

Goal: a custom keyboard that dictates Riff's voice transcription into any app (Notes, Mail, Slack, a browser field), transcribed on your own ElevenLabs subscription, with Wispr-Flow parity (the mic stays warm, no per-dictation app switch).

Status (build 393): all four slices are on-device. The host holds a warm background mic, the keyboard's Darwin trigger arms it from any app, the host uploads the clip to /riff/transcribe-only, and the keyboard types the returned transcript at the cursor.

The platform constraint that shapes everything: a keyboard extension cannot record the microphone, even with Full Access. Measured in the build-278 probe: AVAudioEngine.start() inside the appex fails with coreaudio 2003329396 ('what') and AVAudioRecorder.record() is refused. Apple Developer Forum threads 742601 / 775077 / 800500 confirm it is sandbox policy, not a bug. So the keyboard is only the trigger (a cross-process wake) and the text inserter (textDocumentProxy); the host app owns the mic. This is how Wispr Flow works.

The architecture:

  • Host warm mic (WarmMic). The host keeps the mic delivering live audio while backgrounded, with no CallKit and no foreground bounce (both Mark-vetoed). An AVAudioEngine input-node tap runs over an AVAudioSession PlayAndRecord / .default / [.defaultToSpeaker] session (Bluetooth recording off) with setAllowHapticsAndSystemSoundsDuringRecording(true), under the host's UIBackgroundModes: [audio]. The engine is started in the foreground and survives backgrounding; AudioSessionRecovery (a pure, unit-tested policy) restarts it through the backgrounding route-change so a recoverable event is not a false failure. The tap drives an ActivityKit Live Activity (RiffWidget, Dynamic Island + Lock Screen), so a frozen timer or level bar after backgrounding would mean the mic stopped: the proof-of-life.
  • Keyboard trigger. The keyboard, with Full Access, wakes the warm backgrounded host with a Darwin notification (CFNotificationCenterGetDarwinNotifyCenter, name mark.riff.dictate.toggle) to begin or end dictation in place, while the user stays in the target app. Darwin notifications carry no payload and need no entitlement or App Group; they reach only a running process, and the host stays running-active backgrounded precisely because it holds the audio endowment. The pure trigger logic (DictationBridge.action(warm:dictating:), which never begins dictation unless the engine is warm) is unit-tested; the live Darwin wire is device-only. The Live Activity is created at arm (foreground) and the keyboard toggle only updates it (DictationLifecycle.activityCommand(for:)), because ActivityKit refuses to start an activity from the background. Closing the app ends it. A Live Activity outlives its app by design (ActivityKit's own ceiling is ~8h island / 12h lock screen), so force-quitting used to leave a waveform with no process behind it; applicationWillTerminate now ends every one through ActivityTeardown, which blocks briefly for the async end to land because the process is being killed the moment it returns. A relaunch reconciles a fresh activity at the next foreground arm, so the warm-idle liveness invariant below is untouched. Closing means TERMINATION, never backgrounding: no scene phase may stop the mic (WarmMicPhasePolicy, a two-case map with no "stop" to reach for). Build 399 stopped it on .background and killed out-of-app dictation outright — dictating from inside another app IS the backgrounded case.
  • Transcription. On stop, the host closes the clip and uploads it to the signed /riff/transcribe-only endpoint (HMAC on the host). A clip too short to be speech is dropped (DictationClip.worthTranscribing); a generation token discards a stale result so an in-flight upload can't clobber a newer dictation.
  • Text injection. The host writes the transcript to an App-Group file (DictationHandoff) and pings the Darwin center; the keyboard reads it, inserts it at the cursor via textDocumentProxy, and consumes it (consume-on-insert dedupe). The transcript never appears on the Dynamic Island: output goes only to the target text box.

The secret-less invariant (load-bearing). The keyboard NEVER holds RIFF_SHARED_SECRET and NEVER makes the network call: the host does both. Shared types compiled into the appex import only Foundation / CoreFoundation, so no RiffClient / CryptoKit / HMAC code is dragged in. /riff-ota and /riff-publish grep the built .appex for secret keys and otool/strings-check the binary for crypto and transcribe symbols. A distributed .ipa unzips to a keyboard that cannot leak the secret.

One-time setup (per device). Settings → General → Keyboard → Keyboards → Add New Keyboard… → Riff, then tap Riff and enable Allow Full Access (needed to post the Darwin trigger and read the App-Group handoff).

Battery — the warm-idle invariant

The warm mic runs 24/7 and the Live Activity stays alive around the clock, so any recurring work on the warm-idle path is an overnight drain multiplied by ~8 dark hours. CLAUDE.md holds the full invariant and is the authority; the short version is zero Live Activity pushes from a quiet room, dark cadences no faster than darkInterval (1s), and no new periodic work without a screen-dark gate. Pinned by WarmMicPushBoundaryTests, WarmMicPushPolicyTests, WarmMicScreenGateTests, WarmMicHeartbeatTests.

Action Button — Riff App Shortcut (App Intent)

The Action Button binds to an AppIntent (RiffToggleIntent) auto-registered via AppShortcutsProvider (RiffShortcuts in RiffIntents.swift). iOS runs perform() in the app's process (foregrounding it via openAppWhenRun = true), which re-runs deterministically on every press. It posts .riffToggle, which ContentView turns into start-or-stop on the warm mic — the same path the in-app disc and the keyboard's Darwin trigger converge on.

Why an App Intent, not riff://toggle: the URL scheme's .onOpenURL does not reliably re-fire when the app is already foregrounded, so a second press did nothing once Riff was open. The URL is retained as a zero-cost fallback and posts the same notification.

Set it up: Settings → Action Button → swipe to ShortcutChoose a Shortcut → pick the auto-registered Riff shortcut. There is no Shortcut to author by hand.

Locked-screen Face ID (OS limit). iOS requires a Face ID / passcode unlock to launch Riff from a locked screen, even via the App Intent. Not a Riff bug, and not engineered around.

Architecture

Why server-side cloud STT (ElevenLabs Scribe v2)

Decision (2026-05-22): transcription happens server-side via ElevenLabs Scribe v2, not on-device. The phone records audio and uploads it; the Mac transcribes it. This replaced the original on-device SFSpeechRecognizer path.

Two on-device failures forced the swap:

  1. The ~60s on-device session ceiling. SFSpeechRecognizer caps a single one-shot transcription at ~1 minute, which broke the "unlimited-length dictation" goal outright.
  2. Accuracy on technical jargon. On-device Speech mangled terms like "Kalshi", "Hyperliquid", "git rebase" — the exact vocabulary Mark dictates most.

Scribe v2 is the best mainstream STT API (~2.2% WER; the only other serious candidate was OpenAI gpt-4o-transcribe at ~4.1%). The trade-off is explicit and accepted: an audio upload + a network STT round-trip (and the latency/cost that implies) in exchange for materially better accuracy and no on-device model-download UX. Confirmed batch model id: scribe_v2 (verified live against POST /v1/speech-to-text, 2026-05-22). The realtime variant is the distinct scribe_v2_realtime (not used). Overridable via ELEVENLABS_STT_MODEL in ~/.env; fallback id is scribe_v1.

Batch, not streaming. The whole clip uploads on stop and the server transcribes it in one Scribe call. There is no live transcript while speaking — the LED waveform shows the mic is live, then all-lit means transcribing, then the text lands. Streaming (scribe_v2_realtime over a WS relay) is a materially bigger build and a pricier model (~$0.39/hr vs ~$0.22–0.28/hr batch); spec it if the missing live preview ever bothers anyone, not now.

Privacy delta

Audio leaves the device — to the Mac, then to ElevenLabs. The /riff/transcribe-only endpoint stays tailnet-only (no Funnel), so the audio→Mac leg never touches the public internet; only the Mac→ElevenLabs leg does (over TLS). On macOS the audio goes straight from the laptop to ElevenLabs; there is no server hop and no tailnet involved.

Why Tailscale, not Funnel

Funnel is internet-public; Riff would expose a microphone-attached endpoint to the world. Tailscale ACL keeps the endpoint reachable only from Mark's devices on the tailnet — the iPhone is already on it. That's the auth layer: device on tailnet = trusted. Layer a single shared HMAC secret on top so a stolen iPhone with the tailnet still joined can be revoked by rotating the secret server-side.

Cost & latency (Scribe v2)

  • Cost: Scribe v2 batch$0.22–0.28 per hour of audio. For dictation — seconds to a couple of minutes at a time — this is fractions of a cent per request; a 30-second clip ≈ $0.002.
  • Latency: upload + STT. The upload is a small AAC clip over the tailnet (sub-second for typical clips); Scribe v2 batch returns a short clip in a few seconds. That is the whole wait: there is nothing downstream of the transcript.

The Mac server

server/riff_server.py, an aiohttp app on :8903, bound to the tailnet. Two endpoints, both requiring X-Riff-Secret = HMAC-SHA256(body, RIFF_SHARED_SECRET) in hex:

endpoint body returns
GET /riff/health empty {"ok": true, "uptime_s": N}
POST /riff/transcribe-only raw audio bytes (Content-Type picks the format) {"transcript": "…"}

Bad or missing signature → 401. Body over the per-path cap → 413 (MAX_AUDIO_BODY = 25 MB on the audio path, MAX_BODY = 8 KB everywhere else). Missing ELEVENLABS_API_KEY → 503 on transcribe, while health stays up. See server/QUICKSTART.md.

Why 8903, not 8902. Wayfinder's server (~/wayfinder) owns 8902 on the same Mac. Riff briefly rode on it — Wayfinder served a compat /riff/* prefix — but the two are independent deployments: their own port, HMAC secret (RIFF_SHARED_SECRET vs WAYFINDER_SHARED_SECRET), LaunchAgent, and log file. The single thing they share is the ELEVENLABS_API_KEY in ~/.env. PORT in riff_server.py is the only source of truth; a server test pins every shipped script's health URL to it, and Settings.port on the iOS side is a compile-time constant, so moving the port means a new build.

Assets — the app icon and the in-app glyph

One mark, everywhere: a Shure 55, drawn by tools/logo/shure55.html. The app icon and the in-app glyph are two renders of the SAME tools/logo/riff-mark.svg.

The app icon — Riff.icon

An Icon Composer document at the repo root. ios/project.yml lists ../Riff.icon as a source and sets ASSETCATALOG_COMPILER_APPICON_NAME: Riff; there is no AppIcon.appiconset any more — actool compiles the .icon into the Liquid Glass icon stack AND the legacy flat sizes older OS versions fall back to. RiffDictate carries no app icon: it is LSUIElement, so it has no Dock tile and draws the menu-bar glyph instead.

One layer, white on nothing: the fill is fully transparent, so the system supplies the glass, the shadow and the tint. Edit icon.json (translucency, shadow, supported platforms) in Icon Composer, or by hand — it's small.

The layer PNG is not hand-maintained. Its source is the mic itself:

tools/logo/shure55.html     # the model, the 3-D viewer and the SVG exporter, one file
tools/logo/riff-mark.svg    # what Save wrote: white shapes over black ones
./tools/logo/render-icon.swift   # → Riff.icon/Assets/mark.png + Shared/Marks.xcassets

Re-pose the mic in the page, Save, drop the export over riff-mark.svg, run the script, commit both. The script is a swift shebang — no Xcode project, no signing, so it runs over SSH.

The black in the export is not ink: it's the occluder that cuts the grille slots and hollows the shell, so the layer can't just drop it. render-icon.swift renders the mark as exported onto black, then reads that back as coverage — how white a pixel came out is how opaque it becomes. White ink, everything else clear.

The in-app glyph — RiffMark

RiffMark(ink: …) — the host app's dictate disc, the keyboard's DICTATE button and the Dynamic Island, all through DictationDisc.

The artwork is RiffMic in ios/Shared/Marks.xcassets, a template image, so ink tints the whole thing in one go. That is exactly why the asset is white on NOTHING rather than white on black: a template keeps alpha and throws colour away, so the grille slots and the hollow shell have to be genuinely transparent or the tint fills them in. render-icon.swift writes this asset from the same SVG, tight and at its own aspect (560×1024) instead of squared with the icon margin.

The catalog is a source of the app, the keyboard AND the widget, so Image("RiffMic") resolves in whichever bundle is running.

Testing

A simulator-only Swift Testing suite (ios/RiffTests) — no physical device, no network, no ElevenLabs. import Testing (@Test/#expect), not XCTest; the bundle is hosted in Riff.app (TEST_HOST/BUNDLE_LOADER) so @testable import Riff resolves.

Policy: every change adds a test here, features and bug fixes alike. A bug-fix test is red without the fix, green with it. And ./test.sh must pass before shipping any build (/riff-ota, /riff-publish) — a green suite is a deploy precondition.

Everything on-device is device-only, so the suite covers the pure halves:

  • Warm mic + battery gatesWarmMicPushPolicyTests, WarmMicPushBoundaryTests, WarmMicScreenGateTests, WarmMicHeartbeatTests, WarmMicHopPolicyTests, WarmMicReviveTests, WarmMicBufferPolicyTests, WarmMicPeakHold*Tests, WarmMicForegroundGateTests, ScreenWakePolicyTests. These are the battery invariant's teeth: dedup/jitter tolerance, dark cadences, heartbeat suppression.
  • Trigger + lifecycleDictationBridgeTests (the warm/dictating reducer + the Darwin name), DictationLifecycleTests (the activity is created at arm, not on a backgrounded toggle), DictationHandoffTests (App-Group write/consume), DictationTranscriptTests, ActivityFreshnessTests, DictationActivityReconcileTests.
  • Keyboard surfaceKeyboardChromeTests, KeyboardGlassTests, KeyboardLivenessTests, ClusterAnchorTests (the palette's drag anchors, shared with the app).
  • AudioAudioLevelTests (peak/RMS summarisation feeding the meter), AudioFileWriterTests, AudioSessionRecoveryTests (restart through a recoverable route-change instead of false-failing), AudioCoexistenceTests.
  • VisualsLEDGridParityTests (the keyboard, the app and the island draw the same grid), DictationVisualsTests, RiffMarkRenderTests (the RiffMic asset resolves, ink really tints it, the grille reads as slots).

The server suite is stdlib unittest (deliberately dependency-free — no pytest): cd server && python3 -m unittest discover -s tests -t . -p 'test_*.py'.

cd ios && ./test.sh                                   # all tests, default sim
SIM='iPhone 17' ./test.sh                             # override the simulator
./test.sh -only-testing:RiffTests/DictationBridgeTests # forward extra args
./verify.sh                                           # the whole ship gate

test.sh runs xcodegen generate first (Riff.xcodeproj is gitignored) and needs no output formatter. A clean run boots a simulator; it is multi-minute.

SourceKit noise. In-editor diagnostics here lie — "No such module UIKit", "cannot find type" are false positives. A phase is green only when xcodebuild test exits 0 and prints ** TEST SUCCEEDED **.

Repo layout

riff/
├── README.md                       # this file
├── CLAUDE.md                       # the battery invariant (authoritative)
├── dictation-requirement.md        # the Wispr-Flow parity spec, from Mark's screen recording
├── install.sh                      # Mac server install + iOS bootstrap
├── verify.sh                       # ship gate: skills wiring, live server, both suites
├── Riff.icon/                      # Icon Composer app icon, BOTH apps — see Assets
├── tools/logo/                     # the mic: model, viewer, exporter, icon renderer
├── ios/
│   ├── project.yml                 # XcodeGen manifest (Riff + RiffKeyboard + RiffWidget + RiffTests)
│   ├── test.sh, Makefile           # simulator suite
│   ├── Riff.xcconfig.example       # committed; the real one is generated + gitignored
│   ├── incidents/                  # on-device warm-liveness post-mortems
│   ├── Riff/                       # the HOST app: owns the mic, the network, the secret
│   │   ├── RiffApp.swift           # @main; Notification.Name.riffToggle
│   │   ├── ContentView.swift       # the dictation screen (waveform + disc + gear)
│   │   ├── WarmMic.swift           # the warm background mic: tap → clip → transcribe → handoff
│   │   ├── DictationActivityController.swift  # ActivityKit driver
│   │   ├── RiffClient.swift        # HMAC-signed POST to /riff/transcribe-only
│   │   ├── HMACSecretStore.swift   # the user-entered secret, in the Keychain
│   │   ├── RiffIntents.swift       # Action Button App Intent
│   │   ├── ScreenWake.swift        # the screen-dark gate the battery invariant rides on
│   │   └── Dictation/AudioSessionRecovery.swift  # pure recovery policy
│   ├── RiffKeyboard/               # the appex (mark.riff.keyboard): SECRET-LESS, no network
│   │   ├── KeyboardViewController.swift   # posts the Darwin trigger; types the transcript
│   │   └── KeyboardDictationView.swift    # the keyboard's own surface
│   ├── RiffWidget/                 # Live Activity (mark.riff.widget): Dynamic Island + Lock Screen
│   ├── Shared/                     # compiled into more than one target
│   │   ├── DictationBridge.swift   # the pure toggle reducer + the Darwin names
│   │   ├── DictationHandoff.swift  # App-Group transcript handoff
│   │   ├── KeyboardDictationModel.swift  # the keyboard's display-link read side of DictationLevels; here only so RiffTests can pin it
│   │   ├── LEDHeat.swift           # the waveform ramp, shared with macOS
│   │   └── Marks.xcassets          # the RiffMic template glyph
│   └── RiffTests/                  # Swift Testing suite (simulator-only)
├── macos/                          # RiffDictate: the menu-bar daemon — see macos/README.md
│   ├── project.yml
│   ├── install-dictate.sh
│   └── RiffDictate/main.swift      # status item, ⌥Space hot-key, tap → ElevenLabs → ⌘V
├── server/
│   ├── riff_server.py              # aiohttp on :8903 — health + transcribe-only
│   ├── QUICKSTART.md
│   └── tests/                      # stdlib unittest
├── skills/                         # /riff-setup, /riff-ota, /riff-publish, /riff-update
└── LaunchAgents/
    ├── com.mark.riff-server.plist  # the iOS transcription server
    └── com.mark.riff-dictate.plist # the macOS menu-bar daemon

Dependencies

iOS

  • iOS 17+, Xcode 26 (Swift 6.3).
  • No SwiftPM dependencies. SwiftTerm and swift-nio-ssh left with the terminal.
  • Frameworks: AVFAudio / AVFoundation (mic capture + AAC encode via AVAudioFile
  • AVAudioConverter), ActivityKit (the Live Activity), AppIntents (Action Button), Accelerate (the spectrum), CryptoKit (HMAC, host only), WidgetKit.
  • Capabilities: UIBackgroundModes: [audio] on the host, App Group group.mark.riff, and Full Access on the keyboard.

Mac

  • Python 3.11+ and aiohttp — that is the whole server dependency list. aiohttp serves :8903 and makes the outbound multipart POST to ElevenLabs; there is no elevenlabs SDK.
  • ElevenLabs Scribe v2 for STT, billed to the account behind ELEVENLABS_API_KEY.
  • macOS 14+ on Apple Silicon for RiffDictate, plus xcodegen to generate either project.

Apple Developer Program

Membership active; bundle IDs mark.riff, mark.riff.keyboard, mark.riff.widget registered under the team; Background Modes: audio on the Riff target.

Secrets

Both live in ~/.env:

RIFF_SHARED_SECRET=<32-byte hex>             # phone ↔ Mac server HMAC
ELEVENLABS_API_KEY=<elevenlabs key>          # STT, both platforms
# ELEVENLABS_STT_MODEL=scribe_v2             # optional override (default scribe_v2)

The server reads them on boot. A missing RIFF_SHARED_SECRET is fatal; a missing ELEVENLABS_API_KEY leaves health up and 503s every transcribe.

RIFF_SHARED_SECRET in distributed builds is USER-ENTERED, not baked. Baking it into an .ipa would hand every tester a working credential to the server. So:

  • A local dev build (./install.sh --bootstrap-ios) bakes the real secret + host into ios/Riff.xcconfig (gitignored) for one-tap use.
  • A distributed build ships an EMPTY RIFF_SHARED_SECRET / MAC_MINI_HOST (/riff-publish and /riff-ota both refuse to publish otherwise). The user pastes the secret their own server printed into Settings → Voice server, where it lives in the Keychain (HMACSecretStore, service mark.riff.hmac).
  • Settings.sharedSecret reads the Keychain first and falls back to the baked xcconfig value, so both paths work with no code change.

The ElevenLabs key never leaves the Mac; only the Mac→ElevenLabs leg sends it, as the xi-api-key header over TLS.

Install

# Mac server: deps, ~/bin symlink, launchd job, env validation.
./install.sh

# iOS bootstrap: writes ios/Riff.xcconfig from ~/.env, then xcodegen.
./install.sh --bootstrap-ios

# Health check (HMAC-signed curl to /riff/health).
./install.sh --health

# Tear down launchd + the ~/bin symlink.
./install.sh --uninstall

# macOS menu-bar dictation (separate, no server needed).
cd macos && ./install-dictate.sh

The default install prints a status table (env keys, launchd job, server reachable, xcconfig + Xcode project present) and is idempotent. /riff-setup wraps the whole bring-up, including minting RIFF_SHARED_SECRET if it is absent (it never rotates an existing one — that would 401 every paired device).

The launchd plist lives at ~/Library/LaunchAgents/com.mark.riff-server.plist (copied, not symlinked, because launchd distrusts symlinked plists). Logs land at ~/Library/Logs/riff-server.log.

Distributing

OTA to your own phone (/riff-ota, the primary dev-deploy)

skills/riff-ota/riff-ota.sh builds an ad-hoc-signed .ipa and hosts it plus an itms-services manifest and a one-tap install page on the Tailscale Funnel (public HTTPS). You install by opening the page in Safari and tapping Install Riff — off-LAN, over cellular, no cable. /riff-update (devicectl, same LAN) is the cabled fallback.

~/.claude/skills/riff-ota/riff-ota.sh             # Debug build → publish → print the URL
~/.claude/skills/riff-ota/riff-ota.sh --release   # Release build (TestFlight-equivalent smoke)

No-real-secret requirement (load-bearing). A Funnel-hosted .ipa is publicly downloadable by anyone with the URL, so the OTA build passes RIFF_SHARED_SECRET="" and MAC_MINI_HOST="" as build settings on archive, leaving ios/Riff.xcconfig byte-for-byte untouched (checksummed before and after) so the cabled flow keeps its baked config. It then unzips the exported .ipa and verifies the embedded Info.plist carries no secret and no host before publishing. Enter the secret once in Settings; the Keychain value survives install-over.

The export uses ios/ExportOptions-adhoc.plist with method = development, which signs with the Apple Development cert and the profile embedding the registered device UDID (release-testing/ad-hoc need a Distribution cert this keychain lacks, and minting one over SSH hits "No Accounts").

TestFlight (/riff-publish)

skills/riff-publish/riff-publish.sh runs: guardrail → bump CFBundleVersion (host + keyboard + widget in lockstep) → xcodegen → archive (Release) → export via ios/ExportOptions.plist (method = app-store-connect) → upload via xcrun altool (App Store Connect API key from ~/.env: ASC_KEY_ID + ASC_ISSUER_ID; the .p8 at ~/.appstoreconnect/private_keys/AuthKey_<ASC_KEY_ID>.p8). --no-upload produces the .ipa only. Publishing burns a build number and may trigger Beta App Review, so it is invoked by hand, never on every commit.

GUARDRAIL: /riff-publish REFUSES to publish if ios/Riff.xcconfig bakes a real RIFF_SHARED_SECRET or a real MAC_MINI_HOST. To produce a clean build: printf 'RIFF_SHARED_SECRET =\nMAC_MINI_HOST =\n' > ios/Riff.xcconfig && (cd ios && xcodegen generate).

Notes: CFBundleVersion must be unique and monotonic per upload. The privacy questionnaire answer is "no data collected by us" (audio goes to the user's own server and their own ElevenLabs account). TestFlight builds expire 90 days after upload, so re-run /riff-publish before the beta goes dark. NSAllowsArbitraryLoads is needed for the plaintext-over-tailnet transport; TestFlight generally accepts it.

Gotchas. Open the install page in Safari (itms-services:// links are intercepted by SpringBoard). A build signed with an expired development profile won't launch — re-running /riff-ota re-signs, so routine use self-heals. iCloud Private Relay can break Funnel pages (TLS error on both cellular and Wi-Fi); toggle it off for the install.

Constraints / gotchas

  • A keyboard extension cannot hold a microphone. The whole two-process architecture exists because of this. See the build-278 probe above.
  • ActivityKit refuses to start an activity from the background, which is why the Live Activity is created at arm (foreground) and only updated by the keyboard toggle.
  • No transcription ceiling. Recording is unbounded; the cap is the server's MAX_AUDIO_BODY = 25 MB, roughly 30 minutes of AAC.
  • Wispr Flow conflict. Wispr Flow holds the system audio session for system-wide dictation. Riff opens its own .playAndRecord session and expects Wispr Flow to yield it; verify on first install.
  • Tailnet resolution after a cold launch occasionally times out while routes warm up; RiffClient retries once on connection-never-established errors.
  • macOS TCC ties permission grants to the code signature, so ad-hoc-signed reinstalls silently lose the mic and Accessibility grants (the toggles still read "on" while the paste stops working). See macos/README.md for the self-signed cert that fixes it.
  • The Action Button is a launch trigger, not a held-down switch. iOS does not surface press/release to apps, so hold-to-talk is unavailable on iOS. Apple's PushToTalk framework doesn't change that (it is gated on VOIP use cases and would not deliver press/release either).

Risks

risk mitigation
Scribe model id (scribe_v2) drift Module constant ELEVENLABS_STT_MODEL (one-line change), overridable via ~/.env; fallback id scribe_v1.
Upload retry double-billing RiffClient retries only on connection-never-established errors (cannotConnectToHost/notConnectedToInternet), never on timedOut, so a slow Scribe call is never re-run and re-billed.
Audio leaves the device (privacy) Deliberate, accepted trade for accuracy. The audio→Mac leg is tailnet-only (no Funnel); only Mac→ElevenLabs hits the public internet, over TLS.
The warm mic drains the battery overnight The screen-dark gates in CLAUDE.md, pinned by the WarmMic* suites. The real gate is an overnight on-device check: battery settings must not show Riff with hours of background activity.
A distributed .ipa leaking the HMAC secret /riff-ota and /riff-publish both build secret-less and verify the exported .ipa before publishing; the keyboard appex is grepped for crypto and transcribe symbols.
Tailscale on the phone disconnects (reset, OS update) The upload fails with the normal offline error and the transcript is lost; re-enable Tailscale and re-dictate. No fallback path by design.
Wispr Flow doesn't yield the audio session Disable it temporarily; the app surfaces an audio-session-unavailable error if the engine fails to start.