Skip to content

Inbox mobile: the iPhone app, pair, list, thread, picks, Send (M1) - #1778

Merged
backnotprop merged 8 commits into
mainfrom
feat/mobile-m1-app
Oct 8, 2026
Merged

backnotprop merged 8 commits into
mainfrom
feat/mobile-m1-app

Conversation

@backnotprop

@backnotprop backnotprop commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

The story

This adds the Plannotator iPhone app at apps/ios: a SwiftUI app that pairs with the Inbox on a computer, shows the list as inset sections, opens a thread with native question cards, saves a pick with one tap and sends the reply. It talks only to the device door in adr/implementation/inbox-mobile.md, through a small local Swift package, PlannotatorKit, with no third-party code. It is proved in action: an XCUITest flow drives the app in a simulator against a compiled Inbox while real agents write through plannotator inbox mcp, and one paths-gated macOS job runs that flow in CI.

Proved against main at 7bdb99532 (P1 c7bb5050d, its follow-up 1779 2f57787f1, P2 1780 d7b3d064b, S1 1777 d252b3a19, then 1783, all merged): the pairing routes, the device door and the tailnet listener. The final run used a binary compiled from this branch rebased on that commit, so its server code is main's. Earlier runs used PR 1776's head f56368bc4 from a detached worktree, with the same result.

Review fixes (REVIEW-PLANNOTATOR-MOBILE-M1-2026-10-08, four blockers, one push)

  1. Send keeps words still in a field. The cards' Other, note and answer fields now live in the session (SourceSession.drafts), not in view state. Send commits them as picks first, waits for the pick chain, then sends. A field also saves when it loses focus, as before. Proved: a note typed and still focused, then Send. The agent's wait_for_reply body has both the pick and the note (frames note-before-send, note-after-send).

  2. Dynamic Type at the accessibility sizes. At those sizes:

    • The list row stacks: project, then count and time, then the marks, then the subject on up to four lines.
    • The card's header and footer stack.
    • The Recommended pill scales rather than forcing width.
    • The sender row moves the time under the name.
    • The agent mark stops growing at 32 pt.

    Frames: ax-list, ax-thread, ax-thread-card, at accessibility-extra-extra-extra-large, light and dark.

  3. No false "Not sent". When a Send gets no answer, the phone reads the thread again:

    • If the person's reply to that message is there, the Send counts as delivered and the line stays clear.
    • If the thread can be read and the reply is not there, the line says "Not sent… Tap Send to try again".
    • If the thread cannot be read, it says "Your computer can't be reached, so it is not certain this was sent. Tap Send again; it is sent once either way." (InboxError.sendUnconfirmed). The key is kept, so a retry is applied once.

    Proved through a proxy that lets the Inbox apply the Send and then kills the connection. The card shows "Sent", no problem line appears, the agent gets the answer, and the thread holds exactly one reply from the person (frame send-dropped-after-apply).

  4. The thread's unreachable and gone states.

    • A thread this phone never read, while the computer is out of reach: "Can't reach " with "This thread has not been read on this phone yet. Check that the Inbox is running on your computer and that Tailscale is on." and Try Again. These words are not in the record; they are the plainest sentences.
    • A thread deleted on the computer: "This thread was deleted", "It was deleted in the Inbox on .", and "Back to Inbox". No controls are left.
    • The Inbox writes no event for a deletion, so the phone learns of it at its next read of that thread: a pull, a pick, reopening it, or the app returning to the front. An event on deletion is a server item for a follow-up, not M1.

    Proved: the proxy down, a never-opened thread shows Can't reach, the proxy back up, Try Again loads it (frame thread-unreachable). A thread deleted through the window route while open, then a pull, shows the gone state (frame thread-deleted).

The proof runner now puts a loopback proxy in front of the door. The test can switch it to down or drop-reply. It also gained /person-replies, /delete-on-computer and /text-size. Also fixed while there: the Send problem line's VoiceOver label is now its words, not the symbol's name. The review's other items are follow-ups. The flow now takes 206 s locally. The branch is rebased on main d7b3d064b (P2), and the proof binary was compiled there. P2 changes nothing the app calls: digits still go to the tailnet or loopback, and the app does not try the LAN address yet.

plannotator-ops's blocker: pairing links (one push)

  • Content never reaches the URL handler. Agent markdown keeps only https and mailto links tappable (inlineMarkdown in Support.swift). A plannotator:// link, or a link in any other scheme, is drawn as plain text.
  • An outside link asks first. A pair link opened from outside the app (onOpenURL) shows a confirmation with Cancel and Pair (its wording is in the next section). The address is the one that will be contacted: the link's tailnet address, the only path this build tries. Nothing is redeemed until Pair. The in-app scanner is unchanged.
  • Proof steps. An agent's message carries a live pairing link as a markdown link. It reads as plain text, a tap opens nothing, and the computer still lists one device (frame link-in-message). simctl openurl with a live offer's link shows the confirmation (frame pair-link-confirm). The device count stays 1 through Cancel, and Pair makes it 2.
  • The + fix. The confirmation showed Michael's+MacBook+Pro+(2), because a link written by URLSearchParams uses + for spaces. PairLink now reads + as a space; a literal plus arrives as %2B. There is a package test for it.
  • CI. ios.yml gains a concurrency group, so a new push cancels the stale run.
  • Base. Rebased on main d252b3a19 (S1), and the proof binary was compiled there. The flow now takes 242 s locally.

plannotator-ops's delta review: the address the person reads

  • What an address may be. InboxAddress now refuses user info, a path and a query. A link with tailnet=macbook-pro.tail0000.ts.net:8443@evil.example would have shown the first host and dialled evil.example; it now parses to no address.
  • One value, shown and dialled (second delta). InboxAddress reads the raw percentEncodedHost. It accepts only a plain host name (letters, digits, dots and hyphens; IPv4 literals included) or a bracketed IPv6 literal, and refuses % and @ (%2540 is refused). It keeps the host and port as fields, builds baseURL from them with URLComponents, and writes hostPort from the same fields. No string is parsed twice and nothing is force-unwrapped. [fd7a::1]:8443 is accepted and dialled with its brackets. It is stored on disk as the host:port string, read back through the same checks. There are package tests for each case.
  • The confirmation. The title is "Pair with this computer?". The message's first line is the dialled host:port on its own, then Named “<link's name>”, then the warning. A name that reads like an address cannot push the real host out of view (frame pair-link-confirm). A link whose address carries user info gets "no address your phone can reach" and no Pair button (frame pair-link-userinfo).
  • Known computers. A scanned link's computer joins the "On your tailnet" list only after it pairs.
  • Tests. One package test refuses user-info addresses. The proof flow opens a user-info link (no Pair button) and an impostor-named link (the message leads with the real host).
  • The earlier CI red. At 7f714b704 and 4dad7c9a6 the CI simulator left the app right after the pairing cover opened. The pairing cover no longer touches VisionKit on the simulator (08711d570). The proof now copies any crash report into the CI files.
  • Base. Rebased on main 7bdb99532, and the proof binary was compiled there. The flow now takes 253 s locally.

What changed

  • apps/ios/project.yml: the project for XcodeGen. Plannotator.xcodeproj is generated from it and committed, so a fresh clone builds with xcodebuild alone. The bundle id is ai.plannotator.app, the target is iOS 26 and iPhone only. There is an app target, a UI test target and a scheme that runs both the package tests and the UI tests.
  • apps/ios/PlannotatorKit/ (local Swift package):
    • InboxClient.swift: the door's calls (pair, health, threads, a thread, seen, picks, reply, resolve, delete, the decision switch, revoke) and the errors in the Inbox's own words. InboxAddress uses plain HTTP for a loopback host and HTTPS for anything else, which is the tailnet publication.
    • EventStream.swift: GET events?cursor= with its SSE framing.
    • Models.swift: the wire shapes from packages/core/inbox-types.ts.
    • PairLink.swift: the plannotator://pair link (section 1).
    • Keychain.swift: the token and the pairing secret, kept on this device only (AfterFirstUnlockThisDeviceOnly).
    • MessageBlocks.swift: the markdown splitter, cut at the question blocks.
    • Tests for the splitter, the link and the SSE framing (Swift Testing).
  • apps/ios/Plannotator/:
    • App/AppModel.swift: the paired sources, pairing, and Remove this source.
    • App/SourceSession.swift:
      • The list, which draws from a cache first and then refreshes.
      • The event stream while the app is in front, caught up from the list's cursor when the app returns.
      • The "N new" hold while the list is scrolled.
      • Picks, sent one after another per thread, drawn before the Inbox answers.
      • Send, with one idempotency key per target message, kept in UserDefaults across retries and restarts until a definite answer.
      • Resolve and Delete.
    • Inbox/: 1.1, 2.1B, 2.2, 2.3 and 2.4.
    • Thread/: 3.1 to 3.5. This covers the question card, the reply bar and composer, the native markdown blocks, and the row for blocks that are not drawn natively.
    • Pairing/: 1.2 and 1.3.
    • Settings/: 9.1 and 9.2 (the tailnet row only).
    • Colors.xcassets: generated. Assets.xcassets: the app icon, the Tater frame and the agent marks.
  • apps/ios/scripts/gen-colors.ts: writes Colors.xcassets from packages/ui/themes/plannotator.css (oklch to sRGB, light and dark). --check fails on drift.
  • apps/ios/scripts/proof.ts: the proof runner (see Proofs).
  • apps/ios/PlannotatorUITests/ProofTests.swift: the flow.
  • apps/ios/README.md.
  • .github/workflows/ios.yml: one job (shape below).
  • bunfig.toml: pathIgnorePatterns gains apps/ios/**.
  • AGENTS.md: Project Structure has an apps/ios entry, and the Inbox section's "On a phone" line (as P1 left it) now lists the app as built, with its proof.

Record sections this makes real

Renders 1.1, 1.2, 1.3, 2.1B, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3A, 3.4A, 3.5 and 9.1, plus 9.2 with the tailnet row. Side-by-side sheets (record light, built light, record dark, built dark) for each one are in research/operational/mobile-proofs/M1-2026-10-08/compare/. The raw screenshots are in built/. The recording of pair, pick and send is M1-pair-pick-send.mp4. Contract sections 1 and 2 are covered: the link, the redemption, the door routes the app calls, the error codes, and section 6's keys for Send.

Proofs

  • XCUITest against the compiled Inbox: bun apps/ios/scripts/proof.ts --binary <plannotator compiled at c7bb5050d>. The runner starts inbox --background under a temp data dir with PLANNOTATOR_BROWSER=none, connects six agents through scripts/inbox-sim.ts, creates a fresh simulator (iPhone 17, iOS 26.5) and runs xcodebuild test. The test calls a loopback control server for the things only the computer can do. Each step below is asserted:

    1. 1.1 draws.
    2. Pairing by a typed address. A wrong code answers "4 tries left". The right code pairs.
    3. The list draws (2.1B).
    4. With the list scrolled, two agents write. The pill reads "2 new", and a tap shows the new threads (2.3).
    5. A swipe shows Delete and Resolve, and Resolve works (2.4).
    6. In the thread, a pick and a note on question 1 and Other on question 2 make the bar read "2 picks". The composer opens with the picks written in, and the test adds words and sends.
    7. The agent's wait_for_reply returns a body that contains the pick, the note, the Other text and the words.
    8. The cards read "Sent".
    9. Resolve moves the thread to Quiet. Delete from the thread's menu removes it.
    10. 9.1 and 9.2 draw.
    11. The computer removes the phone (window route 7.37). The phone's next call answers 401 device_revoked, and the app draws "This phone was removed" with Pair Again.
    12. Pairing again goes through the computer the phone already knows (1.3's tailnet list).
    13. Remove this source revokes on the computer: GET /api/inbox/devices comes back empty.
    14. 1.1 draws again.

    Result: ** TEST SUCCEEDED **, 135 s for the flow, plus 8 package tests.

  • Package tests (critical pure logic only):

    • Questions split in body order, and a :::question inside a fence stays code.
    • Tables, diagrams, math and HTML become rows.
    • List markers and depth are kept.
    • The contract's link parses, an unknown v asks for an update, and loopback uses plain HTTP.
    • An SSE record is delivered at its blank line.
  • Colours: bun apps/ios/scripts/gen-colors.ts --check.

  • Still needs the iPhone 15 (no Apple ID in Xcode yet): camera pairing with VisionKit's scanner (the simulator has no camera and draws "The camera isn't available"), the torch, the feel of haptics, and a phone reaching the Inbox over a real tailnet (P1 files the tailnet run).

CI job shape (.github/workflows/ios.yml)

  • Where it runs: one job, ios-app, on macos-latest (the macOS image release.yml's test matrix uses). There is no matrix and one simulator destination (iPhone 17 on the newest iOS runtime). Timeout is 45 minutes.
  • When it runs: on PRs to main and pushes to main that touch any of these paths: apps/ios/**, packages/server/inbox*, packages/shared/inbox/**, packages/core/inbox*, packages/ui/themes/plannotator.css, apps/hook/server/index.ts, apps/hook/server/inbox*, scripts/inbox-sim.ts, and the workflow file itself.
  • Steps, in order:
    1. checkout, setup-bun 1.3.14, bun install --frozen-lockfile.
    2. Select the newest /Applications/Xcode_26*.app. The job fails plainly if the image has none.
    3. The colour drift check.
    4. Build the apps and compile the binary with release.yml's flags.
    5. Cache ~/Library/Caches/org.swift.swiftpm and ~/ios-derived (DerivedData). The key is a hash of the Swift sources, the pbxproj and Package.swift.
    6. bun apps/ios/scripts/proof.ts --binary .local/plannotator --device "iPhone 17" --derived ~/ios-derived --shots .local/proof/ios.
    7. Upload .local/proof/ios/ as the ios-proof artifact.
  • What it never touches: signing, provisioning or TestFlight material.
  • Owner item: the macOS minutes, which plannotator-ops puts to the owner.
  • Latest green run: run 37742598232 on head 2f0e2a56e, 21 minutes. One run before it failed at its first tap because the shared runner took over a minute to start the app; the test's waits are now 30 s. First green run: 37733503673, 17.6 minutes with a cold cache. The first run failed because the agent's wait_for_reply hit the MCP SDK's 60 s client timeout on the slower runner; the proof now calls again, as an agent would (second commit).

Gates run

  • bun run typecheck passed.
  • bun run check:release-version passed (v0.28.8). apps/ios keeps its own version line (MARKETING_VERSION 1.0, build 1, in project.yml) and stays outside the release-bumped files.
  • bun test scripts under a temp data dir: 211 pass, 0 fail.
  • gitleaks git . --log-opts=origin/main..HEAD: no leaks.
  • Semgrep was not available locally. The two new TypeScript files use only argv spawn and spawnSync and no exec, and no request data reaches a filesystem or network sink.
  • xcodebuild build has no warnings from our code.
  • The full XCUITest proof above.

What it leaves out

  • Attachments, the "3 annotations" chip, the attachment tiles and the surface (M2).
  • Decisions: the tab, the decision card sheet, and New message with its button (M3).
  • Guides (M4), notifications and the 9.1 notification switches (M5), and the relay and 9.2's Wi-Fi and relay rows (P2 and M6).
  • Workspaces: the 1.1 row and 1.4 (M7).
  • The iPad layout.
  • Bonjour and the LAN address (P2). A scanned link's lan address is not tried yet; only tailnet is.

Deviations from the record or the contract, with the reason

  1. Two tabs, not three. The Decisions tab would be empty until M3. An empty tab is a rough edge, so it comes with M3.

  2. 1.1 has no Workspaces row. Its subtitle reads "Connect the Inbox on your computer." A row that does nothing is worse than none. The row and the drawn words come back with M7.

  3. The source switcher is the system's. It is toolbarTitleMenu with navigationSubtitle. iOS 26 draws its chevron on the inline title (2.3) but not beside the large title, and the subtitle uses the system's style without the laptop glyph. This is native behaviour; a hand-built title would lose the system's collapse on scroll. With one computer the menu holds that computer and "Add a source"; with several, each one.

  4. 1.3 lists only "On your tailnet". It shows the computers this phone knows plus a field for an address. "On this Wi-Fi" needs Bonjour and the LAN listener (P2). The footer is the record's words.

  5. The decision switch saves at once. It saves recording through 7.21 and opens no card. The card sheet (5.1) is M3. Until then the drafted words apply, which is the Inbox's default.

  6. 2.4's swipe buttons are iOS 26's own round buttons, not the record's flat panels. A full swipe resolves, as drawn.

  7. The question card's small switch is the system switch drawn at 78% to match the record's small switch (tgl.sm), so the row's words fit on one line.

  8. Colours come from plannotator.css tokens as they are. The record's sheet used a few hand-tuned phone values (for example --fg2 at oklch 0.47, where the theme's muted-foreground is 0.40), and the brief asks for generation from the theme. Three colours are not one token:

    • The question card is the mix of card and background.
    • "Screen" is card in light and background in dark, as the record's white screens and dark ground are.
    • "Raised" is background in light and card in dark.

    Those choices are named in gen-colors.ts.

  9. The record does not draw these states; this PR builds them:

    • Loading: a spinner until the first list arrives.
    • An empty list: "Nothing waiting".
    • Unreachable with no cache: "Can't reach …" with Try Again. With a cache, the subtitle reads "· Not reachable".
    • Removed on the computer: "This phone was removed" with Pair Again.
    • A wrong code: the boxes shake and the message gives the tries left.
    • A failed Send: one line above the bar, and Send retries with the same key.
    • A resolved thread: a "Resolved · Reopen" bar.
    • The delivery line under the person's reply, in the desktop's words.
    • The row for a table, diagram, math or raw HTML block: "Read it in the Inbox on your computer." It becomes the surface once S1 lands.
    • "Inbox · on this Mac" and "Connected on this Mac" for a loopback address, which only the simulator uses.
  10. The composer's draft follows the desktop ("Prompt: answer.") but drops the extra period when an answer already ends with one.

  11. The app icon is a placeholder. It is the 256 px Plannotator mascot (apps/vscode-extension/images/icon.png) scaled to 1024. A 1024 master is an owner item. The 1.1 Tater is the first frame of the original 5376x256 sheet, taken from Workspaces' history (853d293af^), mirrored, at 3x, because the 77x88 frame in the repo is too small for 84 pt.

  12. The screenshots show this Mac's real name, "Michael's MacBook Pro (2)". P1 reads scutil --get ComputerName.

There are no contract changes. The pick body sends "answer": null explicitly to clear a pick.

Limits

  • The app adds none.
  • It shows the two limits P1 enforces (an offer lives 10 minutes; 5 wrong codes close it) only as the Inbox's words.
  • The event stream reconnects 2 s after a drop. That is a retry pause, not a cap.

Dependencies

No third-party Swift or npm dependencies. XcodeGen (brew install xcodegen) is needed only to regenerate the project after editing project.yml.

Not done, named

  • Dynamic Type at accessibility sizes is not captured in a screenshot. Text uses text styles and @ScaledMetric.
  • CI does not check that Plannotator.xcodeproj matches project.yml; that would install XcodeGen on every run.
  • 2.2's pull-to-refresh spinner is not captured; it is the system control.

apps/ios: a SwiftUI app (ai.plannotator.app, iOS 26) with a local Swift
package, PlannotatorKit, that talks to the Inbox through the device door
(adr/implementation/inbox-mobile.md). Pairing by typed address and six
digits or the QR link, the list as inset sections with the N new pill,
threads with native question cards, one-tap picks, the docked reply bar,
Send with a kept idempotency key, Resolve, Delete, Settings and the
tailnet row. Colours generated from plannotator.css with a drift check.
The XCUITest proof runs against a compiled Inbox through
apps/ios/scripts/proof.ts, in one paths-gated macOS job (ios.yml).
Send saves what the cards' fields still hold (the drafts live in the
session and Send commits them first). The list and the thread wrap at the
accessibility text sizes. A Send whose answer is lost re-reads the thread:
applied is shown as sent, an unknown outcome says so, and a retry keeps the
key. The thread screen draws Can't reach (with Try Again) and This thread
was deleted. The proof adds a step for each.
Agent markdown keeps only https and mailto links tappable, so a
plannotator:// link in a message is plain text and never reaches the
app's URL handler. A pair link opened from outside the app asks
"Pair with <name> at <address>?" (the address it will contact) before it
redeems; the in-app scanner is unchanged. A + in the link's query reads
as a space. ios.yml cancels a stale run on a new push.
InboxAddress refuses user info (a@b would show a and dial b), a path, a
query; hostPort is rebuilt from the parsed host and port. The outside-link
confirmation reads "Pair with this computer?" with the dialled address on
its own first line and the link's name after it, marked as a name. A
scanned link's computer joins the known list only after it pairs.
InboxAddress reads percentEncodedHost and accepts only a plain host name,
an IPv4 literal or a bracketed IPv6 literal (no %, no @). It keeps the
host and port as fields, builds baseURL from them with URLComponents and
writes hostPort from the same fields, so the shown and the dialled
address are one value; no string is parsed a second time and nothing is
force-unwrapped. Tests: %2540 refused, [fd7a::1]:8443 dialled with its
brackets.
@backnotprop
backnotprop merged commit 5da2ffc into main Oct 8, 2026
29 checks passed
backnotprop added a commit that referenced this pull request Oct 8, 2026
# Conflicts:
#	AGENTS.md
#	bunfig.toml
@backnotprop
backnotprop deleted the feat/mobile-m1-app branch October 8, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant