Skip to content

Inbox mobile: the surface mode (S1) - #1777

Merged
backnotprop merged 7 commits into
mainfrom
feat/mobile-s1-surface
Oct 8, 2026
Merged

backnotprop merged 7 commits into
mainfrom
feat/mobile-s1-surface

Conversation

@backnotprop

@backnotprop backnotprop commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

The story

A phone's native shell needs Plannotator's own viewers without a website in a frame. This PR builds apps/hook/dist/surface.html, one self-contained file that mounts only the attachment viewers and the guide, with no network, and the versioned bridge the iPhone app talks to it over. The composer stays native: the viewers hand their drafts to the shell through new additive ui seams, and the shell answers with the saved annotation.

What changed

  • The bridge packages/core/inbox-surface-bridge.ts (re-exported by @plannotator/core/inbox-types, so core's export map is unchanged): every message of contract section 5 both ways, INBOX_SURFACE_BRIDGE_VERSION = 1, and readSurfaceShellMessage (envelope, version, what the surface needs to draw). The surface's end: packages/inbox/surface/bridge.ts (window.plannotatorSurface.receive in, window.webkit.messageHandlers.plannotatorSurface.postMessage out).
  • The surface packages/inbox/surface/: SurfaceApp.tsx (theme and Dynamic Type from the shell, settings in memory, diffs unified and wrapped), SurfaceDocument.tsx (markdown and text through Viewer with AnnotationToolstrip; HTML through HtmlViewer with pins; diagrams through DiagramViewer, lazy in SurfaceDiagram.tsx), SurfaceGuide.tsx with the reader packages/inbox/guide/SurfaceGuideReader.tsx behind #surface-guide-reader (lazy): the sections with ticks and Continue (6.1), then one GuideSectionCard per screen with Previous, Reviewed and Next (6.2). surface.css (sf- classes, the system face on Plannotator's tokens).
  • The build apps/inbox: surface.html (CSP default-src 'none', connect-src 'none'; an HTML page's folder from plannotator-asset:), surface.tsx, surface.css, vite.surface.config.ts, script build:surface; build:hook copies it to apps/hook/dist/surface.html beside inbox.html. tsc --noEmit -p apps/inbox/tsconfig.json and -p packages/inbox/tsconfig.json now cover the new files.
  • ui seams (additive, opt-in, Plannotator's render unchanged; packages/ui/HANDOFF.md "The composer handed to the host ... (unreleased)"):
    • HostDraft (types) and onHostDraft on Viewer / useAnnotationHighlighter, HtmlViewer / useHtmlAnnotation, and DiagramViewer (DiagramHostDraft): the draft goes to the host, no toolbar or composer is drawn.
    • ViewerHandle.stepPin? (HtmlViewer) and the bridge-script message plannotator-bridge-step-pin: Parent and Child for a pinned draft.
    • AnnotationToolstrip hideRedline (4.1 draws Select, Pinpoint, Markup, Comment).
    • A fix: HtmlViewer and useHtmlAnnotation listen for frame messages from a layout effect (a lost ready, blocker 3 below).
    • Annotate or Interact from outside needed no seam: HtmlViewer's annotateModeActive already is one.
  • Contract adr/implementation/inbox-mobile.md section 5, edited first: open_section (shell to surface) and section (surface to shell) added; link comes only from the surface's own markdown, HTML-page links being a navigation the shell decides (review blocker 1); plus what a pin's and a draft's label is, when a null selection is sent, and the surface's CSP. Reason: 6.1 and 6.2 put the guide's title ("02 of 04") and its close or back button in the shell's bar but Continue, Previous, Reviewed and Next in the document, so each side has to tell the other where the guide is.
  • Proof and CI: tests/e2e/surface.spec.ts (WebKit), added to tests/e2e/playwright.config.ts; .github/workflows/inbox-e2e.yml installs and caches WebKit beside Chromium (plannotator-ops: one more browser in the existing job, no new job). tests/entry-assets.test.ts: the surface's CSP pinned, its reader and diagram engine held lazy, its bundle in the no-WASM list.
  • AGENTS.md: Project Structure lines and "The surface" under Plannotator Inbox.

Record sections it makes real

The surface's half of 3.6 and 4.1 to 4.4 (the document and its marks: selection, pinpoint, HTML pins with Annotate or Interact and Parent or Child, a diagram node), and 6.1 and 6.2 (sections, ticks, Continue, one section with Previous, Reviewed, Next); light and dark; Dynamic Type. The bars, sheets, the changed line and the annotations sheet are the shell's (M2, M4).

Proofs

bun run test:e2e:inbox (or bunx playwright test -c tests/e2e/playwright.config.ts surface.spec.ts) after bun run build:review && bun run build:hook and the compiled binary at .local/plannotator. Playwright 1.63.0 WebKit loads the built surface.html from a file URL as the main frame; a compiled plannotator inbox --background under a temp data dir; Claude Code, Codex, OpenCode and Pi write a markdown plan, the "Low Tide" ticket page, a Mermaid flow and a guided review through plannotator inbox mcp (scripts/inbox-sim.ts). The spec plays the shell: it reads every byte through the window routes (the device door is P1's), rewrites the page's base onto plannotator-asset://inbox/..., saves drafts and ticks through the window's routes and answers commit_annotation. window.webkit.messageHandlers.plannotatorSurface exists in every frame, as WKWebView injects it, and records which frame posted.

  1. ready with bridge: 1; a v: 2 message answers error bridge_version.
  2. A fetch from the surface fails under connect-src 'none'.
  3. Markdown: a drag selection sends selection with its quote and an empty-text draft; a press elsewhere sends the null selection and the pending mark goes; saved and committed, the comment mark is drawn; a tap on it sends annotation; a Pinpoint tap sends draft (kind block); remove_annotation cancels it; a link goes to the shell as link and the page does not navigate.
  4. HTML: a tap pins "Get tickets" (pin with label and selector); step_pin parent twice and child once lands back on the button; saved, its marker is drawn in the page; Interact lets the page's own click run and sends no pin; Annotate pins again.
  5. The agent's frame: parent.plannotatorSurface.receive and top.webkit throw SecurityError; its own frame's handler post arrives marked as a child frame (the shell drops it); a bridge-shaped postMessage to the surface relays nothing; the viewer's own plannotator-bridge-link-click forged at load (an https: and a mailto: href) sends no link, and neither does a real tap on the page's own link.
  6. Diagram: a tap on "Pick a host" sends draft { kind: "node", label: "Pick a host (node E)" }; saved, its badge shows.
  7. Guide: the sections with the snapshot's tick, Continue with 02, a tick toggled (reviewed), Continue (section 1 of 2), Reviewed on the section page, open_section null back to the list; the message's guide_reviewed in the store reads [true, true].
  8. Theme and text size: light and dark from set_appearance; at text_scale 1.6 the same paragraph reflows to more than 1.5 times its height, and every toolstrip button is still what a tap at its centre hits.

PNGs light and dark, 393 by 852 at 3x: /Users/ramos/workspaces/research/operational/mobile-proofs/S1-2026-10-08/ (with index.html), and the CI artifact inbox-window-proof.

Gates run

  • bun run typecheck (with the new files under packages/inbox and apps/inbox; tsconfig.guide.json reports nothing in packages/inbox).
  • bun test packages apps/hook tests scripts under a temp data dir: 5821 pass, 0 fail. The one failure of a first run was the fresh worktree before apps/pi-extension/vendor.sh had written apps/pi-extension/generated/ (the reviewer reproduced it), not this PR.
  • DOM_TESTS=1 bun test --isolate over the 217 files of test.yml's DOM step: 1550 pass.
  • tests/entry-assets.test.ts, tests/favicon-surfaces.test.ts: pass.
  • bun run --cwd packages/ui smoke:package: pass. bun run --cwd apps/guides-show build:viewer && check:manifest: in sync (no class added to the guide chain).
  • bun run check:release-version: consistent.
  • bun run test:e2e:inbox: 65 passed, 1 skipped (P1's phone proofs, the window's Chromium proofs and the 8 surface proofs in WebKit).
  • Gitleaks over merge-base..HEAD: no leaks.

Review blockers, fixed (review: research/reviews/REVIEW-PLANNOTATOR-MOBILE-S1-2026-10-08.md)

  1. An agent's page could forge plannotator-bridge-link-click and reach the shell's link with no tap. A first fix gated on navigator.userActivation and failed CI at a01676c67: the forged post fired about 5 s after the last driven tap and rode the still-active flag (a weak gate, not a flaky test). A tap inside the agent's frame never reaches the main frame (proved in WebKit), so the surface cannot tell a real tap from a forged one. Now the surface never forwards a link from an agent's HTML page (no onOpenLink); the shell decides that navigation in M2 (WKNavigationDelegate, .linkActivated, SFSafariViewController), written into contract section 5. Markdown links keep link. Proof 5 asserts at once that neither the forgery nor a real tap in the page sends a link; proof 3 keeps the markdown link.

  2. The document drew over the toolstrip (Viewer's layer is z-50). .sf-toolstrip, .sf-gfoot and .sf-gnav take z-index: 100; proof 8 asserts the buttons are the hit target.

  3. Found by CI after the link fix (not a review item): the HTML proof failed on the Linux WebKit runner because the frame's ready reached the main window before HtmlViewer listened (the failure report showed the main window received plannotator-bridge-ready and the frame never got its input method or mode). A passive effect registered the listener after a fast srcdoc frame had posted. HtmlViewer and useHtmlAnnotation now register their frame-message listeners in a layout effect, in the commit that inserts the iframe; noted in HANDOFF.md. The phone's WKWebView would hit the same race. The spec waits for the frame's pin mark after each mode switch and reports what each side received if it ever does not arm.

Leaves out

The native shell, any list or thread UI, Pencil, multi-block selections (the brief). Also, each named for a later step:

  • The changed line ("Changed since ... Open the version it sent") is a bar under the shell's title in 4.1, and open_attachment carries no agent name, so it is the shell's (M2).
  • A relative link in a markdown file does nothing; only http(s) and mailto go to the shell as link. Links in an agent's HTML page go to the shell as navigations in M2 (contract section 5), which also needs a ui seam so the viewer's bridge lets the frame navigate.
  • A Pinpoint tap on a fenced code block does nothing under the host composer; text inside it is still selectable.
  • The HTML page's own folder assets need the shell's plannotator-asset: scheme handler, which a browser test cannot register; the proof page is self-contained, and M2 proves assets on the device.
  • The pin's label is Plannotator's pinpoint label ("Button"), not the record's div.btnx; the selector rides beside it if the sheet wants the tag and class.
  • Whether WKWebView keeps the native selection (and its edit menu) while Plannotator paints the pending highlight is the device spike's question.
  • The review's six items, as follow-up PRs: (1) a forged plannotator-bridge-selection can open the shell's composer with its own label and no tap (the same activation check fits); (2) open_attachment.focus is not proved and likely does not scroll on markdown (the scroll runs before the stored marks are painted), which M2's "3 annotations" sheet needs; (3) 6.2's diff box has blank space above and below the diff at phone size; (4) the agent page in surface.spec.ts fetches 127.0.0.1:1, which proves nothing about the CSP: point it at the live Inbox; (5) the SurfacePin and HostDraft.label comments still say div.btnx, the contract says "Button"; (6) inbox-e2e.yml does not list apps/hook/package.json, which now carries the surface's build step.

Deviations from the record or the contract

  • Contract section 5: open_section and section added, the reason above; edited in this PR.
  • 4.1's toolstrip: Plannotator's compact toolstrip expands the active button of each of its two groups, so "Select" and "Markup" both show their words; the record shows only "Select". Kept as Plannotator draws it.
  • 6.2's section is Plannotator's GuideSectionCard (title, position, overview, file chips, then the diffs), so the file chips and the "02 / 02" position sit inside the card as on desktop; the card's own Reviewed checkbox is hidden (showReviewed={false}) so the thumb bar's Reviewed is the only one, and a reviewed section does not collapse.
  • On the last section the Next button is absent, and on the first the Previous button; the record draws a middle section only.

Limits

None added. The bridge has no caps; the surface adds none.

Depends on

Contract on main (PR 1775). Rebased onto main after P1 (PR 1776), its follow-up (PR 1779) and P2 (PR 1780) merged; the conflicts were only in tests/e2e/playwright.config.ts and the inbox-e2e.yml header comment, both resolved to list every spec. The proof uses the window routes; the device door maps the same handlers.

@backnotprop
backnotprop force-pushed the feat/mobile-s1-surface branch from 2833a50 to a01676c Compare October 8, 2026 05:34
One self-contained file a phone's shell loads (apps/hook/dist/surface.html):
Viewer with its toolstrip, HtmlViewer with pins, DiagramViewer and the
guide, no network, and the versioned bridge of contract section 5. The
composer is the shell's, through additive ui seams (onHostDraft, stepPin,
hideRedline). Proved in WebKit against a compiled Inbox.
A frame's link reaches the shell only after a real tap
(navigator.userActivation), so an agent page cannot forge the viewer's
link-click; the toolstrip and the guide's controls sit above the document.
Both proved in the WebKit spec.
The surface never forwards a link from an agent's HTML page: the page can
forge the frame's message and its taps never reach the main frame, so no
gate in the surface can tell them apart (the userActivation gate failed
CI). Contract section 5 says the shell decides those navigations (M2);
markdown links keep the bridge link.
The first pin raced the bridge's ready on the CI runner (no pin within
15 s at 4c4bdf0); each mode switch now waits for the page body's
pinpoint mark before the tap.
A passive effect could register the message listener after a fast srcdoc
frame had already posted the bridge's ready; the lost ready left the bridge
with no input method or mode (the surface's HTML proof on the Linux WebKit
runner). HtmlViewer and useHtmlAnnotation now listen from a layout effect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant