Skip to content

ci: run the mobile e2e suite on a production build, one job per browser - #8948

Merged
waleedlatif1 merged 4 commits into
stagingfrom
ci/mobile-e2e-prod-build
Oct 12, 2026
Merged

waleedlatif1 merged 4 commits into
stagingfrom
ci/mobile-e2e-prod-build

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

e2e (mobile) is CI's critical path: p50 22 min, up to 26. About 40% of that is next dev compiling routes on first visit, which is CPU-bound and swings with runner speed (jobs run 15–25% slower on weekdays than on weekends). Chromium (~15 min) and WebKit (~8 min) also ran one after the other.

  • New build (e2e) job. Gated on full, like e2e. 16 vCPU. Uses its own Turbo cache key, so it never evicts entries from the build cache.
    • It builds with NEXT_PUBLIC_APP_URL=http://localhost:3000, the default every shipped image bakes (docker/app.Dockerfile). In that mode every route is dynamic and NEXT_PUBLIC_* values are read at startup, so one build serves any per-port URL.
    • It packages .next without cache or trace, zstd-compressed, as an artifact kept 3 days.
  • New e2e-mobile job, one per browser (chromium, webkit). Each downloads the build and serves it with next start against the realtime server, installing only its own browser.
    • Before serving, it checks the build is not hosted: it fails if prerender-manifest.json lists any /blog/* route.
    • The HTTP status log is rebuilt from the app's own per-request log lines, with LOG_LEVEL=INFO set on that process only.
  • build is unchanged. It still bakes https://www.sim.ai, which makes next build prerender every post. That's CI's only full render of all posts and landing pages.
  • Other e2e groups stay on next dev. scim can't move, because hosted mode can be forced only outside production. cli, stop-after and desktop-inbox are short and off the critical path.
  • ci gate. Now needs e2e-build and e2e-mobile, both gated on full, with new ci-gate.test.ts cases.

Expected: the mobile critical path drops from ~22–26 min to ~11–14 min (shorter on a Turbo cache hit). Cost is about +75 vCPU-min per full run on a build-cache miss, partly offset by the two shorter browser jobs.

Rollback: revert this PR. e2e (mobile) returns to the next dev matrix row.

Test plan

  • First run: record artifact upload and download times, next start ready time, and per-browser suite time; confirm the non-hosted guard passes
  • build still prerenders posts (same log as before)
  • ci-gate.test.ts / ci-changes.test.ts

…per browser

The build job now packages .next (zstd, cache excluded) and uploads it for a
day. A new e2e-mobile job, one per browser (chromium, webkit), extracts it and
serves it with `next start`, so the suite no longer waits on dev compiles or
runs both browsers back to back. The mobile group leaves the e2e matrix.

On runs that execute e2e, the build bakes NEXT_PUBLIC_APP_URL=http://localhost:3000,
which is what production images bake, so the served build is the one that ships.
Content-only pull requests keep the hosted value, under which next build
prerenders every post.

http-e2e.sh gains a `next start` mode used only by mobile: it installs only the
selected browser, drops the dev heap override, and rebuilds the HTTP status log
from RouteHandler's JSON lines at LOG_LEVEL=INFO. The ci gate treats e2e-mobile
like e2e: a skip passes only when `full` is exactly false.
…untouched

The `build` job is back to staging byte for byte: its www.sim.ai build is CI's
only render of every post, and it must stay on every run. A new `e2e-build`
job, gated on `full` with its own Turbo cache key, builds with
NEXT_PUBLIC_APP_URL=http://localhost:3000 (the Dockerfile default, so the same
public-env mode production images use; images run the standalone server) and
packages and uploads it for e2e-mobile, now kept 3 days and without
.next/trace.

e2e-mobile fails fast when the downloaded build prerendered posts, which only a
hosted build does; the URL is no marker, since next.config's redirects name
www.sim.ai in every build. The ci gate treats e2e-build like e2e.
@vercel

vercel Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 12, 2026 11:23pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High impact] The PR appears safe to merge; no actionable issues were found.

Summary

Moves the mobile suite to a shared production build, with separate Chromium and WebKit jobs.

  • Mobile browser tests run on a production build in parallel.
  • The required CI check includes the new mobile jobs.

No actionable issues were found. This review did not run the build or browser suites.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Changes[changes] --> Build[e2e-build]
  Build --> Artifact[Compressed production build]
  Artifact --> Chromium[Mobile Chromium job]
  Artifact --> WebKit[Mobile WebKit job]
  Changes --> Other[Existing checks]
  Build --> Gate[Required ci check]
  Chromium --> Gate
  WebKit --> Gate
  Other --> Gate
Loading

Reviews (1) · Last reviewed commit: "ci: build mobile's artifact in its own j..." · Reviewed by Greptile

Each document load mints one realtime token from a route capped at 30/min
per IP. On the production build the suite loads well over 30 documents a
minute from loopback, so the route's designed 429 is recorded apart from
httpErrors (only with Retry-After), and each browser must still mint at
least once.
@waleedlatif1
waleedlatif1 merged commit caa925a into staging Oct 12, 2026
48 checks passed
@waleedlatif1
waleedlatif1 deleted the ci/mobile-e2e-prod-build branch October 12, 2026 23:54

This branch was previously deployed

1 inactive deployment
Preview — 8173df90 Deployed Oct 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant