[AISAGE-781] perf: bypass the staging usage writer relay

services/usage-ingest/README.md

@@ -226,16 +226,16 @@ attributing any hosted latency effect.
 The first test sequence is diagnostics → EU → batching. Diagnostics and EU are
 already on main; staging-host middleware remains deferred. This refresh preserves
 the sole selected EU source/DLQ, its preparation and routing rollback behavior,
-both diagnostics and parallelism variables, and the relay. Keep one consumer at
-cap 200 / batch 2 / parallelism 2 (at most 400 writer calls); do not reconnect a
+both diagnostics and parallelism variables, and the writer target. Keep one
+consumer at cap 200 / batch 2 / parallelism 2 (at most 400 writer calls); do not reconnect a
 legacy consumer or split the cap. The queue-budget and relay runtime tests pin
 the actual EU queue names and policy. Verify the refreshed source and those
 checks before merge. Historical four-PR compatibility results do not validate
 this first-test source, which keeps middleware unchanged.
 
 For a full configuration rollback, restore absent parallelism (or `1`), batch 1,
 timeout 0 and cap 200 for the same single queue. Preserve diagnostics, the current
-writer, relay, retry delay/budget and per-message dispositions.
+writer route, retry delay/budget and per-message dispositions.
 
 ## Deploy
 
@@ -247,9 +247,9 @@ or `staging`.
 Normal staging CI has one queue preparation step before any Worker upload.
 It reads queue names from `wrangler.jsonc`, creates missing resources with
 explicit `jurisdiction: "eu"`, and verifies/reuses existing EU resources.
-Unknown jurisdiction or lookup/API errors stop deployment. Then the existing
-Frankfurt relay and consumer deploy in order; both consumer upload attempts disable Wrangler's
-global auto-provisioning for EU selections. The preparation step also detaches the obsolete
+Unknown jurisdiction or lookup/API errors stop deployment. The consumer then
+deploys with direct HTTP to the staging writer; both consumer upload attempts
+disable Wrangler's global auto-provisioning for EU selections. The preparation step also detaches the obsolete
 staging consumer before Wrangler attaches the selected source; it preserves both queue pairs.
 See [deployment-driven cutover and routing rollback](eu-queues.md).
 Production keeps automatic provisioning and its original names/policy.
@@ -262,7 +262,6 @@ account in `wrangler.jsonc`:
 ```bash
 # Staging: use the same preparation entrypoint as CI, then existing deploy commands.
 pnpm --filter cloudflare-usage-ingest queues:prepare
-pnpm --filter cloudflare-usage-ingest exec wrangler deploy --config wrangler.writer-relay.jsonc
 pnpm --filter cloudflare-usage-ingest exec wrangler deploy --env staging --experimental-provision=false
 
 # Production
@@ -312,15 +311,22 @@ The queue is simulated locally and delivered to the shipped `queue()` handler in
 the same Worker. Runtime tests cover HTTP publishing, receipt consumption, and opt-in processing
 in both environments.
 
-## Staging Frankfurt writer relay
+## Staging direct writer and retained Frankfurt relay
 
-The existing `staging` configuration sends queued writes through a private
-service binding's **fetch** method to `usage-writer-relay-staging`. The relay's
-fetch handler forwards to the same staging Vercel writer with a placement hint
-of `aws:eu-central-1`. Production keeps direct writes. The writer, middleware and
-database stay unchanged.
+Normal `staging` has no `USAGE_WRITER_RELAY` binding. The existing writer client
+calls `https://staging-app.kilo.ai/api/internal/usage/record` directly, preserving
+authentication, optional Vercel protection bypass, usage identity, the 160-second
+budget, response validation and per-message ACK/retry behavior. The single EU
+queue/DLQ, batch 2, parallelism 2, cap 200, retries and diagnostics stay unchanged.
+Production remains direct; the writer, middleware and database stay unchanged.
+This removes one hop for a controlled comparison; no latency improvement is
+established without hosted measurement.
 
-This is an outbound-request placement experiment. [Placement's shared limitations](https://developers.cloudflare.com/workers/configuration/placement/#review-limitations-1)
+The private `usage-writer-relay-staging` code, config, tests and deployed resource
+are retained for revert. Its fetch handler forwards to the same writer with a
+placement hint of `aws:eu-central-1`.
+
+The retained relay is an outbound-request placement experiment. [Placement's shared limitations](https://developers.cloudflare.com/workers/configuration/placement/#review-limitations-1)
 apply to fetch handlers and do not pin the queue consumer. The hint selects a
 Cloudflare data center near Frankfurt, not an AWS-hosted Worker. A US consumer
 still crosses to Europe and waits for the return; the relay adds a hop and may
@@ -340,23 +346,28 @@ responses ACK. The relay adds no retries or persistence.
 Normal staging CI uses `--env staging`. Both the single-worker and changed-worker
 paths in `deploy-workers.yml` deploy `wrangler.writer-relay.jsonc` **first**, then
 deploy the existing consumer when the selected source's staging binding requires
-the relay. Historical direct-path revisions skip it. A required relay's missing
-config or failed deployment stops the consumer deploy.
+the relay. The current direct-path configuration skips relay deployment while
+still preparing the EU queues and uploading the consumer. A required relay's
+missing config or failed deployment stops the consumer deploy.
 The relay's nonstandard config filename keeps it out of independent discovery;
 its deployment is owned by usage-ingest. Other Workers and production skip this
 relay step. There is no separate `staging-relay` environment or manual activation.
 
-Record both Worker revisions and the unchanged writer deployment SHA. Send one
-authenticated smoke event, verify its stable usage ID/ACK/persistence, then inspect
-the relay's actual placement and the Vercel ingress/function trace. Compute region
+For an authorized comparison, record the consumer revision, relay revision when
+used, and unchanged writer deployment SHA. Send one authenticated smoke event and
+verify its stable usage ID/ACK/persistence. Inspect the Vercel ingress/function
+trace and, when bound, the relay's actual placement. Compute region
 is `executionRegion`/`proxy.lambdaRegion`; `proxy.region` is the ingress hop.
 For a subsequent approved comparison, keep queue cap/workload and cold/warm
 conditions fixed and capture aligned event age, full consumer HTTP p95/p99, queue
 backlog/retries/DLQ and DB pool/lock limits. Accounting creation → persistence
 excludes return/ACK; the full consumer HTTP duration includes the return leg.
-Local runtime tests prove the service-binding path and ACK/retry behavior, not
-geographical placement or speedup.
-
-For relay-only rollback, deploy a reviewed direct-path change preserving the EU
-queue pair and single consumer. Queue routing rollback retains the deployment helper and changes only staging queue
-selection as described in the [EU setup](eu-queues.md); a full PR revert cannot detach EU automatically. The unused relay can remain private.
+Local runtime tests cover the normal direct route and the optional service-binding
+route with ACK/retry behavior; geographical placement and speedup remain unproved.
+
+To restore the relay, revert the staging bypass change and redeploy staging.
+This restores the service binding; CI deploys the retained relay before the
+consumer. Keep the EU queue pair, single consumer and all queue policy unchanged.
+Do not delete the unused private relay. Queue routing rollback is separate: retain
+the deployment helper and change only staging queue selection as described in the
+[EU setup](eu-queues.md); reverting the earlier EU change cannot detach EU automatically.

services/usage-ingest/diagnostics.md

@@ -1,11 +1,13 @@
 # Queue delivery diagnostics
 
 `USAGE_INGEST_DIAGNOSTICS=all` enables 100% diagnostic logging on the staging
-publisher/consumer and staging writer relay. No random sampling is applied: each
-valid test usage ID can join across stages. An absent or different value disables
+publisher/consumer and retained staging writer relay (when bound). No random
+sampling is applied: each valid test usage ID can join across stages. An absent or different value disables
 the extra logs and correlation header. Production omits the setting. Existing
 processing outcome logs remain enabled, now including `queue` for old/EU filters.
-Disable both staging settings after the investigation to reduce log volume.
+Normal staging calls the writer directly, so it emits no relay-forward records or
+relay correlation header. Disable diagnostic settings after the investigation to
+reduce log volume.
 
 | Event | Fields and boundary |
 |---|---|
@@ -43,8 +45,8 @@ existing response validation and overload failure still determine disposition.
 1. Confirm both deployed versions/settings and exact source/config first. Send a
    small approved staging cohort with known usage IDs and reconcile persistence
    using the existing shared writer. This PR does not authorize or perform it.
-2. Join enqueue start/acceptance, handler entry, processing and relay records by
-   usage ID; retain queue/message/attempt/batch fields. Confirm finite runtime
+2. Join enqueue start/acceptance, handler entry, processing and, when bound, relay
+   records by usage ID; retain queue/message/attempt/batch fields. Confirm finite runtime
    timestamps, actual send/batch metadata shape and unchanged ACK/duplicate/retry
    behavior. Check missing-log coverage before computing tails.
 3. Compare per-event publisher-before-send → handler-entry and accepted-send →

services/usage-ingest/eu-queues.md

@@ -4,16 +4,15 @@
 with processing parallelism2 (at most 400 concurrent writer calls).
 The preparation step explicitly supplies `jurisdiction: "eu"` for both queues.
 
-Normal staging deployment has one preparation step before the existing relay and
-consumer uploads. The [helper](scripts/provision-queues.mjs) looks up both exact
-names, verifies existing EU jurisdiction, creates only missing queues with explicit
+Normal staging deployment has one preparation step before the consumer upload.
+The [helper](scripts/provision-queues.mjs) looks up both exact names, verifies existing EU jurisdiction, creates only missing queues with explicit
 EU jurisdiction (DLQ first), and reads each creation back. Reruns reuse the pair;
 wrong/unknown jurisdiction, ambiguous/incomplete lookup or API/network errors stop
 upload. No queue is deleted or modified. Native global auto-provisioning is disabled
 for EU deployments. Historical revisions without EU config keep their existing path.
 
 Local deployment uses `pnpm --filter cloudflare-usage-ingest queues:prepare` before
-the existing relay/consumer commands in README, with an already injected
+the consumer commands in README, with an already injected
 `CLOUDFLARE_API_TOKEN`. No login or credentials file is loaded. Wrangler retains
 responsibility for consumer settings; retention remains separately managed as
 before. Production, writer/relay code, retry policy and dependencies are unchanged.

services/usage-ingest/src/consume-usage.runtime.test.ts

@@ -62,16 +62,6 @@ it.each([undefined, 'staging'])(
               USAGE_WRITER_PROTECTION_BYPASS: 'local-bypass',
             },
           },
-          ...(env === 'staging'
-            ? [
-                {
-                  configPath: new URL('../wrangler.writer-relay.jsonc', import.meta.url),
-                  vars: {
-                    USAGE_WRITER_URL: `http://127.0.0.1:${address.port}/api/internal/usage/record`,
-                  },
-                },
-              ]
-            : []),
         ],
       });
       harness = server;

services/usage-ingest/src/index.runtime.test.ts

@@ -34,17 +34,6 @@ it.each([undefined, 'staging'])(
           ...(env === 'staging' ? { vars: { USAGE_INGEST_MODE: 'receipt' } } : {}),
           secrets: { USAGE_INGEST_PUBLISH_SECRET: 'synthetic-local-secret' },
         },
-        ...(env === 'staging'
-          ? [
-              {
-                configPath: new URL('../wrangler.writer-relay.jsonc', import.meta.url),
-                vars: {
-                  USAGE_WRITER_URL:
-                    'https://must-not-call-writer.invalid/api/internal/usage/record',
-                },
-              },
-            ]
-          : []),
       ],
     });
     try {

services/usage-ingest/src/publish-usage.ts

@@ -12,6 +12,8 @@ export type Env = Omit<
   | 'USAGE_INGEST_PARALLELISM'
 > & {
   USAGE_INGEST_QUEUE: Queue<UsageRecordRequest>;
+  // Optional for the retained relay path; normal deployments use direct HTTP.
+  USAGE_WRITER_RELAY?: Fetcher;
   USAGE_INGEST_PUBLISH_SECRET?: string;
   USAGE_INGEST_MODE?: string;
   USAGE_INGEST_PARALLELISM?: string;

services/usage-ingest/src/writer-relay.runtime.test.ts

@@ -3,7 +3,7 @@ import { createTestHarness, unstable_readConfig } from 'wrangler';
 import { expect, it, vi } from 'vitest';
 import { usage } from './usage.test-fixture';
 
-it('keeps production direct and routes the single staging EU consumer through the private relay', () => {
+it('keeps production and the single staging EU consumer direct, with the private relay available', () => {
   const config = new URL('../wrangler.jsonc', import.meta.url).pathname;
   const production = unstable_readConfig({ config });
   const staging = unstable_readConfig({ config, env: 'staging' });
@@ -18,9 +18,7 @@ it('keeps production direct and routes the single staging EU consumer through th
     { binding: 'USAGE_INGEST_QUEUE', queue: 'usage-ingest-processing-staging-eu' },
     { binding: 'USAGE_INGEST_DLQ', queue: 'usage-ingest-processing-dlq-staging-eu' },
   ]);
-  expect(staging.services).toMatchObject([
-    { binding: 'USAGE_WRITER_RELAY', service: 'usage-writer-relay-staging' },
-  ]);
+  expect(staging.services).toEqual([]);
   expect(staging.name).toBe('usage-ingest-staging');
   expect(staging.queues.consumers).toHaveLength(1);
   expect(staging.queues.consumers).toMatchObject([
@@ -45,7 +43,7 @@ it('keeps production direct and routes the single staging EU consumer through th
   expect(relay.vars.USAGE_INGEST_DIAGNOSTICS).toBe('all');
 });
 
-it('runs the real queue -> service-binding fetch -> relay -> HTTP writer path', async () => {
+it('runs the optional queue -> service-binding fetch -> relay -> HTTP writer path', async () => {
   const calls: { path: string | undefined; body: unknown; headers: Record<string, unknown> }[] = [];
   const release = Promise.withResolvers<void>();
   const writer = createServer(async (request, response) => {
@@ -82,6 +80,8 @@ it('runs the real queue -> service-binding fetch -> relay -> HTTP writer path',
         {
           configPath: new URL('../wrangler.jsonc', import.meta.url),
           env: 'staging',
+          // Restore the optional binding only in this rollback-path fixture.
+          bindingOverrides: { USAGE_WRITER_RELAY: 'usage-writer-relay-staging' },
           vars: {
             // Direct egress here cannot reach the loopback writer. Only the relay
             // knows that target, proving this test traverses the service binding.

services/usage-ingest/worker-configuration.d.ts

@@ -1,13 +1,12 @@
 /* eslint-disable */
-// Generated by Wrangler by running `wrangler types --env-file=/dev/null --include-runtime=false --env-interface=CloudflareEnv` (hash: 7f6081d15e95320306815756ba244c38)
+// Generated by Wrangler by running `wrangler types --env-file=/dev/null --include-runtime=false --env-interface=CloudflareEnv` (hash: 529a6825960508d75e0a8d5a96e1a5d8)
 interface __BaseEnv_CloudflareEnv {
 	USAGE_INGEST_QUEUE: Queue;
 	USAGE_INGEST_DLQ: Queue;
 	USAGE_INGEST_MODE: "processing" | "receipt";
 	USAGE_INGEST_DIAGNOSTICS?: "all";
 	USAGE_INGEST_PARALLELISM?: "2";
 	USAGE_WRITER_URL?: "https://staging-app.kilo.ai/api/internal/usage/record";
-	USAGE_WRITER_RELAY?: Fetcher /* usage-writer-relay-staging */;
 }
 declare namespace Cloudflare {
 	interface GlobalProps {
@@ -20,7 +19,6 @@ declare namespace Cloudflare {
 		USAGE_INGEST_DIAGNOSTICS: "all";
 		USAGE_INGEST_PARALLELISM: "2";
 		USAGE_WRITER_URL: "https://staging-app.kilo.ai/api/internal/usage/record";
-		USAGE_WRITER_RELAY: Fetcher /* usage-writer-relay-staging */;
 	}
 	interface Env extends __BaseEnv_CloudflareEnv {}
 }

services/usage-ingest/wrangler.jsonc

@@ -38,7 +38,6 @@
         "USAGE_INGEST_PARALLELISM": "2",
         "USAGE_WRITER_URL": "https://staging-app.kilo.ai/api/internal/usage/record",
       },
-      "services": [{ "binding": "USAGE_WRITER_RELAY", "service": "usage-writer-relay-staging" }],
       "queues": {
         "producers": [
           { "binding": "USAGE_INGEST_QUEUE", "queue": "usage-ingest-processing-staging-eu" },