Develop

cloudbuild.yaml

@@ -23,7 +23,6 @@ steps:
           pubsub.googleapis.com \
           storage.googleapis.com \
           translate.googleapis.com \
-          iamcredentials.googleapis.com \
           --project=$PROJECT_ID
   - name: 'node:24'
     id: 'install-deps'
@@ -78,20 +77,6 @@ steps:
   - name: ghcr.io/lessify/firebase:edge
     id: 'firebase-deploy'
     args: [ 'deploy', '--config=firebase.$PROJECT_ID.json', '--project=$PROJECT_ID', '--only=hosting,functions,storage,firestore,auth', '--force' ]
-  # let the functions sign Storage URLs as themselves: assets over 31 MiB are redirected to one,
-  # because Hosting truncates function responses at 32 MiB (mirrors scripts/localess/url-signing.mjs)
-  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
-    id: 'grant-url-signing'
-    entrypoint: 'bash'
-    allowFailure: true
-    args:
-      - '-c'
-      - |
-        SA="$PROJECT_NUMBER-compute@developer.gserviceaccount.com"
-        gcloud iam service-accounts add-iam-policy-binding "$$SA" \
-          --member="serviceAccount:$$SA" \
-          --role=roles/iam.serviceAccountTokenCreator \
-          --project=$PROJECT_ID
 options:
   logging: CLOUD_LOGGING_ONLY
 substitutions:

docs/cdn-caching.md

@@ -37,8 +37,7 @@ Whenever the marker is rewritten, its generation increments. The content marker
 | Asset response (incl. `/original`, `/download`) | 365 days | 365 days | browser / CDN |
 | Asset `304 Not Modified` | 365 days | 365 days | browser / CDN |
 | Asset canonical-size redirect (`w`/`h` above source) | 365 days | 365 days | browser / CDN |
-| Asset over 31 MiB — `302` to a signed Storage URL (valid 24 h) | 1 hour | 1 hour | browser / CDN |
-| Asset over 31 MiB that could not be signed — `500` | no-cache | no-cache | — |
+| `413` — asset over 31 MiB (known limitation, see [v1-functions-api.md](v1-functions-api.md#known-limitations)) | no-cache | no-cache | — |
 | 400 — invalid asset transform param | 1 hour | 1 hour | browser / CDN |
 | 404 — space not found | 7 days | 7 days | browser / CDN |
 | 404 — content cache marker missing / content not found on disk (content routes only) | 10 min | 10 min | browser / CDN |

docs/deployment/check.md

@@ -136,20 +136,6 @@ invoked by Google's own service agents, so making them public would widen access
 
 ---
 
-## URL signing
-
-Firebase Hosting buffers a rewritten function response and cuts it off at 32 MiB, still answering
-`200` with a matching `Content-Length` — so a large video "downloads" as its first 32 MiB. The
-asset API therefore redirects anything over 31 MiB to a short-lived signed Storage URL. The
-functions have no key file, so they sign through the IAM Credentials `signBlob` API as their own
-runtime service account (the default compute account, `<project-number>-compute@developer.gserviceaccount.com`),
-which needs `roles/iam.serviceAccountTokenCreator` **on itself**. Nothing grants that by default.
-
-`localess:deploy` grants it after each functions deploy; `--fix` grants it here. The account only
-exists once functions have been deployed, so before that the row reads `?` and points at a deploy.
-
----
-
 ## `--fix`
 
 ```bash
@@ -165,7 +151,6 @@ Applies only the repairs that are **free and reversible**:
 | Project labels | Writes the three markers |
 | Local files / region | A full `syncLocalFiles`, propagating the live Firestore location everywhere |
 | Invoker bindings | Adds `allUsers` to `roles/run.invoker`, preserving every other binding |
-| URL signing | Grants the runtime service account `roles/iam.serviceAccountTokenCreator` on itself |
 | Admin user | Prompts for credentials, then creates the account, its admin claim and its documents |
 
 Everything else is reported with the command that does it, because it is not safe to do as a side

docs/deployment/firebase-setup.md

@@ -92,7 +92,7 @@ In order:
 1.  Preflight        firebase-tools present and >= 15.29.0, CLI authenticated
 2.  Project          adopt --project, or pick from a list / create one
 3.  Billing          link a billing account, then wait for it to become active
-4.  APIs             check the 16 required APIs, enable whichever are off
+4.  APIs             check the 15 required APIs, enable whichever are off
 5.  Firestore        adopt the existing location, or ask and create in --region
 6.  Storage          create and link the default bucket in --region
 7.  Storage CORS     allow GET and HEAD from any origin, when the bucket has no rules
@@ -116,15 +116,8 @@ firestore             identitytoolkit        firebasestorage
 firebaseextensions    cloudfunctions         cloudbuild
 artifactregistry      run                    eventarc
 pubsub                storage                translate
-iamcredentials
 ```
 
-`iamcredentials.googleapis.com` is what the functions sign Storage URLs through — assets over
-31 MiB are redirected to a signed URL, because Firebase Hosting truncates function responses at
-32 MiB. Signing also needs the runtime service account to hold
-`roles/iam.serviceAccountTokenCreator` on itself; deploy grants that after the functions are up
-(the account does not exist before), and `localess:check` reports it.
-
 ### Authentication
 
 `firebase.json` carries an `auth` block, which the `auth` deploy target sends to Google's

docs/deployment/overview.md

@@ -89,6 +89,18 @@ you do not.
 
 ---
 
+## Known issues
+
+- **Assets over 31 MiB cannot be downloaded through the API.** Firebase Hosting truncates function
+  responses at 32 MiB while still reporting success, so the asset API refuses larger objects with
+  `413` rather than serve a corrupt file. Uploads are unaffected and the file stays intact in
+  Storage. A self-hosted import from such an install skips those assets with an error — copy them
+  from the bucket by hand (`gsutil cp gs://<bucket>/spaces/<spaceId>/assets/<assetId>/original .`).
+  Lifting the limit needs a signed-URL redirect, which needs a manual IAM grant per install; see
+  [Known limitations](../v1-functions-api.md#known-limitations).
+
+---
+
 ## Two deployment models
 
 This documentation covers **local deploy**: an operator runs the commands from a checkout on their

docs/v1-functions-api.md

@@ -37,9 +37,17 @@ Content delivery with cache-busting and asset transformation. All content/transl
 - **`thumbnail` param** — Collapses an animated WebP/GIF to its first frame, and extracts a video frame via FFmpeg (requires `w`). Has no effect on other image types. Since animations now resize with every frame intact, `thumbnail` is how you ask for a *still* rather than how you make resizing work.
 - **EXIF orientation is applied** — Sharp strips the orientation tag on re-encode, so a rotated source is baked into the pixels. Without it a portrait phone photo returned landscape with its aspect ratio transposed.
 - **Embedded colour profiles are carried through** — via `keepIccProfile()`. Sources without one gain nothing; tagging every response as sRGB would add ~506 bytes each.
-- **`/original` and `/download`** — serve the stored bytes exactly as uploaded, inline and as an `attachment` respectively. Since a bare transform request re-encodes, these are the only way to retrieve the original file. Neither enters Sharp, and both reject `w`/`h`/`q`/`f`/`fit`/`thumbnail`/`download` with `400` rather than ignoring them. Objects over 31 MiB are answered with a `302` to a signed Storage URL (24 h validity, redirect cached 1 h) carrying the same `Content-Type` and `Content-Disposition`, because Firebase Hosting truncates function responses at 32 MiB; smaller ones are streamed from Storage with `Range` support. The same applies to a bare `/assets/:assetId` request for a non-image type (video, audio, PDF). Signing needs the runtime service account to hold `roles/iam.serviceAccountTokenCreator` on itself — see [check.md](deployment/check.md#url-signing).
+- **`/original` and `/download`** — serve the stored bytes exactly as uploaded, inline and as an `attachment` respectively. Since a bare transform request re-encodes, these are the only way to retrieve the original file. Neither enters Sharp, and both reject `w`/`h`/`q`/`f`/`fit`/`thumbnail`/`download` with `400` rather than ignoring them. Both stream from Storage with `Range` support; objects over 31 MiB are refused with `413` — see [Known limitations](#known-limitations).
 - **Removed in v4** — the `?download` flag and `f=original`. Both return `400` with a message naming the replacement route. Responses already cached under the old spellings keep serving for the remainder of their 365-day TTL.
 
+#### Known limitations
+
+- **Assets over 31 MiB cannot be served (`413`).** Firebase Hosting buffers every rewritten function response and cuts it off at 32 MiB — while still answering `200` with a matching `Content-Length`, so the client believes the transfer completed. Observed on a 104 MiB `.mov`: through `*.web.app` it arrived as its first 33,553,834 bytes; called directly on Cloud Run the function sent all 108,726,933. Streaming does not help — chunked encoding gets past Cloud Run's own 32 MiB limit, not Hosting's.
+
+  So the API **refuses** such objects rather than send a truncated body: `/original`, `/download`, and a bare `/assets/:assetId` on a passthrough type (video, audio, PDF) answer `413` with `Cache-Control: no-cache` and a plain-text explanation. Clients must treat any non-`200` as a failure — the self-hosted Firebase import does, and skips the asset instead of storing a broken copy. Uploading still works; the file is intact in Storage and can be fetched from the Firebase console or `gsutil`.
+
+  The fix is a `302` to a short-lived signed Storage URL. It was implemented and reverted, because signing without a key file needs the functions' runtime service account to hold `roles/iam.serviceAccountTokenCreator` **on itself**, and that grant needs an account with IAM admin rights — Cloud Build's deploy account cannot apply it, so it would be a manual step on every install. Limit: `MAX_SERVED_BYTES` in `functions/src/utils/stream-object.ts`.
+
 #### Asset resize combinations (`w` / `h`)
 
 Sharp is called as `resize(width ?? null, height ?? null, { fit })`, defaulting to `cover` when `fit` is not given.

functions/src/utils/stream-object.ts

@@ -3,26 +3,14 @@ import { logger } from 'firebase-functions';
 import type { Readable } from 'stream';
 
 /**
- * Largest object served through the function; anything bigger is redirected to Storage.
- *
- * Firebase Hosting buffers a rewritten function response and cuts it off at 32 MiB — a 104 MiB
- * video arrived as its first 32 MiB with a `200` and a matching `Content-Length`, so the browser
- * reported a finished download. Chunked encoding does not help: it gets past Cloud Run's own
- * 32 MiB limit, but not past Hosting's. Kept a little under the limit to leave room for headers.
- */
-export const MAX_PROXIED_BYTES = 31 * 1024 * 1024;
-
-/**
- * How long a signed Storage URL stays valid. Long enough that a video player can still seek
- * (each seek is a fresh `Range` request against the signed URL) well after the page loaded.
- */
-export const SIGNED_URL_TTL_SECONDS = 24 * 60 * 60;
-
-/**
- * How long the redirect itself may be cached, by the browser and by the Hosting CDN. Well under
- * the signed URL's lifetime, so a cached redirect never points at an expired signature.
+ * Largest object this API will serve. Known issue: Firebase Hosting buffers a rewritten function
+ * response and cuts it off at 32 MiB while still answering `200` with a matching
+ * `Content-Length`, so a larger asset would arrive silently truncated — and a client such as the
+ * self-hosted import would store the broken copy as if it were complete. Anything bigger is
+ * refused outright instead (see docs/v1-functions-api.md, "Known limitations"). Kept a little
+ * under the limit to leave room for the headers.
  */
-export const SIGNED_REDIRECT_MAX_AGE = 60 * 60;
+export const MAX_SERVED_BYTES = 31 * 1024 * 1024;
 
 export type ByteRange = { start: number; end: number };
 
@@ -58,99 +46,38 @@ export function parseByteRange(header: string | undefined, size: number): ByteRa
 }
 
 /** The slice of a Storage `File` this needs, so tests can hand in a fake. */
-export interface StoredFile {
+export interface StreamableFile {
   createReadStream(options?: { start?: number; end?: number; validation?: false }): Readable;
-  getSignedUrl(config: {
-    version: 'v4';
-    action: 'read';
-    expires: number;
-    responseDisposition?: string;
-    responseType?: string;
-  }): Promise<[string]>;
-}
-
-/** The headers the stored object must be served with, wherever it is served from. */
-export interface StoredObjectHeaders {
-  contentType: string;
-  contentDisposition: string;
 }
 
 /**
- * Serves a stored object: streamed through the function when Hosting can carry it, otherwise a
- * `302` to a short-lived signed Storage URL. The caller sets the type, disposition, cache and
- * security headers first; the redirect overrides the cache header and repeats type and
- * disposition as signed response overrides, so Storage serves the same filename.
+ * Streams a stored object to the response, honouring a single `Range`. Headers other than
+ * length and range (type, disposition, cache) must already be set by the caller.
  *
- * Signing calls the IAM Credentials `signBlob` API, so the runtime service account needs
- * `iam.serviceAccounts.signBlob` on itself (`npm run localess:check -- --fix` grants it). When
- * that is missing this answers an uncached `500` — never the truncated body Hosting would make
- * of a streamed one.
+ * Replaces `download()` to `/tmp` + `sendFile`, which held the whole object in memory: `/tmp` is
+ * RAM on Cloud Functions. Objects above {@link MAX_SERVED_BYTES} are refused with `413`.
  * @param {Request} req Incoming request, read for `Range` and the method
- * @param {Response} res Response to write
- * @param {StoredFile} file Storage object to serve
+ * @param {Response} res Response to stream into
+ * @param {StreamableFile} file Storage object to read
  * @param {number} size Object size in bytes, from its metadata
- * @param {StoredObjectHeaders} headers Type and disposition to serve the object with
  * @param {string} tag Log prefix of the calling route
- * @return {Promise<void>} resolves once the response is handed off
  */
-export async function serveStoredObject(
-  req: Request,
-  res: Response,
-  file: StoredFile,
-  size: number,
-  headers: StoredObjectHeaders,
-  tag: string
-): Promise<void> {
-  if (size <= MAX_PROXIED_BYTES) {
-    streamStoredObject(req, res, file, size, tag);
-    return;
-  }
-  let url: string;
-  try {
-    [url] = await file.getSignedUrl({
-      version: 'v4',
-      action: 'read',
-      expires: Date.now() + SIGNED_URL_TTL_SECONDS * 1000,
-      responseDisposition: headers.contentDisposition,
-      responseType: headers.contentType,
-    });
-  } catch (error) {
-    logger.error(`${tag} could not sign a Storage URL for a ${size}-byte object`, error);
+export function streamStoredObject(req: Request, res: Response, file: StreamableFile, size: number, tag: string): void {
+  if (size > MAX_SERVED_BYTES) {
+    // Refused with an explicit status rather than streamed: Hosting would truncate the body and
+    // still report success. Uncached, so lifting the limit later takes effect immediately.
+    logger.warn(`${tag} refused a ${size}-byte object: above the ${MAX_SERVED_BYTES}-byte limit Hosting can serve`);
     res.removeHeader('Content-Disposition');
     res
-      .status(500)
+      .status(413)
       .header('Cache-Control', 'no-cache')
       .contentType('text/plain')
       .send(
-        'This asset is too large to serve through Hosting, and the function could not sign a Storage URL for it. ' +
-          'Grant the functions service account roles/iam.serviceAccountTokenCreator on itself: npm run localess:check -- --fix'
+        `This asset is ${size} bytes, above the ${MAX_SERVED_BYTES}-byte limit the API can serve through Firebase Hosting. ` +
+          'Known issue: see "Known limitations" in docs/v1-functions-api.md.'
       );
     return;
   }
-  logger.info(`${tag} ${size} bytes, redirecting to Storage`);
-  res.removeHeader('Content-Disposition');
-  res.header('Cache-Control', `public, max-age=${SIGNED_REDIRECT_MAX_AGE}, s-maxage=${SIGNED_REDIRECT_MAX_AGE}`).redirect(302, url);
-}
-
-/**
- * Streams a stored object to the response, honouring a single `Range`. Headers other than
- * length and range (type, disposition, cache) must already be set by the caller.
- *
- * Replaces `download()` to `/tmp` + `sendFile`, which held the whole object in memory: `/tmp`
- * is RAM on Cloud Functions.
- * @param {Request} req Incoming request, read for `Range` and the method
- * @param {Response} res Response to stream into
- * @param {StoredFile} file Storage object to read
- * @param {number} size Object size in bytes, from its metadata
- * @param {string} tag Log prefix of the calling route
- */
-export function streamStoredObject(
-  req: Request,
-  res: Response,
-  file: Pick<StoredFile, 'createReadStream'>,
-  size: number,
-  tag: string
-): void {
   res.header('Accept-Ranges', 'bytes');
   const range = parseByteRange(req.headers.range, size);
   if (range === 'unsatisfiable') {