Skip to content

Apply the configured Tornado settings, with debug mode off when deployed, and trim the deployment config to its own settings - #50

Open
gaurav wants to merge 10 commits into
biothings:mainfrom
TranslatorSRI:deployment-config
Open

gaurav wants to merge 10 commits into
biothings:mainfrom
TranslatorSRI:deployment-config

Conversation

@gaurav

@gaurav gaurav commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

None of the webserver.SETTINGS in the config files has ever reached Tornado: get_app() passed them positionally, into Application's default_host parameter. This PR makes them take effect, with debug mode off in the deployment, and cleans up the configuration around them. docker/configuration/config.json was a full copy of config.default.json because the --conf file was merged with dict.update(), which replaces any section it touches; it is now merged section by section, so the deployment file holds only what the deployment changes.

No issue tracks this; it came out of reading the config while reviewing #42. Nothing was leaking before: the deployments ran with Tornado's defaults, and a 500 never carried a traceback. The debug-off setting matters from this PR on, because the old deployment config asked for debug mode, which puts the traceback and exception message in every 500 body.

Calls worth overruling

  • Tornado's static_path is no longer set. load_configuration() added it with static_url_prefix: "/", and once settings apply, Tornado turns that into a catch-all route ahead of every handler, so every API path answers 404. build_handlers() already serves / and /webapp/, and a comment at the old spot says why not to add it back.
  • The development default keeps debug mode and autoreload on, as it always asked for; they now actually apply when running without --conf, including in the integration tests that build the app from the default.
  • The Helm env vars are removed, not wired in. The chart set ES_HOST, OPENTELEMETRY_ENABLED and PORT, which the app never read. PORT is 8000, where Caddy listens in front of the app (on 9001), and the real ES_HOST is in ITRB's values-ci.yaml, outside this repo, so I can't confirm it is a URL the client accepts. Driving these from the chart would be a follow-up for someone who can see values-ci.yaml.
  • ES_INDEX and ES_ALIAS are repeated in the deployment file though they match the default, so a change to the default for local development can't silently repoint production; the price is that a rename touches both files. Everything else the deployment leaves out follows the default. The README's Configuration section records this.
  • The image takes its config files from the builder's clone of NODENORM_BRANCH, like its code, instead of from the build context, so the two always come from the same commit, and docker-compose.yml (context: the repo root) builds again. A local config edit now needs pushing to a branch before an image picks it up, as code edits already do.

What's here

  • merge_configuration() in namespace.py; lists are still replaced whole.
  • docker/configuration/config.json names only the in-cluster Elasticsearch host, index and alias, telemetry on with the cluster's Jaeger collector, and debug and autoreload off.
  • tests/test_configuration.py tests the merge, checks that the deployment file names no setting the default lacks (so a misspelt key fails), and builds the app through get_app() from the deployment config to check the settings arrive and the routes still answer.
  • The Dockerfile no longer copies config.default.json to an unused configuration/default.json; docker-compose.yml's build args (written ARG NAME=value, so never applied) and port mapping (to container port 9000, where nothing listens) are fixed; the README has a Configuration section.

One commit, "Turn off Tornado debug mode in the deployed service", says the deployments served tracebacks; they didn't, and the later commit "Pass the webserver settings to Tornado as keywords" says so.

This conflicts with #42 in two easy places: #42 adds ^/openapi\.json$ to OPENTELEMETRY_EXCLUDED_URLS in config.json, which this PR removes the list from (keep the default's line, drop the config.json one), and both add a README section at the end (keep both).

Verification

On 78d7404, every test passes except the four set-ID and normalized-node integration tests that need the su10 host. The image, built from the pushed branch with docker/ as the context as Jenkins does, ran against a local Elasticsearch 8.19 container answering to the in-cluster hostname: /status, /, /version and /webapp/openapi.json answered 200, and a 500 showed no traceback; the same image with main's debug-on config.json mounted showed the full traceback. docker compose build succeeds from the repo root, where main's Dockerfile fails, and helm template renders the chart.

🤖 Generated with Claude Code

gaurav and others added 10 commits October 12, 2026 02:01
load_configuration() laid the --conf file over config.default.json with
dict.update(), which only merges at the top level: an override naming
one key of a section replaced the whole section. That is why
docker/configuration/config.json had to repeat the entire default, and
why every shared setting had to be edited in both files. Nested
sections are now merged key by key; lists are still replaced whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docker/configuration/config.json repeated all of config.default.json
and differed in three values: the in-cluster Elasticsearch host, and
telemetry turned on with the cluster's Jaeger collector. Now that --conf
files are merged section by section, it lists just those three. Merged
over the default, it gives exactly the configuration it used to spell
out. A test checks it names no setting the default lacks, so a misspelt
key fails instead of being ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deployment inherited "debug": true and "autoreload": true from the
development default. In debug mode Tornado puts the full traceback and
exception message in the body of every 500, so the public deployments
were showing their internals to anyone who triggered an error; it also
watches the source for changes. The default keeps both on for local
development.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The chart set ES_HOST, OPENTELEMETRY_ENABLED and PORT, but NodeNorm
doesn't use biothings' settings loader (the one that honors env vars)
and reads these settings only from its config files. Changing them in
the chart did nothing. Removed, with a comment saying where the
settings do come from.

Wiring them in instead was considered and rejected for now: PORT is
8000, the port Caddy listens on in front of the app (which runs on
9001), so honoring it would collide; and the real ES_HOST is set in
ITRB's values-ci.yaml, which isn't in this repo and which the chart
also uses as a bare hostname with a separate port in an NCATS
annotation, so it may not be a URL the client accepts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The builder stage clones NODENORM_BRANCH for the code, but config.json,
supervisord.conf and the Caddyfile were copied from the build context.
An image could therefore pair one commit's code with another's
configuration, which now matters: the slimmed config.json needs the
section-by-section merge. It also broke docker-compose.yml, whose build
context is the repository root, where configuration/ doesn't exist.
All three now come from the clone.

The Dockerfile also copied config.default.json to
configuration/default.json, which nothing reads (the default is loaded
from the installed package); that copy is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The args were written "ARG NODENORM_REPO=...", which Compose reads as
an argument named "ARG NODENORM_REPO", so the Dockerfile's own
defaults were used whatever the file said. The port mapping sent host
port 9000 to container port 9000, where nothing listens: Caddy serves
on 8000 and proxies to the app on 9001.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They match the default, so the slimmed config.json had dropped them,
but they name the index in the deployment's cluster: left to the
default, a change made there for local development would repoint
production on the next build with nothing to flag it. The README says
why they are repeated and what that costs (a rename touches both
files), so maintainers can weigh it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
load_configuration() added static_path (the webapp directory) and
static_url_prefix "/" to the webserver settings. Those settings have
never reached Tornado (see the next commit), but once they do, a
static_path makes Tornado insert a catch-all "/(.*)" route ahead of
every handler, and every API path, /webapp/openapi.json included,
answers 404. build_handlers() already routes / and /webapp/ to the
webapp directory, so nothing needs them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
get_app() called cls(handlers.values(), settings). Application's second
positional parameter is default_host, so the whole webserver.SETTINGS
block was stored there and none of it took effect: no deployment, and no
local run, has ever had debug mode or autoreload on. This corrects the
earlier commit "Turn off Tornado debug mode in the deployed service",
which said the deployments served tracebacks in their 500 pages; they
didn't, and with main's debug-on config mounted into the image a 500
still showed none. Its debug: false only takes effect from this commit,
and the development default's debug and autoreload now do too.

Tests build the app through get_app() from the deployment config and
check that the settings arrive and that the routes still reach their
handlers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gaurav gaurav changed the title Turn off debug mode in the deployed service and keep only its own settings in its config Apply the configured Tornado settings, with debug mode off when deployed, and trim the deployment config to its own settings Oct 12, 2026
@gaurav
gaurav marked this pull request as ready for review October 12, 2026 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant