Repository navigation
Conversation
The document was copied from NodeNorm Redis and never pruned, so it advertised TRAPI 1.5 (info.x-trapi), carried 145 schemas no path referenced, named RENCI's developer as the contact and sent readers to /get_curie_prefixes, which this service has never served. It also failed openapi-spec-validator: the /get_allowed_conflations response had no description, and FilterKgraphTopNParameters.max_edges declared maximum 0 (NCATSTranslator/NodeNormalization#409). Rewritten from scratch against build_handlers(): the six routes this service registers, nothing else. Responses that were `schema: {}` now have schemas (NormalizedNodes, SetIDResponse), the FastAPI 422 responses are replaced by the 400 the Tornado handlers raise, and the examples are taken from the live CI instance. The servers block lists the ci and test deployments at ITRB instead of RENCI. Contact is Will Liu and the team is DOGSURF (biothings#35); infores and termsOfService are unchanged for now. info.version is bumped to 1.1.0 because /status reports it and the response shapes change in the next commit. handlers/curie_prefix.py was an unregistered copy of the semantic-types handler (its class is even named SemanticTypeHandler), so it goes too. Refs biothings#35, biothings#36 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GET /get_allowed_conflations finished a bare list, and Tornado's write()
refuses a top-level list ("Lists not accepted for security reasons"),
so it returned HTTP 500 on every deployment. It now returns
{"conflations": [...]}, which is also what NodeNorm Redis returns.
GET /get_semantic_types lower-cased every class ("biolink:namedthing");
NodeNorm Redis returns the Biolink class URI as is ("biolink:NamedThing").
/openapi.json now redirects to /webapp/openapi.json so the FastAPI
default address used by NodeNorm Redis, SmartAPI and Babel Validation
works here too; it is excluded from OpenTelemetry like the other static
paths.
The POST set-ID test posted a single object where the handler expects
a list of sets, and the normalized-nodes docstring showed a "curie" key
where the handler reads "curies"; both corrected. POST /get_setid still
answers with an object keyed "0", "1", ... (the same Tornado limitation);
returning a list as NodeNorm Redis does is a breaking change, so it is
left to its own pull request.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The document is hand-written and served as a static file, so these tests are what stops it drifting: the spec validates; its paths are exactly the routes build_handlers() registers; each path documents the GET/POST methods its handler defines; each GET documents the query parameters its handler reads (found by scanning the handler source for get_argument calls); and the SetIDResponse schema lists the dataclass's fields. Each was checked to fail when the spec is mutated. Also a regression test for /get_allowed_conflations, which returned HTTP 500 until the previous commit: it mounts just that handler and validates the body against the documented ConflationList schema. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
It ran only against the su10 Elasticsearch host before. This mounts the handler alone with a fake Elasticsearch client and validates the body against the documented schema. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dot in each redirect pattern was unescaped, so /openapiXjson and /faviconXico redirected too. The OpenTelemetry exclusions already escape it. A test now checks the /openapi.json redirect and that a near-miss path is a 404. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ment
The query-parameter check scanned only get() for get_argument calls, so
a parameter read in a helper method would drift unnoticed; it now scans
the whole handler class. A new check compares the fields each post()
reads with .get("...") against the properties of the documented request
body (CurieList, and the SetIDQuery items of the /get_setid array).
Both were confirmed to fail when a field is removed from the document or
a parameter is renamed in a helper.
The checks still only see literal names; the README now says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 tasks
generate_setid() returned the SetIDResponse dataclass itself on the invalid-conflation and no-normalized-CURIEs paths. Tornado can only write dicts, so GET /get_setid?conflation=Bogus answered 500 and the documented `error` field never reached anyone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The body has to be a JSON array of {curies, conflations} objects. A
single object (the shape the old document described) passed the length
check and then failed with an AttributeError, and invalid JSON failed in
json.loads; both were 500s. They are now 400s that say what shape the
body must have.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The index mapping (for the Babel version) was read before the try
block, so an unreachable cluster raised and /status answered 500
instead of the documented {"status": "error"}. The read now happens
inside the try; babel_version and babel_version_url are null when the
mapping couldn't be read, and the document says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The patterns were written r"/get_setid?", r"/status?" and so on. In a Tornado route that ? makes the last character optional, so /get_seti and /statu reached the handlers though the OpenAPI document lists only the full paths. test_openapi.py hid this by stripping the ? before comparing routes with the document; it now compares them as written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request-body check read only post()'s source, though the README says fields are found by scanning the handler, and the query-parameter check already scans the whole class. A body field read in a helper method was either missed or reported as undocumented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The methods check compared only GET and POST, so the HEAD handler on /get_allowed_conflations, which the document didn't list, went unnoticed. It now compares every method Tornado routes except OPTIONS (answered by the base handler for CORS), and the document lists HEAD. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/test_openapi.py imports it directly; it was only installed because openapi-spec-validator happens to depend on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parse_boolean() treated every string except "false" as true, so
conflate=0 or conflate=no turned conflation on. The flags now accept the
strings FastAPI/Pydantic accepted in NodeNorm Redis (true/false, 1/0,
yes/no, on/off, t/f, y/n, in any case) and answer 400 for anything else,
including an empty value. POST bodies go through the same check, so
{"conflate": "false"} is no longer truthy. The document lists the
accepted values.
This is a behavior change: a caller sending an unrecognized value now
gets a 400 where it used to get conflation silently switched on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers the second path biothings#47 describes: a set whose CURIEs normalize to nothing. get_normalized_nodes() is patched out, so it needs no host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
https://nodenorm-es.transltr.io/ is now live. It goes first, since Swagger UI and generated clients default to the first server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
json.loads() raises UnicodeDecodeError rather than JSONDecodeError for bytes that aren't valid UTF-8, so such a body escaped the check and became a 500, though the document promises a 400 for a malformed body. The decoding moves to a load_json_body() helper on the base handler so POST /get_normalized_nodes can use it too. Raised by Copilot on biothings#42. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The handler called json.loads() with no error handling and then .get() on the result, so a body that wasn't JSON, wasn't UTF-8, or was an array instead of an object all became 500s. It now uses load_json_body() and checks for an object, and the document lists these under the 400 response. Same cause as the UTF-8 case Copilot raised for POST /get_setid on biothings#42. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tornado's write() refuses a list ("Lists not accepted for security
reasons"), a guard against JSON hijacking through <script> tags in very
old browsers. That made GET /get_allowed_conflations return HTTP 500,
and made POST /get_setid answer with an object keyed "0", "1", ...
instead of the list NodeNorm Redis returns. NodeNormalizationBaseHandler
now encodes a list the way Tornado encodes a dict, so every handler can
finish() one.
/get_allowed_conflations keeps its {"conflations": [...]} wrapper, which
is the shape NodeNorm Redis returns; only its comment changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /get_setid answered with an object keyed "0", "1", ... because Tornado would not write a list. Now that the base handler can, it returns the list of SetIDResponse objects in input order, which is what NodeNorm Redis returns and what the OpenAPI document now says. This is a breaking change for any client that reads body["0"]. The "if not set_identifiers: 500" check after the loop could never fire (the body is already known to be non-empty and each set appends one result), so it is gone. The unit test that mounts the handler with a fake generate_setid() used to assign the module attribute and never put it back, so under pytest-randomly the fake could leak into tests/test_set_identifiers.py. It now patches with unittest.mock and undoes the patch after each test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit used to add the POST /get_setid body checks themselves: a body that wasn't valid JSON, was a single object, or held anything other than objects reached group.get() and failed with HTTP 500. biothings#42 now makes those checks (through load_json_body()) and tests them, so after rebasing onto it all that is left is the empty-body case, which its test didn't cover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each set awaited its own Elasticsearch lookups before the next began, so a POST with many sets took one round trip per set in series. The sets are independent, so asyncio.gather() now runs them together; it returns the results in input order. This raises the number of concurrent Elasticsearch requests one POST can make, in proportion to the number of sets (see biothings#22 on how much load the cluster takes). It is a separate commit so it can be dropped on its own if that is a concern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gaurav
force-pushed
the
setid-post-list
branch
from
October 14, 2026 17:35
85e22d6 to
22f1970
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #42 — review only the last four commits (from "Let handlers write a top-level JSON list" on). Until #42 merges, this PR's diff against
mainincludes it.POST /get_setidanswers with an object keyed by input index ({"0": {...}, "1": {...}}) because Tornado refuses to write a top-level list. NodeNorm Redis returns a list, and the clients were written against it. This PR letsNodeNormalizationBaseHandlerwrite lists, and then returns theSetIDResponseobjects as a list in input order, as the OpenAPI document now says. This is a breaking change for any client of NodeNorm ES that readsbody["0"].Calls worth overruling
<script>tags in very old browsers, which doesn't matter for a public JSON API that allows any origin. Allowing lists in the base handler means the next handler that returns a list doesn't hit the same 500/get_allowed_conflationsdid. The comment onwrite()inhandlers/base.pysays why.asyncio.gather. One POST can now make as many concurrent Elasticsearch requests as it has sets (see Is there a limit to how many simultaneous CURIEs NodeNorm ES can support? #22 on cluster load). It's the last commit, so it can be dropped on its own.info.versionstays at 1.1.0, the bump in Rewrite the OpenAPI document to describe the Elasticsearch service, and make the endpoints match it #42, on the assumption that both PRs ship in the same release. If Rewrite the OpenAPI document to describe the Elasticsearch service, and make the endpoints match it #42 is deployed first, this one should bump it again, because/statusreports it.What's here
POST /get_setidreturns a JSON array; the unreachable "empty result → 500" check after the loop is gone. The OpenAPI document's 200 response for it is now an array ofSetIDResponse.tests/test_openapi.pythat mounts the handler with a fakegenerate_setid()patched throughunittest.mock, so the fake doesn't leak intotests/test_set_identifiers.pywhen pytest-randomly runs that later. Rewrite the OpenAPI document to describe the Elasticsearch service, and make the endpoints match it #42's tests of the POST errors now read the list, and cover an empty body as well.tests/test_set_identifiers.pyexpects a list.What it deliberately does not do
The POST body checks (400 for a body that isn't a non-empty JSON array of objects) and the
generate_setid()fixes for unknown conflations and empty sets are in #42, not here.Verification
On 22f1970,
openapi-spec-validatorpasses, and the whole suite passes under several random orderings except the four integration tests that need thesu10host:test_node_normalization_getandtest_node_normalization_postintest_normalized_nodes.py, andtest_get_endpointandtest_post_endpointintest_set_identifiers.py. So the list-expecting integration test hasn't been run.Nothing is blocking beyond #42 merging first.
🤖 Generated with Claude Code