Repository navigation
Conversation
- ARAX_database_manager.symlink_database() now runs `ln -sfn`, so the CLI update_databases() path no longer dies when a link already exists. It raises FileExistsError instead of replacing a real file. - generate-db-symlinks.sh now reads every database path from config_dbs.json (it had fda on the wrong tier0 dir and NGD files at v1.0). DB_DIR and LATEST_TIER0_VER overrides still work. - Add test_ARAX_database_manager.py for symlink_database().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes two problems in the ARAX database tooling. They were found in the 2026-10-01 audit of arax.ncats.io.
generate-db-symlinks.shlinks files thatconfig_dbs.jsondoes not use anymore.1. Database manager crashes with "File exists"
On master,
symlink_database()runs a plainln -s. If something is already at the link path,lnfails withFile existsand the whole run stops withCalledProcessError.This happens when the links on disk and
db_versions.jsondo not agree. Three cases I reproduced:os.path.exists(), which returns False for a broken link. So the manager tries to create the link again and fails.db_versions.jsonhas no entry for a database, but its link is already there.ARAX_database_manager.py -fon an instance that already has links.The same CLI run is a startup step in three places:
start_app.shL21. The script hasset -euo pipefail, so if the manager fails here, the script exits before Apache and ARAX start.deploy/preview/deploy.sh, at L632 (fast path) and L720 (full rebuild).The fix:
RTX/code/ARAX/ARAXQuery/ARAX_database_manager.py
Lines 374 to 379 in a75a874
ln -sfnreplaces a link that is already there, so a second run gives the same result as the first. But-fwould also delete a real file at that path, and that file can be a full database copy. So the function now raisesFileExistsErrorfor a real file and leaves it alone. The startup code already had the same rule.The Flask startup on arax.ncats.io (
__main__.pyL168-L172) usessymlink_from_central_and_write_versions(). That function removes a stale link itself before it callssymlink_database(), so it works the same as before.2.
generate-db-symlinks.shdid not matchconfig_dbs.jsonThe old script had every file name written by hand, and some of them were out of date:
RTX/code/generate-db-symlinks.sh
Lines 48 to 50 in 28bf11b
curie_ngdandcurie_to_pmidswere v1.0, butconfig_dbs.jsonsays v1.1. ARAX opens the file name from the config (compute_ngd.py,Path_Finder/utility.py), so in a dev setup made by this script ARAX could not find the NGD databases.fda_approved_drugs_v1.0.picklewas taken fromtier0-20260621, but the config points totier0-20260408.Now the script reads all paths from
config_dbs.jsonwithpython3. When someone changes a version in the config, the script does not need a change. The only thing still written in the script is which RTX folder each database goes to. These are the same folders as inARAX_database_manager.py. If the config gets a new database that the script does not know, the script stops before it creates any link and says what to add.The overrides for local setups still work:
DB_DIR: edit it in the script like the wiki says, or set it in the environment.LATEST_TIER0_VER: by default it is the tier0 version from the config. If you set it, it changes only the files in that tier0 folder. fda stays ontier0-20260408.One new link:
tier0_sqlitenow goes toKnowledgeSources/Tier0/, becauseARAX_decorator.pyreads it from there. Nothing in the repo created this link before.Testing
test_ARAX_database_manager.pywith 4 cases: same link twice, stale link, broken link, real file. All 4 fail on master and pass on this branch. In the Test Build run for this PR they passed together with the rest of the suite (181 passed, 144 skipped). The "Create symlinks" step in that run used the new code on Linux and linked all 7 databases without errors.File existsin 5 cases: same link twice, stale link, broken link inupdate_databases(), missingdb_versions.jsonentry, and-f. With this PR all 5 work.symlink_from_central_and_write_versions()gives the same result as before in all its cases./bin/bash3.2 and checked that all 8 links matchconfig_dbs.json. I also tried a second run, aLATEST_TIER0_VERoverride, an unknown database key, a missing source file, and a real file at the destination.Not in this PR
update_databases()does not fix a link when only its target changed and the file name stayed the same. For example, an fda link that points intotier0-20260621stays there, because the version is still1.0and the manager skips it. I did not check if any host has such a link. This will be a separate PR.