Repository navigation
Conversation
…parser hierarchies
copybara-service Bot
pushed a commit
that referenced
this pull request
Oct 22, 2026
Add pure python Tree-sitter base parser framework. This change adapts the core Tree-sitter AST traversal routines from GitHub PR #27 (#27) and refactors them into `GenericTreesitterParser` with unit tests. Co-authored-by: Awe Srush <srushti.erudite@gmail.com> Co-authored-by: Avantika Sharma <avantikaspare15@gmail.com> Co-authored-by: Ishika Johari <ishijohari@gmail.com> PiperOrigin-RevId: 978013230 Change-Id: I8e5edd0f3e44185d878f3b1c953227d38d53f589 GitOrigin-RevId: 0b3f04926bfe3ad782f39725b558fc6f57dc7b94
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Python language support to Vanir via a tree-sitter-backed parser, along with the shared infrastructure required to plug additional tree-sitter languages into the parser pipeline in the future.
This PR consolidates #21–#25 into a single atomic change as requested on the original PRs, so CI can validate the complete Tree-sitter Python integration in one pass. Each commit preserves its original author for attribution.
What this adds
Vanir can now scan
.pyfiles for missing security patches. Once merged, no changes to any upstream Vanir logic are required - the newPythonParseris auto-discovered byget_parser_class()and dispatched for.pyfiles on Python 3.10+. On Python 3.9, tree-sitter dependencies are not installed and.pyfiles are silently skipped.Commits (in dependency order)
1.
feat(language_parsers): add TreeSitterParserBase— @awe-srushIntroduces a shared abstract base class between
AbstractLanguageParserand future tree-sitter-backed parsers. Handles the fullParseResultsassembly pipeline (function discovery via tree-sitter queries,FunctionChunkBase/LineChunkBaseconstruction,ParseErrorcollection) using iterative cursor-based DFS to avoid Python recursion limits. Language-agnostic — all grammar-specific logic is delegated to three abstract methods (_extract_param_names,_extract_annotations,_collect_locals_calls) that concrete subclasses implement. Tree-sitter imports are lazy so the module remains importable on Python 3.9.2.
bug(language_parsers): fix subclass discovery to support multi-level parser hierarchies— @awe-srushFixes a bug in
get_parser_class()where parsers registered two or more levels deep in theAbstractLanguageParserhierarchy were silently ignored. The existing implementation used__subclasses__(), which only returns direct subclasses — meaning any parser inheriting throughTreeSitterParserBasewas invisible. Replaces it with a recursive_all_subclasses()helper that walks the full hierarchy and filters out abstract intermediates via__abstractmethods__. Existing C++ and Java parser behavior is unchanged.3.
feat(language_parsers): add Python parser via tree-sitter— @awe-srushConcrete
PythonParseron top ofTreeSitterParserBase. Implements all three abstract methods against Python's tree-sitter grammar, handling the full range of Python parameter forms (typed, defaulted,*args,**kwargs, positional-only markers), return/parameter type annotations, and local/call collection through a compiled query covering standard assignments, augmented assignments, for-loop targets, walrus expressions, and direct calls. Nested-function byte ranges are excluded so inner-scope names don't leak into the outer function's signature.4.
feat(requirements): add tree-sitter dependencies for Python parser— @avshar15Adds
tree-sitterandtree-sitter-pythontorequirements.txt, both guarded withpython_version>='3.10'so Python 3.9 environments remain unaffected.5.
fix(requirements): bound tree-sitter version to <0.26— @avshar15Pins the upper bound after API changes in tree-sitter 0.26.
6.
feat(language_parsers): add Bazel build targets for Python parser— @ishijoAdds
tree_sitter_baseandpython_parserbuild targets and wirespython_parserinto thelanguage_parsersaggregator so the dispatcher picks it up automatically.7.
chore(language_parsers): drop py_test target from python BUILD.bazel— @ishijoRemoves an unnecessary
py_testtarget.Note for maintainers
Lock files (
requirements_lock_3.10.txtthroughrequirements_lock_3.13.txt) need to be regenerated after merge:bazel run //requirements:requirements_3.10.update
bazel run //requirements:requirements_3.11.update
bazel run //requirements:requirements_3.12.update
bazel run //requirements:requirements_3.13.update
Validation
GHSA-xjgw-4wvw-rgm4(path traversal inmcp-atlassian) end-to-end on Python sourceAuthor attribution
Per the review guidance, each commit preserves its original author. Base commit is mine (@awe-srush); additional contributors @avshar15 and @ishijo are attributed on their own commits and will be picked up as
Co-authored-by:trailers on import.Supersedes