Skip to content

fix(collections)!: correct list ownership and implement LLVM lowering - #3307

Draft
doug-q wants to merge 3 commits into
mainfrom
doug/fix-hugr-lists
Draft

doug-q wants to merge 3 commits into
mainfrom
doug/fix-hugr-lists

Conversation

@doug-q

@doug-q doug-q commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

List operations could lose owned handles, copy linear elements, and disagree across their declared signatures, constant folding, and LLVM results. Make lists always linear, return the list alongside get's optional copied element, and require copyable elements for get. Add new and discard_empty; empty disposal accepts arbitrary element types and panics if any elements remain. Allow insertion at the length and use an empty success row, while failed mutations preserve the list and return the input element.

Implement LLVM list operations with an owned byte buffer and checked capacity growth. All handle and buffer allocations and matching frees use customizable codegen hooks, including constant construction and allocate-copy-free growth. Nonempty disposal and allocation/capacity failures use the selected PreludeCodegen. Reconcile the Rust and Python types, generated declarations, specification, and constant folding, and encode Python list constants with the Rust-compatible tuple payload.

BREAKING CHANGE: collections.list is now version 0.2.0. List<T> is always linear and get returns (List<T>, Option<T>) for copyable elements. Thread returned handles through list operations and consume emptied handles with discard_empty. Successful insert carries an empty row.

List operations could lose owned handles, copy linear elements, and disagree across their declared signatures, constant folding, and LLVM results. Make lists always linear, return the list alongside `get`'s optional copied element, and require copyable elements for `get`. Add `new` and `discard_empty`; empty disposal accepts arbitrary element types and panics if any elements remain. Allow insertion at the length and use an empty success row, while failed mutations preserve the list and return the input element.

Implement LLVM list operations with an owned byte buffer and checked capacity growth. All handle and buffer allocations and matching frees use customizable codegen hooks, including constant construction and allocate-copy-free growth. Nonempty disposal and allocation/capacity failures use the selected `PreludeCodegen`. Reconcile the Rust and Python types, generated declarations, specification, and constant folding, and encode Python list constants with the Rust-compatible tuple payload.

BREAKING CHANGE: `collections.list` is now version `0.2.0`. `List<T>` is always linear and `get` returns `(List<T>, Option<T>)` for copyable elements. Thread returned handles through list operations and consume emptied handles with `discard_empty`. Successful `insert` carries an empty row.
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.92229% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.33%. Comparing base (0202557) to head (ff2c8b2).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
hugr-llvm/src/extension/collections/list.rs 90.97% 11 Missing and 61 partials ⚠️
...e/src/std_extensions/collections/list/list_fold.rs 84.00% 1 Missing and 3 partials ⚠️
hugr-llvm/src/emit/func.rs 90.00% 0 Missing and 2 partials ⚠️
hugr-core/src/std_extensions/collections/list.rs 99.24% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3307      +/-   ##
==========================================
+ Coverage   82.05%   82.33%   +0.28%     
==========================================
  Files         245      245              
  Lines       47553    48509     +956     
  Branches    41144    42088     +944     
==========================================
+ Hits        39020    39941     +921     
+ Misses       6629     6605      -24     
- Partials     1904     1963      +59     
Flag Coverage Δ
python 89.92% <100.00%> (+0.39%) ⬆️
rust 81.17% <91.90%> (+0.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Inkwell 0.9 requires copied memory-buffer input to end with a NUL byte. Terminate the embedded list IR before parsing it in both codegen and the direct runtime exercise, preserving compatibility with both supported Inkwell versions.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Constant folding can create invalid polymorphic constants, while LLVM ABI assumptions break supported custom codegen configurations.

3 open findings
What changed in this PR

Updates linear list ownership semantics and adds an owned-buffer LLVM implementation.

Changes:

  • Makes lists linear and reconciles operation signatures, folding, serialization, and documentation.
  • Adds new, discard_empty, append insertion, and corrected mutation results.
  • Implements LLVM list allocation, growth, disposal, panic hooks, and execution tests.
File Description
specification/​stdlib.md Documents revised list semantics.
resources/​std_extensions/​collections/​list.json Updates generated extension schema.
hugr-py/​tests/​test_val.py Tests list constant serialization.
hugr-py/​tests/​test_tys.py Tests list bounds and signatures.
hugr-py/​src/​hugr/​std/​collections/​list.py Makes lists linear and fixes serialization.
hugr-py/​src/​hugr/​std/​_json_defs/​collections/​list.json Updates Python extension definitions.
hugr-llvm/​src/​extension/​collections/​list.rs Implements LLVM lowering and tests.
hugr-llvm/​src/​extension/​collections/​list_runtime.ll Adds the byte-buffer runtime.
hugr-llvm/​src/​emit/​func.rs Extracts function-definition support.
hugr-core/​src/​std_extensions/​collections/​list/​list_fold.rs Reconciles constant folding.
hugr-core/​src/​std_extensions/​collections/​list.rs Revises core list API and types.
hugr-core/​src/​extension/​resolution/​test.rs Consumes linear constants in tests.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__set@pre-mem2reg@llvm21.snap Updates set lowering snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__set@llvm21.snap Updates optimized set snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__pop@pre-mem2reg@llvm21.snap Updates pop lowering snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__pop@llvm21.snap Updates optimized pop snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__new@pre-mem2reg@llvm21.snap Adds new-list snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__new@llvm21.snap Adds optimized new-list snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__get@pre-mem2reg@llvm21.snap Updates get lowering snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__get@llvm21.snap Updates optimized get snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__discard_empty@pre-mem2reg@llvm21.snap Adds disposal snapshot.
hugr-llvm/​src/​extension/​collections/​snapshots/​hugr_llvm__extension__collections__list__test__discard_empty@llvm21.snap Adds optimized disposal snapshot.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +35 to +38
let [ty] = args else { return None };
if !inputs.is_empty() {
return None;
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ff2c8b2. The new-list folder validates the element type with an empty variable scope before constructing a constant. A regression covers free variables directly and inside tuple/list types for both bounds, while concrete copyable and linear element types still fold. The regression failed before the fix; the full core suite now passes.

&[
iwc.i64_type().const_zero().into(),
elem_ty.size_of().unwrap().into(),
iwc.i64_type().const_int(1, false).into(),

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ff2c8b2. Both dynamic new and constant construction now pass elem_ty.get_alignment(), preserving the constructor ABI while supplying target-derived alignment. An external-runtime execution regression observes this argument for usize, qubit, and unit elements in both construction paths; all six cases pass. The embedded byte-copy runtime remains independent of buffer element alignment.

Comment on lines +118 to +120
define i1 @__rt__list__get(ptr %list, i64 %idx, ptr %out) {
entry:
%len = call i64 @__rt__list__length(ptr %list)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ff2c8b2. Operation boundaries now convert the registered prelude usize to/from the fixed i64 runtime ABI. Wider indices remain out of bounds instead of wrapping, and unrepresentable narrow length results panic through the selected PreludeCodegen. Execution regressions cover the operation matrix and constant lists with i32/i64/i128 preludes, indices above 64 bits, and the custom panic for length overflow. The full LLVM suite passes on both current dependencies and Rust 1.91/minimum dependencies.

Skip empty-list constant folding when the element type contains free variables, since constants have no enclosing type-variable scope. Pass the element's target alignment consistently for dynamic and constant constructors.

Convert configured prelude indices and lengths at the fixed-width list runtime boundary. Preserve out-of-bounds wide indices and panic through the selected PreludeCodegen when a length cannot be represented by the configured usize type.
@croyzor
croyzor self-requested a review October 15, 2026 08:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants