Skip to content

#4916 - Widen INT32 comparisons before subtraction - #4919

Open
0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:codex/polygraphy-int32-comparison
Open

0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:codex/polygraphy-int32-comparison

Conversation

@0z5a

@0z5a 0z5a commented Nov 4, 2026

Copy link
Copy Markdown

Fixes #4916.

Widen INT32 outputs to INT64 before computing differences so the simple comparator does not wrap and falsely pass at zero tolerance. For PyTorch INT64 arrays, compute the absolute value directly rather than round-tripping through FLOAT32, which also loses low integer bits.

Testing: the new INT32 boundary regressions fail on the original implementation. On CPU, 132 tests passed and 4 were skipped across the existing simple/L2 comparator tests and the new NumPy/PyTorch tests. Runtime warnings from division by zero and non-finite statistics were present in the test run. Wider INT64 subtraction overflow is outside this change's scope. No GPU or TensorRT sample tests were run.

This draft is provided for the issue discussion; it can be marked ready after the issue has been reviewed as required by the contribution guidelines.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@0z5a
0z5a marked this pull request as ready for review November 5, 2026 21:58
@0z5a
0z5a requested a review from a team as a code owner November 5, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Polygraphy INT32 comparison can overflow and falsely pass at zero tolerance

2 participants