Skip to content

Use the public BF16 odd-lane conversion API on Ascend - #236

Open
huangxinV587 wants to merge 1 commit into
deepseek-ai:mainfrom
huangxinV587:fix/ascend-public-bf16-conversion
Open

huangxinV587 wants to merge 1 commit into
deepseek-ai:mainfrom
huangxinV587:fix/ascend-public-bf16-conversion

Conversation

@huangxinV587

@huangxinV587 huangxinV587 commented Oct 21, 2026 •

Copy link
Copy Markdown

The Ascend sparse prefill kernel calls asc_float2bfloat16_rn_v2_impl, which is not declared by the CANN 9.2 public headers. Replace both call sites with the declared asc_float2bfloat16_rn_v2 API, retaining the paired conversion layout.

Add a compiler regression that compiles the conversion names used by the actual kernel header, plus eight NPU numerical cases covering paired lanes, positive/negative round-to-nearest-even ties, query tails and variable top-k lengths against an independent CPU reference.

NPU correctness validation completed for exact PR head 8483019afca46e81895a1f859a284d47316f4f99 on 2026-10-21.

  • tests/test_ascend_sparse_bf16.py: 8 passed, 0 failed, 0 skipped, executing on an Ascend NPU (dav-3510, aarch64).
  • Coverage: paired BF16 lanes and positive/negative round-to-nearest-even ties at top-k 128/256; independent CPU-reference comparisons for query lengths 1/17/65 with fixed and variable top-k. Test shapes and tolerances were unchanged.
  • Environment: CANN 9.2.0/Bisheng, GCC 13.2.0, Python 3.12.13, PyTorch 2.10.0+cpu with torch_npu 2.10.0.post7, pytest 8.4.2.

Build qualification: the stock toolchain first failed in GCC 13's std::pair declarations, independently of the BF16 conversion calls. To build the native extension, a toolchain-only wrapper pre-included the following header for .asc compilation; the macro is restored afterward. This overlay was not added to the PR, and the tested PR source bytes were unchanged.

#pragma push_macro("__cpp_concepts")
#undef __cpp_concepts
#define __cpp_concepts 0
#include <utility>
#pragma pop_macro("__cpp_concepts")

The wrapper adds -include /path/to/shim.h to Bisheng's .asc invocations. With that prerequisite, FLASH_MLA_BUILD_TARGET_PLATFORM=ASCEND python3 setup.py build_ext --inplace completed, followed by:

python3 -m pytest --confcutdir=tests -vv -rA tests/test_ascend_sparse_bf16.py

These results validate the eight correctness cases under the stated toolchain overlay; they do not establish stock-toolchain compatibility, performance, CUDA behavior, or the full upstream suite.

@huangxinV587

Copy link
Copy Markdown
Author

NPU correctness validation completed for exact PR head 8483019afca46e81895a1f859a284d47316f4f99 on 2026-10-21.

  • tests/test_ascend_sparse_bf16.py: 8 passed, 0 failed, 0 skipped, executing on an Ascend NPU (dav-3510, aarch64).
  • Coverage: paired BF16 lanes and positive/negative round-to-nearest-even ties at top-k 128/256; independent CPU-reference comparisons for query lengths 1/17/65 with fixed and variable top-k. Test shapes and tolerances were unchanged.
  • Environment: CANN 9.2.0/Bisheng, GCC 13.2.0, Python 3.12.13, PyTorch 2.10.0+cpu with torch_npu 2.10.0.post7, pytest 8.4.2.

Build qualification: the stock toolchain first failed in GCC 13's std::pair declarations, independently of the BF16 conversion calls. To build the native extension, a toolchain-only wrapper pre-included the following header for .asc compilation; the macro is restored afterward. This overlay was not added to the PR, and the tested PR source bytes were unchanged.

#pragma push_macro("__cpp_concepts")
#undef __cpp_concepts
#define __cpp_concepts 0
#include <utility>
#pragma pop_macro("__cpp_concepts")

The wrapper adds -include /path/to/shim.h to Bisheng's .asc invocations. With that prerequisite, FLASH_MLA_BUILD_TARGET_PLATFORM=ASCEND python3 setup.py build_ext --inplace completed, followed by:

python3 -m pytest --confcutdir=tests -vv -rA tests/test_ascend_sparse_bf16.py

These results validate the eight correctness cases under the stated toolchain overlay; they do not establish stock-toolchain compatibility, performance, CUDA behavior, or the full upstream suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant