Repository navigation
ARM64 SVE: GatherVectorWithByteOffsetFirstFaulting failures during stress testing #106621
Description
With today's HEAD (dc0432b)
When running stress tetsing, I sometimes get errors with GatherVectorWithByteOffsetFirstFaulting. Seems to be inconsistent when it occurs.
Using HardwareIntrinsics_Arm_ro.dll
===================Running jitstress===================
------------------- {'JitMinOpts': '1'} -------------------
------------------- {'JitStress': '1'} -------------------
------------------- {'JitStress': '2'} -------------------
------------------- {'JitStress': '1', 'TieredCompilation': '1'} -------------------
------------------- {'JitStress': '2', 'TieredCompilation': '1'} -------------------
Test failed:
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
Sve.GatherVectorWithByteOffsetFirstFaulting<Single>(Single, Single, Int32): RunBasicScenario_LoadFirstFaulting failed:
firstOp: (1E-45, 0, 1E-45, 1)
secondOp: (0.6533055)
thirdOp: (0, 237, 3, 12)
result: (0.6533055, 0, 1.5155E-41, 0)
faultResult: (<1, 1, 0, 0>)
..........................................
System.Exception: One or more scenarios did not complete as expected.
at JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsetFirstFaulting_float_int() in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/Arm/Sve/Sve_ro/Sve_ro/gen/Sve.GatherVectorWithByteOffsetFirstFaulting.float.int.cs:line 86
at Program.<<Main>$>g__TestExecutor3308|0_3309(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&) in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/generated/XUnitWrapperGenerator/XUnitWrapperGenerator.XUnitWrapperGenerator/FullRunner.g.cs:line 83607
------------------- {'TailcallStress': '1'} -------------------
------------------- {'ReadyToRun': '0'} -------------------
===================Running jitstressregs===================
------------------- {'JitStressRegs': '1'} -------------------
------------------- {'JitStressRegs': '2'} -------------------
------------------- {'JitStressRegs': '3'} -------------------
------------------- {'JitStressRegs': '4'} -------------------
------------------- {'JitStressRegs': '8'} -------------------
------------------- {'JitStressRegs': '0x10'} -------------------
------------------- {'JitStressRegs': '0x80'} -------------------
Test failed:
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
Sve.GatherVectorWithByteOffsetFirstFaulting<Int32>(Int32, Int32, Int32): RunBasicScenario_LoadFirstFaulting failed:
firstOp: (1, 0, 1, 1)
secondOp: (1532479874)
thirdOp: (0, 155, 3, 12)
result: (1532479874, 0, 9496155, 0)
faultResult: (<1, 1, 0, 0>)
..........................................
System.Exception: One or more scenarios did not complete as expected.
at JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsetFirstFaulting_int() in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/Arm/Sve/Sve_ro/Sve_ro/gen/Sve.GatherVectorWithByteOffsetFirstFaulting.int.cs:line 86
at Program.<<Main>$>g__TestExecutor3309|0_3310(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&) in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/generated/XUnitWrapperGenerator/XUnitWrapperGenerator.XUnitWrapperGenerator/FullRunner.g.cs:line 83631
------------------- {'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStressRegs': '0x2000'} -------------------
===================Running jitstress2-jitstressregs===================
------------------- {'JitStress': '2', 'JitStressRegs': '1'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '2'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '3'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '4'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '8'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x10'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x80'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x2000'} -------------------
On another run:
===================Running jitstress2-jitstressregs===================
------------------- {'JitStress': '2', 'JitStressRegs': '1'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '2'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '3'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '4'} -------------------
Test failed:
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
..........................................
Sve.GatherVectorUInt32WithByteOffsetsZeroExtendFirstFaulting<Int32>(Int32, UInt32, Int32): RunBasicScenario_LoadFirstFaulting failed:
firstOp: (0, 1, 1, 1)
secondOp: (483312, 111609)
thirdOp: (172, 0, 7, 12)
result: (0, 483312, 0, 0)
faultResult: (<1, 1, 0, 0>)
..........................................
System.Exception: One or more scenarios did not complete as expected.
at JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtendFirstFaulting_offsets_int_int() in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/Arm/Sve/Sve_ro/Sve_ro/gen/Sve.GatherVectorUInt32WithByteOffsetsZeroExtendFirstFaulting.offsets.int.int.cs:line 76
at Program.<<Main>$>g__TestExecutor3284|0_3285(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&) in /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/generated/XUnitWrapperGenerator/XUnitWrapperGenerator.XUnitWrapperGenerator/FullRunner.g.cs:line 83031
Activity
dotnet-policy-service commentedon Aug 19, 2024
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.
SwapnilGaikwad commentedon Aug 22, 2024
Hi @amanasifkhalid, do you still see this issue. I couldn't reproduce this.
However, I saw another failure instead and that occurs rarely, 1 out of 10 times. I'll open an issue for it. Although, not sure how to reproduce that.
SwapnilGaikwad commentedon Aug 22, 2024
Opened #106815.
amanasifkhalid commentedon Aug 22, 2024
I was able to repro it after about a dozen stress test runs, though I haven't tried reproing with the latest HEAD. I'll take another look...
Since the intrinsic failing in #106815 uses the same template as this intrinsic, I suspect these failures might be bugs in the template itself (which is better than it being a JIT bug). Something interesting to note is both of the failures Alan posted above were with variants operating on Int32/UInt32, and the failure you posted on #106815 is also working with 32-bit integers as the base type. It's possible that the test template isn't handling the bounded memory correctly, and this size of base type happens to hit the bug. Though if you manage to repro this failure on another base type, that would be a good challenge to my hypothesis.
amanasifkhalid commentedon Aug 29, 2024
I've dug into this a bit, and this seems to be an issue with the test validation logic, particularly the bit in
CheckGatherVectorWithByteOffsetFirstFaultingBehaviorfor calculating the expected FFR value. Right now, we determine that the load should have faulted if a given offset exceeds the length of the data, but we do not account for the fact that the load can walk off the end of the data array even if the starting offset is in-range. For example, if the data array is a single 32-bit integer, and the starting offset is 1, we will read bytes 1-4, which means the last byte read should cause a fault, but the current logic will expect it to not fault. The fix is straightforward: We just need to factor in the size of the load when determining if it will exceed the bounds of the data array.