Skip to content

[Perf] Windows/x64: 15 Regressions on 5/2/2025 11:56:41 PM +00:00 #115340

New issue
New issue
Closed
Closed
[Perf] Windows/x64: 15 Regressions on 5/2/2025 11:56:41 PM +00:00#115340
Assignees
EgorBo
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
10.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on May 6, 2025

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
266.95 μs 340.52 μs 1.28 0.06 False
268.25 μs 341.05 μs 1.27 0.05 False
270.88 μs 341.77 μs 1.26 0.06 False
266.82 μs 340.93 μs 1.28 0.04 False
270.24 μs 340.93 μs 1.26 0.04 False
268.75 μs 341.08 μs 1.27 0.05 False
272.03 μs 340.91 μs 1.25 0.07 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
Details

PerfLabTests.CastingPerf.IntObj

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.IFooObjIsIFooInterAlia

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.ObjObjIsFoo

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.FooObjIsFoo

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.IFooObjIsIFoo

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.IFooObjIsDescendantOfIFoo

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf.FooObjIsDescendant

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.47 μs 15.18 μs 1.32 0.34 False
10.09 μs 12.95 μs 1.28 0.38 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
Details

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)

ETL Files

Histogram

JIT Disasms

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
28.31 ns 31.78 ns 1.12 0.04 False
6.83 ns 8.57 ns 1.26 0.20 False
16.58 ns 19.51 ns 1.18 0.03 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
Details

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 128)

ETL Files

Histogram

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes(Size: 128)

ETL Files

Histogram

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes_Chars(Size: 128)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
984.00 ns 1.04 μs 1.06 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
Details

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfStrings)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.28 ms 3.59 ms 1.10 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Details

PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22621
Queue TigerWindows
Baseline cb3f507bacf3d941c44458613dc4129ef5508265
Compare 4352c7a1d4a71d63cf80cfd8f7c19036bcf131a8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.38 μs 2.66 μs 1.12 0.17 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
Details

System.Collections.ContainsKeyTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-windows
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on May 6, 2025

  2. LoopedBard3 commented on May 6, 2025

    @LoopedBard3
    LoopedBard3
    on May 6, 2025
    Member

    Mix of regressions and noise, likely due to #114410 and related PR's in range: d7e5d95...4352c7a.

    Related Regressions:

  3. LoopedBard3
    added
    os-windows
    on May 6, 2025
  4. LoopedBard3
    transferred this issue fromdotnet/perf-autofiling-issueson May 6, 2025
  5. LoopedBard3
    added
    arch-x64
    untriagedNew issue has not been triaged by the area owner
    runtime-coreclrspecific to the CoreCLR runtime
    on May 6, 2025
  6. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on May 6, 2025
  7. LoopedBard3
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on May 6, 2025
  8. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on May 7, 2025

  9. dotnet-policy-service commented on May 7, 2025

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on May 7, 2025 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
    See info in area-owners.md if you want to be subscribed.

  10. JulieLeeMSFT
    assigned
    BruceForstall
    on May 7, 2025
  11. JulieLeeMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on May 7, 2025
  12. JulieLeeMSFT
    added this to the 10.0.0 milestone on May 7, 2025

  13. JulieLeeMSFT commented on May 7, 2025

    @JulieLeeMSFT
    JulieLeeMSFT
    on May 7, 2025
    Member

    CC @BruceForstall, @saucecontrol.

  14. saucecontrol commented on May 9, 2025

    @saucecontrol
    saucecontrol
    on May 9, 2025
    Member

    Yeah, most of these look like noise. The regressions in PerfLabTests.CastingPerf do repro for me locally. I don't see any asm diffs, but #115134 may be involved in the helper call chain.

    The System.Text.Perf_Ascii changes are from #114040, where 64-byte const loads were replaced with broadcasts. Ex: https://www.diffchecker.com/zMx57oAs/

    Perf numbers from uops.info show 64-byte broadcast to be the same cost as 64-byte load, but there could be something subtly different to how they're handled in hardware. The data size reduction probably makes it a worthwhile trade-off anyway, but we could tweak the rules for broadcast optimization if necessary.

  15. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on May 14, 2025

  16. JulieLeeMSFT commented on May 15, 2025

    @JulieLeeMSFT
    JulieLeeMSFT
    on May 15, 2025
    Member

    @am11, PTAL.

    I don't see any asm diffs, but #115134 may be involved in the helper call chain.

  17. JulieLeeMSFT
    assigned
    EgorBo
    and unassigned
    BruceForstall
    on May 15, 2025

  18. am11 commented on May 15, 2025

    @am11
    am11
    on May 15, 2025
    Member

    It is by design. cc @jkotas

  19. jkotas commented on May 15, 2025

    @jkotas
    jkotas
    on May 15, 2025
    Member

    The regressions in PerfLabTests.CastingPerf do repro for me locally. I don't see any asm diffs, but #115134 may be involved in the helper call chain.

    CastingPerf benchmarks do not exercise boxing. It looks like a noisy benchmark to me.

  20. EgorBo commented on May 30, 2025

    @EgorBo
    EgorBo
    on May 30, 2025
    Member

    Literally all of them are just noise/bi-modal behavior so I'm going to close this.

  21. EgorBo
    closed this as completedon May 30, 2025
  22. github-actions
    locked and limited conversation to collaborators on Jun 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions