Skip to content

[Perf] Linux/x64: 1 Regression on 7/2/2025 9:07:52 AM +00:00 #117424

New issue
New issue
Closed
Closed
[Perf] Linux/x64: 1 Regression on 7/2/2025 9:07:52 AM +00:00#117424
Assignees
huoyaoyuan
Labels
arch-x64area-NativeAOT-coreclros-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Description

@performanceautofiler
performanceautofiler
bot
opened on Jul 8, 2025

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline a3929e2ba319b5899d39bfd39c0314395739408b
Compare 877b0cdea5bfb1a7ca99934053421a7a8262d8fe
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
159.98 μs 182.79 μs 1.14 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'
Details

Benchmark.GetChildKeysTests.AddChainedConfigurationWithCommonPaths

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-linuxLinux OS (any supported distro)
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Jul 8, 2025
  2. DrewScoggins
    transferred this issue fromdotnet/perf-autofiling-issueson Jul 8, 2025
  3. DrewScoggins
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Jul 8, 2025
  4. github-actions
    added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Jul 8, 2025

  5. DrewScoggins commented on Jul 8, 2025

    @DrewScoggins
    DrewScoggins
    on Jul 8, 2025
    Member

    Seems like it might be related to #116903

    Related Regressions:

  6. jeffschwMSFT
    added
    area-NativeAOT-coreclr
    on Jul 9, 2025
  7. github-project-automation
    added this to AppModelon Jul 9, 2025

  8. dotnet-policy-service commented on Jul 9, 2025

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Jul 9, 2025 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
    See info in area-owners.md if you want to be subscribed.

  9. jkotas commented on Jul 11, 2025

    @jkotas
    jkotas
    on Jul 11, 2025
    Member

    Seems like it might be related to #116903

    @huoyaoyuan Could you please take a look?

  10. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Jul 11, 2025

  11. huoyaoyuan commented on Jul 11, 2025

    @huoyaoyuan
    huoyaoyuan
    on Jul 11, 2025
    Member

    The only related call path is List.AddRange -> SZArrayHelper.CopyTo -> Array.CopyTo (fast path). #116903 was focusing on slow path. I'll run more benchmarks tomorrow.

  12. jkotas
    assigned
    huoyaoyuan
    on Jul 13, 2025

  13. huoyaoyuan commented on Jul 13, 2025

    @huoyaoyuan
    huoyaoyuan
    on Jul 13, 2025
    Member

    Basic test results:

    Coreclr:

    Method Job OutlierMode Toolchain MemoryRandomization Mean Error StdDev Ratio RatioSD
    AddChainedConfigurationNoDelimiter Job-VZEHHD Default \After\10.0.0\corerun.exe Default 231.7 us 2.39 us 2.12 us 1.00 0.01
    AddChainedConfigurationNoDelimiter Job-BJTKDS Default \Before\10.0.0\corerun.exe Default 232.5 us 2.21 us 1.96 us 1.00 0.01
    AddChainedConfigurationWithSplitting Job-VZEHHD Default \After\10.0.0\corerun.exe Default 106.7 us 0.39 us 0.36 us 0.94 0.01
    AddChainedConfigurationWithSplitting Job-BJTKDS Default \Before\10.0.0\corerun.exe Default 113.0 us 0.79 us 0.70 us 1.00 0.01
    AddChainedConfigurationEmpty Job-SBWNAD DontRemove \After\10.0.0\corerun.exe True 6,233.3 us 85.33 us 79.82 us 1.00 0.02
    AddChainedConfigurationEmpty Job-XARMTJ DontRemove \Before\10.0.0\corerun.exe True 6,239.1 us 105.13 us 98.33 us 1.00 0.02
    AddChainedConfigurationWithCommonPaths Job-SBWNAD DontRemove \After\10.0.0\corerun.exe True 110.2 us 0.52 us 0.48 us 0.91 0.01
    AddChainedConfigurationWithCommonPaths Job-XARMTJ DontRemove \Before\10.0.0\corerun.exe True 121.3 us 1.86 us 1.74 us 1.00 0.02

    NativeAOT before:

    Method Job OutlierMode MemoryRandomization Mean Error StdDev
    AddChainedConfigurationNoDelimiter Job-FUOXQK Default Default 399.7 us 0.63 us 0.49 us
    AddChainedConfigurationWithSplitting Job-FUOXQK Default Default 195.9 us 1.05 us 0.88 us
    AddChainedConfigurationEmpty Job-HUBBVH DontRemove True 6,946.7 us 66.35 us 62.06 us
    AddChainedConfigurationWithCommonPaths Job-HUBBVH DontRemove True 201.6 us 2.93 us 2.74 us

    NativeAOT after:

    Method Job OutlierMode MemoryRandomization Mean Error StdDev
    AddChainedConfigurationNoDelimiter Job-FURRXE Default Default 417.7 us 3.78 us 3.54 us
    AddChainedConfigurationWithSplitting Job-FURRXE Default Default 196.7 us 1.59 us 1.41 us
    AddChainedConfigurationEmpty Job-FTJYYJ DontRemove True 6,976.2 us 72.55 us 67.86 us
    AddChainedConfigurationWithCommonPaths Job-FTJYYJ DontRemove True 199.5 us 2.01 us 1.88 us

    There is a regression in AddChainedConfigurationNoDelimiter, but local difference is smaller than 5%. There's no local regression in AddChainedConfigurationWithCommonPaths.

  14. jkotas commented on Jul 13, 2025

    @jkotas
    jkotas
    on Jul 13, 2025
    Member

    There is a regression in AddChainedConfigurationNoDelimiter

    Does this scenario call Array.Copy? If yes, what are the arguments passed to it?

  15. huoyaoyuan commented on Jul 13, 2025

    @huoyaoyuan
    huoyaoyuan
    on Jul 13, 2025
    Member

    There is a regression in AddChainedConfigurationNoDelimiter

    Does this scenario call Array.Copy? If yes, what are the arguments passed to it?

    The only call stack I can see is via List<T>.AddRange, which is always invoked with Array.Empty<string>.

  16. jkotas commented on Jul 13, 2025

    @jkotas
    jkotas
    on Jul 13, 2025
    Member

    List.AddRange

    Is T == string or something else?

  17. huoyaoyuan commented on Jul 14, 2025

    @huoyaoyuan
    huoyaoyuan
    on Jul 14, 2025
    Member

    Is T == string or something else?

    T is always string. However, when the added collection is empty, CopyTo shouldn't be touched. I can't find Array.Copy in profiling trace either.

  18. jkotas commented on Jul 14, 2025

    @jkotas
    jkotas
    on Jul 14, 2025
    Member

    Micro-benchmark noise, no local repro (some local runs even show an improvement - e.g. see #117424 (comment)).

    It certainly does not look related to #116903 since this microbenchmark does not exercise the code changed by this PR.

  19. jkotas
    closed this as completedon Jul 14, 2025
  20. dotnet-policy-service
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jul 14, 2025

  21. DrewScoggins commented on Jul 14, 2025

    @DrewScoggins
    DrewScoggins
    on Jul 14, 2025
    Member

    Also, looking at the test history, we see that the value has returned to its previous value.

  22. github-actions
    locked and limited conversation to collaborators on Aug 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-NativeAOT-coreclros-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

  • AppModel
    Status
    No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions