[armanossiloko]

Contributes to #68325.

Summary

Array.Sort, Span<T>.Sort and List<T>.Sort of more than 2048 primitive integers with the default comparer now check how predictable the input is, and sort unpredictable input with a quicksort whose partition loop has no data-dependent branches. On random input the introsort partition loop mispredicts about half of its comparisons. The new loop's cost per element doesn't depend on how predictable the data is.

Sorting 1M random ints is 2.9× faster, sorted and reversed input is 27× faster, and inputs with few distinct values are 4–6× faster. Every other input, and every input of up to 2048 elements, is sorted by the existing introsort exactly as before. The change is a single file (ArraySortHelper.cs) with no public API changes.

Why this doesn't hit the concern raised on #68436

In #68436, jkotas pointed out that replacing the unstable sort algorithm is a semi-breaking change, because the relative order of equal elements changes and breaks test baselines. This PR only switches the algorithm when T is one of byte, sbyte, short, ushort, int, uint, long, ulong, nint or nuint, and only on the default-comparer path. For these types, elements that compare equal are bitwise identical. Every correct sort therefore produces exactly the same output, and the change is unobservable apart from speed.

These stay on the existing introsort:

• floating point (-0.0/+0.0 compare equal but differ in bits)
• char and all other types
• custom comparers
• keys/items sorts

When the new sort is used

A branchless sort isn't faster on every input. When the introsort's branches are predicted correctly, it does less work per element. That happens for nearly sorted input, and for small inputs that are sorted over and over again (as the Sort<Int32> microbenchmark does). So the new code routes each input to whichever sort is faster:

1. n ≤ 2048: always the existing introsort. Branch predictors can learn the complete branch history of re-sorting the same input of up to roughly a thousand elements, and then the introsort stops mispredicting. On the machine below the crossover is around 700 elements; 2048 leaves room for CPUs with larger predictors.
2. Larger inputs: 32 pairs of adjacent elements spread across the input are compared to a reference value (a median of three). If at least a quarter of the pairs straddle it, the introsort would mispredict often, so the input is sorted with the branchless quicksort. The sampling loop itself is written without data-dependent branches.
3. Otherwise, if no sampled pair (nor any of the last 8 pairs, where appended elements land) is out of order, a vectorized scan checks whether the input is already sorted, or strictly descending (which is reversed). This turns re-sorting a sorted array from O(n log n) into one pass.
4. Everything else is sorted with the existing introsort.

The threshold in step 2 was calibrated against inputs where the introsort wins: sorted input with random pairs swapped is the hardest case. The two sorts break even at about 12% straddling pairs; 25% leaves a 2× margin. A reference far from the median can only make an input look more predictable than it is, so errors fall on the side of the introsort.

Algorithm

This closely follows the design of Rust's slice::sort_unstable (ipnsort, by Lukas Bergdoll and Orson Peters), which descends from pdqsort:

• Branchless Lomuto partition. Each element is unconditionally swapped with the first element not yet known to belong on the left, and the left side then grows by (value < pivot) ? 1 : 0. The JIT compiles the loop to cmp/setl/add with no branches on the data.
• Recursive pseudo-median pivot selection (median of 3 below 64 elements). A fixed ninther gave consistently bad pivots on periodic input such as i % 1000, which fell back to heapsort. The recursive pseudo-median fixes that.
• Equal-elements partition. If the chosen pivot is not greater than the pivot of the parent partition, it's the minimum of the partition, so all elements equal to it are split off at once. Inputs with k distinct values become O(n log k).
• Worst case stays O(n log n) through the existing HeapSort, after 2 * (log2(n) + 1) levels, the same limit introsort uses. The existing InsertionSort handles partitions of ≤ 20 elements.

The sort is in place and allocates nothing. The only hardware intrinsics are the cross-platform Vector128/Vector256 APIs in the sorted-input scan, with a scalar fallback.

The C# is my own, written in the style of the surrounding file, but the design and the pivot-selection scheme follow ipnsort (MIT/Apache-2.0). Please let me know if you'd like a THIRD-PARTY-NOTICES entry for that; per CONTRIBUTING.md I haven't touched licensing files.

Performance

Windows 10 x64, Intel Core i9-9900K. main at 72509c6 vs this PR, both Release builds, run through the testhost. Times are ns per element (lower is better), best of 9 after a warmup. Each measurement cycles through up to 2048 distinct inputs of the given shape, so the branch predictor can't memorize a single input. Rows marked † were re-run 5 times interleaved between the two builds; the median is shown.

Inputs that take the new paths (1M elements unless noted):

type	n	input	main	PR	speedup
int	4,096	random	33.82	17.51	1.93×
int	10,000	random	37.18	18.00	2.07×
int	100,000	random	46.33	19.74	2.35×
int	1,000,000	random	55.72	19.32	2.88×
long / uint / short / byte	1,000,000	random	53.49 / 53.22 / 51.79 / 32.07	20.34 / 24.12 / 17.62 / 8.88	2.63× / 2.21× / 2.94× / 3.61×
int	1,000,000	sorted	6.84	0.25	27×
int	1,000,000	reversed	13.54	0.48	28×
int	1,000,000	sorted, runs of 8 equal values	8.44	0.25	34×
int	1,000,000	random % 100	27.30	4.92	5.6×
int	1,000,000	random % 4	15.36	3.94	3.9×
int	1,000,000	4 / 3 interleaved ascending sequences	34.39 / 29.40	16.87 / 16.30	2.0× / 1.8×
int	1,000,000	alternating 0, 1 / zigzag low-high	9.03 / 20.25	2.03 / 15.18	4.5× / 1.3×
int	1,000,000	sorted, 50% of values replaced by random ones	46.69	22.84	2.0×
int	1,000,000	sorted, 25% of elements randomly swapped	34.70	19.19	1.8×
double (control, unchanged path)	1,000,000	random	59.89	59.30	≈

Inputs that stay on the introsort (unchanged apart from the routing check):

n	input	main	PR	ratio
10 / 100 / 1,000	random	6.92 / 18.35 / 27.80	7.04 / 18.58 / 28.15	≈ (noise)
1,000,000	sorted, 0.1% / 1% / 5% / 7% of elements randomly swapped †	5.48 / 7.49 / 14.85 / 17.74	5.50 / 7.48 / 14.82 / 17.51	1.00 / 1.00 / 1.00 / 1.01
1,000,000	sorted + 1 appended element †	8.10	8.09	1.00
1,000,000	sorted + random tail of 1/32 / 1/1024	19.14 / 13.92	19.55 / 13.95	0.98 / 1.00
1,000,000	sorted, 1% / 12% of values replaced by random ones	16.07 / 27.48	16.46 / 27.58	0.98 / 1.00
1,000,000	sawtooth (i % 1000) † / organ pipe † / two interleaved runs †	25.47 / 72.69 / 69.34	25.66 / 70.39 / 70.64	0.99 / 1.03 / 0.98
4,096	sorted, 0.1% / 5% / 7% of elements randomly swapped †	3.21 / 9.10 / 10.87	3.28 / 8.94 / 10.93	0.98 / 1.02 / 0.99

The routing check costs about 60 ns per sort (32 sampled pairs, a median of three and 8 trailing pairs), and only for more than 2048 elements. On the most introsort-friendly input just above that size this is about 0.5%, and less than 0.2% from 16K elements up. That's below what these measurements resolve.

Re-sorting an identical small input, as the dotnet/performance benchmark System.Collections.Sort<Int32>.Array/.List (Size = 512) does, is unchanged, since inputs of up to 2048 elements never leave the introsort. Reproducing that benchmark's setup (one fixed Random(12345) input, 5000 fresh copies per iteration):

	main	PR
Sort<Int32>.Array-style, n = 512	3,013 ns	3,014 ns
Sort<Int32>.List-style, n = 512	3,058 ns	3,015 ns

A note for anyone re-measuring: when the same input of roughly 700–2000 elements is re-sorted, the timing depends on how much of the branch history the predictor memorizes. In short runs that still execute ReadyToRun code, this varies by ±10–20% between builds whose executed code is identical: two builds of this branch that differ only in code reached above 2048 elements measured −12% to +18% apart at n = 768–896. With DOTNET_ReadyToRun=0 (or after tier-up) these sizes measure the same on main and this PR.

Validation

• The System.Memory.Tests (52,935), System.Runtime.Tests (78,658), System.Collections.Tests (33,958), System.Linq.Tests (52,260) and System.Collections.Immutable.Tests (22,093 + 22,621) suites all pass locally against this build. No tests were changed.
• A differential fuzzer compared Array.Sort(T[]) (new paths) against Array.Sort(T[], IComparer<T>) with a non-default comparer, which still runs introsort. It ran 70,000 cases across all ten integer types and the Array.Sort(T[], int, int), Span<T>.Sort() and List<T>.Sort() entry points. Inputs covered random values with 2 to 2⁶⁴ distinct values; ascending, descending and both with duplicates; a single out-of-order element at a random position or near the end (covering the vector scan's tail); sawtooth; organ pipe; sorted with a random tail or with 3% noise; and MinValue/MaxValue extremes. Sizes ranged over 0–70,000, with extra weight on 2040–2120 around the threshold. The output was identical in every case.
• The JIT output for the partition loop and the sampling loop (x64 Tier 1) has no branches on the data and no bounds checks.

I have only measured x64. I'd appreciate a run on Arm64, where the branchless loop should benefit similarly.

Benchmark program (Array.Sort with distinct inputs)

static ulong S;
static ulong Next() { S ^= S >> 12; S ^= S << 25; S ^= S >> 27; return S * 0x2545F4914F6CDD1DUL; }

static void Run<T>(Func<long, T> conv, int n, Func<int, int, long> gen)
{
    int distinct = Math.Max(1, Math.Min(2048, 8_000_000 / n));
    var srcs = new T[distinct][];
    for (int d = 0; d < distinct; d++) { srcs[d] = new T[n]; for (int i = 0; i < n; i++) srcs[d][i] = conv(gen(i, n)); }
    var work = new T[n];
    int inner = Math.Max(distinct, 2_000_000 / n);
    double Measure(bool sort)
    {
        var best = double.MaxValue;
        for (int r = 0; r < 9; r++)
        {
            var sw = Stopwatch.StartNew();
            for (int k = 0; k < inner; k++) { srcs[k % distinct].CopyTo(work, 0); if (sort) Array.Sort(work); }
            best = Math.Min(best, sw.Elapsed.TotalMilliseconds);
        }
        return best;
    }
    Measure(true); // warm up / tier up
    double t = Measure(true) - Measure(false);   // subtract the copy cost
    Console.WriteLine($"{typeof(T).Name} {n} {t * 1e6 / inner / n:F2} ns/elem");
}

🤖 Generated with Claude Code

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

[armanossiloko]

@dotnet-policy-service agree

[tannergooding]

We're trying to remove/reduce unsafe code, not increase it. Any new unsafe code needs significant justification and extra explicit checks for why it is correct.

[tannergooding]

There's quite a bit of repetitive/redundant work on these that wouldn't exist using bools and/or indices

[tannergooding]

This is reducing iterations as you become more sorted, but it still requires doing a lot of redundant memory accesses. It feels like tracking the known sorted runs could help mitigate some of that

[tannergooding]

You're currently doing: true: >= and false: <, which can also just be false: !(>=) or true: !(<) for integer types

So this could just be LessThanAny ^ strictlyDescending (because false ^ LessThanAny preserves it, while true ^ LessThanAny inverts it), avoiding the extra branch and work.

[tannergooding]

This is a lot of extra code, including unsafe code; and while the perf is there, it still seems to be doing a lot of work without strictly moving the needle on the overall premise and problem that people are wanting solved.

[armanossiloko]

Thanks for taking the time to review this, and for the detailed comments. That's fair so I'll close the PR instead of iterating on it.