[6.x] Speed up Stache entry queries by id and by value

[porstendorfer]

Every where on a Stache entry query copies the whole index of each queried collection, with every key prefixed by the collection handle, and then filters that copy. On large sites this means common lookups go through every entry: Entry::find(), the origin lookup every localization does in syncOriginal() (which dominates stache:warm on multisite sites), and the URL lookup of every frontend request.

This PR changes EntryQueryBuilder::getKeysFromCollectionsWithWhere(). A where('id', $id) is now answered from the stores' path indexes, which are keyed by id. If no store has the id, the existing filter runs, because ids are compared case-insensitively. Basic and In wheres filter each store's index first and only prefix the keys that match. The collection list is also deduplicated, since a status query can list a collection twice.

Measured with the script below on 4 sites with 10,000 entries plus 30,000 localizations (medians):

	Before	After
Entry::find() of a localization	20.5 ms	0.29 ms
Entry::find() of a missing id	10.4 ms	5.8 ms
Data::findByRequestUrl()	31.5 ms	13.2 ms
$entry->descendants()	48 ms	10 ms
stache:clear + stache:warm	384 s	71 s

Memory use stays the same.

Behaviour

The path index and the id index are cached separately (see #15052), so the path index can know an id that the id index doesn't have yet, for example while the id index is being built on a cold Stache. In that case find() now returns the entry instead of null. The fast path reads the path index on purpose: the id index can't answer while it's being built, and ColdStacheUriTest fails if the fast path uses it. A cold stache:warm of the benchmark site builds identical indexes before and after the change.

Ids with capital letters don't get faster through find(), because the two-argument where('id', $id) lowercases the value before it reaches the query builder. They're still found through the existing filter.

Tests

The new tests cover finding entries by id across collections (also combined with other wheres and with orWhere), case-insensitive ids, and a collection that's queried twice. They pass on 6.x too, so they pin the current behaviour rather than the speed-up.

Reproduce

Run this in a fresh app, once on this branch and once on 6.x, with STATAMIC_STACHE_WATCHER=false. generate overwrites resources/sites.yaml and content/collections/blog*.

php bench.php generate 10000
php please stache:clear && time php -d memory_limit=-1 please stache:warm
php -d memory_limit=-1 bench.php run

bench.php

<?php

use Illuminate\Contracts\Console\Kernel;
use Statamic\Facades\Blink;
use Statamic\Facades\Data;
use Statamic\Facades\Entry;
use Statamic\Facades\Site;
use Symfony\Component\Yaml\Yaml;

require getcwd().'/vendor/autoload.php';
$app = require getcwd().'/bootstrap/app.php';
$app->make(Kernel::class)->bootstrap();

$sites = ['en' => '/', 'fr' => '/fr/', 'de' => '/de/', 'es' => '/es/'];
$id = function ($site, $i) {
    $hash = md5("$site-$i");

    return substr($hash, 0, 8).'-'.substr($hash, 8, 4).'-'.substr($hash, 12, 4).'-'.substr($hash, 16, 4).'-'.substr($hash, 20, 12);
};
$command = $argv[1] ?? 'run';

if ($command === 'generate') {
    $n = (int) ($argv[2] ?? 10000);

    file_put_contents(resource_path('sites.yaml'), Yaml::dump(collect($sites)->map(fn ($url, $handle) => [
        'name' => strtoupper($handle), 'url' => $url, 'locale' => $handle, 'lang' => $handle,
    ])->all()));

    $root = base_path('content/collections');
    file_put_contents("$root/blog.yaml", Yaml::dump([
        'title' => 'Blog', 'sites' => array_keys($sites), 'propagate' => false, 'route' => '/blog/{slug}',
    ]));

    foreach (array_keys($sites) as $site) {
        @mkdir("$root/blog/$site", 0777, true);
    }

    mt_srand(42);
    for ($i = 1; $i <= $n; $i++) {
        file_put_contents("$root/blog/en/post-$i.md", "---\n".Yaml::dump(['id' => $id('en', $i), 'title' => "Post $i"])."---\nSome text.\n");

        foreach (['fr', 'de', 'es'] as $site) {
            $data = ['id' => $id($site, $i), 'origin' => $id('en', $i)];
            if (mt_rand(1, 2) === 1) {
                $data['title'] = "Post $i ($site)";
            }
            file_put_contents("$root/blog/$site/post-$i.md", "---\n".Yaml::dump($data)."---\n");
        }
    }

    echo "Wrote $n origins and ".($n * 3)." localizations.\n";

    return;
}

$i = intdiv(Entry::query()->where('collection', 'blog')->where('site', 'en')->count(), 2);
$origin = Entry::find($id('en', $i));
$url = Site::get('de')->absoluteUrl()."/blog/post-$i";

$lap = function ($label, $callback) {
    $callback();
    $times = [];
    for ($run = 0; $run < 9; $run++) {
        Blink::flush();
        $start = hrtime(true);
        $result = $callback();
        $times[] = (hrtime(true) - $start) / 1e6;
    }
    sort($times);
    printf("%-36s median %7.2f ms  %s\n", $label, $times[4], is_object($result) ? get_class($result).' '.($result->id() ?? '') : json_encode($result));
};

$lap('Entry::find() of a localization', fn () => Entry::find($id('de', $i)));
$lap('Entry::find() of a missing id', fn () => Entry::find('00000000-0000-0000-0000-000000000000'));
$lap('Data::findByRequestUrl()', fn () => Data::findByRequestUrl($url));
$lap('$origin->descendants()', fn () => $origin->descendants()->count());