Skip to content

test(cluster): seed aclear multipage test with one amset - #800

Draft
vishal-bala wants to merge 1 commit into
fix/cluster-ft-info-resp3-async-initfrom
fix/cluster-aclear-test-seed-with-amset
Draft

vishal-bala wants to merge 1 commit into
fix/cluster-ft-info-resp3-async-initfrom
fix/cluster-aclear-test-seed-with-amset

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

On the first run of the nightly cluster job (#796), test_embeddings_cache_aclear_multipage_cluster failed on the redis-py 8.x leg with AttributeError: 'NoneType' object has no attribute 'writelines'. It passed on the 6.x and 7.x legs, and it passes locally on 8.1.0. The fault is in redis-py 8.x, not RedisVL, but the test only reaches it because of how it seeds its data.

Root cause

The test seeded its 600 cache entries with asyncio.gather over 600 aset() calls on one async RedisCluster. Each aset is a single-key pipeline, so up to about 200 pipelines are in flight on each of the three primaries.

redis-py 8.0 lowered the async cluster's default max_connections per node from 2**31 to 100. When connecting is slow, as on the CI runner, more than 100 pipelines hold a connection at once and ClusterNode.acquire_connection raises MaxConnectionsError. That error subclasses ConnectionError, so the 8.1.0 pipeline treats it as retryable and calls cluster_client.aclose(). That closes the shared client and disconnects connections other coroutines are part-way through using. On Python 3.11, asyncio.wait_for yields between the connection's is_connected check and its write. A connection in the middle of its CLIENT SETINFO handshake therefore writes after disconnect() has set _writer = None, which is the CI traceback. On Python 3.12 and later the same race surfaces as ConnectionError or TimeoutError instead.

A reproduction with redis-py alone (300 concurrent single-key pipelines, max_connections=4, 20 rounds, local six-node cluster):

Python redis-py Result
3.11.9 8.1.0 55 of the CI AttributeError, plus ConnectionError and MaxConnectionsError
3.14.0 8.1.0 MaxConnectionsError, ConnectionError, TimeoutError
3.11.9 7.4.1 MaxConnectionsError only
3.11.9 6.4.0 MaxConnectionsError only

redis/redis-py#4448, merged but not yet released, stops the pipeline closing the client and makes MaxConnectionsError fail immediately. Once it ships, the old seeding would fail with MaxConnectionsError instead, so the test needs changing either way.

Changes

The test now seeds with one amset() call, which is a single pipeline using at most one connection per node. The comment saying amset writes nothing on an async cluster client was out of date, because #697 fixed that, so it's replaced. The now-unused import asyncio is removed.

Notes

This PR goes between #795 and #796 in the stack: #794 → #795 → this PR → #796. It doesn't depend on #794 or #795. It's stacked because #796's nightly job needs it to pass.

Tested locally:

  • On this branch, 11 of the 12 tests in test_redis_cluster_support.py pass against a local six-node Redis 8.4 cluster (Python 3.14, redis-py 8.1.0).
  • The remaining test, test_semantic_router_cluster_client, errored only because that venv lacks sentence-transformers.
  • With redis-py 8.1.0 on both Python 3.11 and 3.14, single-amset seeding writes all 600 keys even with max_connections=4.

Not tested:

  • The edited test on Python 3.11.
  • Anything on redis-py 8.0.x.

Cluster tests don't run on pull-request CI, so #796's nightly job is the first place this runs in CI.

Reviewer Brief

Start here: tests/integration/test_redis_cluster_support.py, the seeding step in test_embeddings_cache_aclear_multipage_cluster. It's the only behavioural change: 600 entries now go in through one amset() pipeline, not 600 concurrent aset() calls.

Not verified: the edited test hasn't run on Python 3.11, where the nightly redis-py 8.x leg failed, or against redis-py 6.x, 7.x or 8.0.x. requires_cluster tests don't run on pull-request CI, so #796's nightly job is the first CI run.

Beyond CI: on a local six-node Redis 8.4 cluster (Python 3.14, redis-py 8.1.0), the edited test and its sync twin pass. Single-amset seeding wrote all 600 keys with max_connections=4 on both Python 3.11 and 3.14. A redis-py-only script that runs concurrent single-key pipelines on one client reproduced the CI AttributeError 55 times in 20 rounds on Python 3.11 with redis-py 8.1.0. The old test itself never failed locally.

Reviewed 8e694b3d · Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviour

The test seeded 600 keys with 600 concurrent aset() calls, each a
single-key pipeline. On redis-py 8.x, whose async cluster defaults to
100 connections per node, that raised MaxConnectionsError on slow
connects; the 8.1.0 pipeline retry then closed the shared client and
tore down other coroutines' connections mid-handshake, failing with
"'NoneType' object has no attribute 'writelines'" on Python 3.11.

Seed with a single amset() instead. The comment that amset writes
nothing on an async cluster was stale: #697 fixed that.
vishal-bala added a commit that referenced this pull request Oct 30, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
@vishal-bala
vishal-bala added this pull request to stack #801 October 30, 2026 12:39
@vishal-bala vishal-bala added the auto:tests Add or improve existing tests label Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:tests Add or improve existing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant