Repository navigation
Retry LSPS0 discovery for undiscovered LSPs before a JIT request - #1151
hash-money wants to merge 1 commit into
Conversation
Protocol discovery runs once, in the background, when the node starts. If a configured LSP is unreachable at that moment (e.g. it is restarting), its `supported_protocols` stay `None`, `get_lsps2_nodes` filters it out, and every JIT invoice fails with `LiquiditySourceUnavailable` until the node restarts, even once the LSP is reachable and connected. Before an LSPS2 JIT request, re-attempt discovery (connecting first) for configured LSPs that are still undiscovered after start-up discovery has finished. Failures are logged and the request proceeds as before. The new integration test starts the client while its LSP is not listening, then starts the LSP and connects; without the fix the JIT invoice fails with `LiquiditySourceUnavailable`.
|
I've assigned @tnull as a reviewer! |
ajaysehwal
left a comment
There was a problem hiding this comment.
Clarification: startup discovery already retries in the background
The premise that a failed LSP discovery is never retried until restart doesn't match main. Node::start already spawns a background task that retries discovery for undiscovered LSPs, starting at 5 seconds and doubling up to a 1-hour cap.
So the actual gap is that retries can be delayed significantly after a startup failure. The on-demand retry in this PR is still useful, but could we update the PR description and test comment to reflect this and position the change as complementing the existing backoff mechanism?
| ) -> Result<LdkBolt11Invoice, Error> { | ||
| let connection_manager = Arc::clone(&self.connection_manager); | ||
| let res = self.runtime.block_on(async move { | ||
| // An LSP that was unreachable when the node started was never discovered; retry it |
There was a problem hiding this comment.
Medium: on-demand discovery can delay every JIT request
rediscover_undiscovered_lsps runs serially before gather_lsps2_offers, awaiting both the peer connection and protocol discovery for each undiscovered LSP. Since discovery can take up to 5 seconds, even one unreachable LSP can delay requests to healthy LSPs, and multiple dead LSPs can multiply that delay.
There's also no cooldown or rate limit, so concurrent invoice requests can all wait on the same failing discovery attempts.
Could we add a cooldown or deduplicate the entire rediscovery operation so requests don't repeatedly pay this latency? Ideally, healthy LSP offers should still be queried without waiting for unreachable peers.
|
You're right, thanks, our premise was wrong: a201d40's backoff retry is in our base and I missed it. Re-running the test on 1dea48a: the first retry fired at +5 s and was refused, about 50 ms before the test connected to the newly started LSP; the invoice was then requested before the second retry at +15 s. Polling the request instead, it succeeds at +15.2 s ( I'm happy to close this. If you'd rather keep an on-demand retry, I'd rework it so the JIT path nudges the existing backoff task to retry now (rate-limited), and only waits for it when no LSPS2 LSP is known yet, so healthy LSPs are never delayed; and I'd fix the description and test comment. Your call. |
Camillarhi
left a comment
There was a problem hiding this comment.
Thanks for re-running, that clears it up. On main, an LSP that isn't discovered at startup gets picked up by the next retry once it's reachable, so JIT requests can fail for up to roughly as long as discovery had been failing (about 10s in your test). Shortening that window would mean a faster backoff schedule or an on-demand retry along the lines you described.
Before an LSPS2 JIT request, re-run LSPS0 discovery for configured LSPs that are still undiscovered. This complements the background retry that
Node::startalready runs for LSPs that failed start-up discovery (a201d40: first retry after 5 s, doubling to a 1 h cap).Correction: the first version of this description said discovery runs once and is never retried until restart. That was wrong for
main; see the discussion below.Gap: after a start-up discovery failure, a JIT request made between background retries fails with
LiquiditySourceUnavailable, and the next retry can be minutes away (up to an hour later on).Test:
lsps2_discovery_retried_after_lsp_unreachable_at_startintests/integration_tests_rust.rs.connectthe client to it.On
main(1dea48a) this fails withLiquiditySourceUnavailable. The first background retry fires at +5 s, just before the service is up, and is refused; the request then lands before the second retry at +15 s. With the request polled instead, it succeeds at +15.2 s.Change:
LiquiditySource::rediscover_undiscovered_lsps. Before an LSPS2 JIT request, it connects to and re-runs LSPS0 discovery for configured LSPs that are still undiscovered after start-up discovery has finished. It does nothing while start-up discovery is in flight, sinceget_lsps2_nodesalready waits for that. Failures are logged and the request proceeds as before.Open review point: as written, the on-demand attempt runs serially and can delay requests to healthy LSPs, with no dedupe or cooldown. If this is kept, I'll rework it so the JIT path nudges the existing backoff task to retry now (rate-limited), and only waits for it when no LSPS2 LSP is known yet. I'm also happy to close it.
Tests: the new test plus the existing
lsps2_*integration tests (5/5) pass. The CI clippy rule (-A warnings -D clippy::unwrap_used) is clean.🤖 Generated with Claude Code