Skip to content

fix(restheart-mongo): flow.sh waits for RESTHeart to answer, not for a deadline - #154

Draft
slayerjain wants to merge 3 commits into
mainfrom
train/fix/restheart-flow-waits-on-app
Draft

slayerjain wants to merge 3 commits into
mainfrom
train/fix/restheart-flow-waits-on-app

Conversation

@slayerjain

Copy link
Copy Markdown
Member

What

restheart-mongo/flow.sh now waits for RESTHeart to answer, not for a deadline:

  • record-traffic used to give RESTHeart a fixed 60 s (a date +%s deadline), then slept 5 s more.
  • bootstrap used to wait for a caller's count of seconds (240).

restheart_wait_for_app now waits until RESTHeart answers its root with 200, or with 401 (asking for credentials), for as long as its container runs:

  • A container that has exited, or does not exist, ends the wait at once and fails it, with its last log lines.
  • A container that runs but never answers is left to the caller's own bound, such as a CI step's timeout.

The 5 s sleep is gone. RESTHeart's compose service starts only once Mongo is healthy, so once RESTHeart answers, it is up.

Also:

  • bootstrap still accepts the seconds argument older callers pass (keploy/enterprise's lane passes 240) and ignores it.
  • .github/workflows/scripts/run-and-measure.sh drops its 120 × 2 s poll and relies on flow.sh bootstrap's wait.
  • The README examples drop the 240.

Why

On a loaded CI host, a JVM can take longer than any fixed count to start. keploy/enterprise's restheart lane recorded nothing when RESTHeart answered after flow.sh's 60 s were up, because the host was holding up keploy's eBPF load.

The enterprise lane now waits for RESTHeart itself before it calls record-traffic (await_ready), so flow.sh finds it answering at the first probe. Its own wait no longer cuts a slow start short.

Testing

  • Three keploy/enterprise restheart compat cells, run on a test box (record-pr-replay-pr, record-stable-replay-pr, record-pr-replay-stable), all against this branch's restheart-mongo: 307 tests recorded per cell, and every replay PASSED.
  • In both bootstrap and record, the app answered at the first probe after the lane's own wait.
  • This PR's restheart-mongo workflow runs the standalone run-and-measure.sh path.

…a deadline

record-traffic gave RESTHeart a fixed 60 s, by a `date +%s` deadline, and
then slept another 5 s. Bootstrap gave it a caller's count of seconds (240).
On a loaded CI host a JVM can take longer than any such count to start:
keploy/enterprise's restheart lane recorded nothing when RESTHeart answered
after the 60 s, on #2922 (pipeline 11773), with keploy's eBPF load held up by
the host.

restheart_wait_for_app now waits for RESTHeart to answer its root (200, or
401 asking for credentials) for as long as its container is running. A
container that has exited, or is not there, ends the wait at once and fails
it, with the container's last log lines. One that runs and never answers is
left to the caller's own bound (a CI step's timeout). The 5 s sleep is gone:
the app is up once it answers, since its compose starts it only after Mongo
is healthy.

bootstrap still accepts a seconds argument from callers that pass one
(keploy/enterprise's lane passes 240) and ignores it. run-and-measure.sh's
120 x 2 s poll goes too: it now relies on flow.sh bootstrap's wait. The
README examples drop the 240.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
@github-actions

Copy link
Copy Markdown

restheart-mongo sample coverage

ref coverage
base (main) 0.0%
this PR 53.1%

Threshold: PR may not drop coverage by more than 1.0pp. Override per-repo via the RESTHEART_COVERAGE_THRESHOLD actions variable.

… docker hiccup, as the verdict

Review of 6870873:

- A container that is created or restarting is on its way to running, so
  the wait keeps going for it. Only one that has exited, died, or is not
  there ends the wait.
- A `docker inspect` that fails for another reason (a daemon slow under
  load) is not read as the container being gone. The wait goes on.
- run-and-measure.sh's error now says that bootstrap failed, which can be
  RESTHeart not coming up or the seeding failing, not only the first.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
…once when it does not answer

Review of 4a65d8e: every docker error other than "No such" was taken as
transient, silently and for ever, including exit 126/127 (no docker to run)
and a socket it may not open. 126/127 now fail the wait at once, with what
docker said; any other error is said once, and the wait goes on.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant