test(dash): add a Behat suite and CI workflow by mrrobot47 · Pull Request #2 · EasyEngine/dash-command

mrrobot47 · 2026-10-19T20:04:53Z

Adds a Behat suite for dash-command (features/: 56 scenarios, outline rows counted) and a Behat workflow that runs it on every pull request, on pushes to develop and on demand.

It covers ee dash init (option checks, the server registration against a local EasyDash stub) and the site sync. features/COVERAGE.md maps each command, flag and behaviour to its scenarios.

The workflow runs the suite inside a fresh core checkout: core develop is cloned, every easyengine/* package is set to dev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's, not Packagist's.
It runs on PHP 7.4 and 8.5.
Scenarios written for open bugs are left out for now: they assert the fixed behaviour, so each comes back with its fix. Every scenario in this PR gates, and COVERAGE.md lists the bugs that aren't covered yet.

To run it locally (as root, on a test machine, since the suite creates and deletes *.test sites): replace a core checkout's features/ with this package's (rm -rf features && cp -R <package>/features features, as the workflow does; copying over it would keep core's own cli.feature, whose scenario runs ee cli self-uninstall --yes) and run sudo -E vendor/bin/behat from the core root.

Green run on my fork: https://github.com/mrrobot47/dash-command/actions/runs/39292411380

…stub Base the context on site-command's, with anchored step regexes, declared properties and a cleanup that deletes only bh-dash-command-* sites. Add generic output/file/DB/fixture steps and an EasyDash sandbox: a local stateful stub API on 127.0.0.1, a private EE_CONFIG_PATH for ed-api-url, and a private mount namespace for /root/.ssh and hostnamectl.

…fter each scenario - Refuse to run ee dash init when any non bh-dash-command-* site exists, since init walks every site. - Only allow a 127.0.0.1 ed-api-url and re-parse the private config, so a bad value can't fall back to the real EasyDash. - Point http(s)_proxy at a closed local port (no_proxy for 127.0.0.1 and .test) so nothing else leaves the host, api.ipify.org included. - Check the bind mounts with -ef and require a marker file, so a failed unshare (exit 1) can't pass as a validation error. - Add an AfterScenario undo stack for DB tweaks, moved files and setup commands. - Fail "the file should not contain" when the file is unreadable instead of passing.

…r more paths - Drop assertions that pinned DASH-8, DASH-12, DASH-14 and DASH-16 behaviour from regular scenarios; the known-bug scenarios own them. - Split the ee.log token check into DASH-6 (header debug line) and LOG-1 (argv). - Cover the api.ipify.org lookup and its DASH-4 prompt fallback, and DASH-14 (disabled sites started, failed enable aborts the sync). - Revert fixture changes through the undo stack and pin the PHP 8.0 fixture's real version. - Update COVERAGE.md.

…known bug The synopsis validator warns and unsets a valueless --api or --org before init() runs, so the empty-value check already rejects them (DASH-2 refuted on CI).

…ar ones Core masks secret flag values in every log line since v4.13.2 (easyengine#1944), so the API token no longer reaches ee.log.

@le

Runs the package's features/ inside a fresh core develop checkout, with this checkout installed as the package through a Composer path repository, so the PR's own code is under test. @le is excluded (no public DNS on runners). @known-bug scenarios run in a separate non-gating job.

Run PHP 7.4 and 8.5, strip control characters from the junit files before building failures.txt, and cap the Behat step at 75 minutes so a hung run still uploads its logs. Diagnostics: the host addresses and routes, and the docker-compose version after the first run (MIG-20).

EE's first run downloads docker-compose 2.27.0 without --fail (MIG-10); on a GitHub error it moves the HTML error page over the binary, as on an admin-tools-command runner. Fetch 2.27.0 again with --fail and retries in that case.

The 15 scenario rows written for open bugs assert the fixed behaviour and fail today, so they stay out of the suite until their fixes land, and COVERAGE.md lists those bugs as not covered yet. Every scenario that ships now gates, and the non-gating known-bug CI lane goes with them.

Copilot

🟡 Changes recommended

Gating scenarios depend on a documented hostname bug, and cleanup deletes unrelated local build artifacts.

2 open issues

What changed in this PR

Adds Behat integration coverage for dash-command, running the package checkout inside EasyEngine core.

Changes:

Adds 56 scenarios covering validation, server registration, and site sync.
Introduces a local API stub, sandbox, and fixture helpers.
Adds PHP 7.4/8.5 CI runs and coverage documentation.

File	Description
features/dash.feature	Command and argument validation scenarios.
features/dash-sites.feature	Site-sync payload and behavior scenarios.
features/dash-server.feature	Server registration and error-handling scenarios.
features/COVERAGE.md	Coverage inventory, requirements, and known gaps.
features/bootstrap/support.php	Shared assertion helpers.
features/bootstrap/FeatureContext.php	Step definitions, sandbox, and cleanup.
features/bootstrap/easydash-stub-router.php	Stateful local EasyDash API stub.
.github/workflows/behat.yml	CI setup, test execution, and result artifacts.

_{Review effort: Balanced}

The AfterFeature cleanup deleted ee.phar and ee-old.phar from the core checkout after every feature, although no scenario here builds them, so a local run removed a developer's own builds. The 'ee phar is generated' step now records that it built them, and the cleanup only removes them then.

Two scenarios passed --hostname=bh-dash.example.test and expected the machine hostname in the payload, which only holds while --hostname is ignored. They now pass the machine hostname, so the payload is the same before and after the fix.

Copilot

🔵 Needs a closer look

Root-level mount isolation and host-mutating fixture cleanup require final human validation.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Fix container count assertion type handling and mismatch logic

features/bootstrap/FeatureContext.php:415

This container-count assertion accepts incorrect counts: Behat supplies the expected count as a string, while $running_containers is an integer, so === is false. Even with matching types, the condition would reject a correct count instead of a mismatch. Cast the expected value to an integer and throw when the counts differ. The current feature scenarios do not use this step, so this does not affect their results.

2 resolved since last review

_{Review effort: Balanced}

… step Neither step is used yet. The count step compared a string with an int and threw on a match; the subsite step swapped the site and the subsite and still used the pre-compose-v2 container name.

…line output compareContents() compared list elements by position, so [1,3,5] wasn't found in [1,2,3,4,5], and a missing key or property raised a notice. The indexed exact-output step and both something-like steps only used the first line of the expected text.

Copilot

🟡 Changes recommended

Two gating scenarios depend on documented transport-error bugs and would reject fixes to those fallbacks.

2 open issues

_{Review effort: Balanced}

…valid once their fallbacks work The unreachable-API scenario no longer asserts that authorized_keys is untouched (a working server-lookup fallback appends the key before add_server fails), and the IP-lookup scenario supplies a known server's IP on STDIN so a working IP prompt doesn't register a server with an empty IP.

Copilot

🔵 Needs a closer look

The root-level mount sandbox and destructive fixture cleanup require final human validation.

No open issues 🎉

2 resolved since last review

_{Review effort: Balanced}

Copying the suite over core's features/ keeps core's cli.feature, whose scenario runs ee cli self-uninstall --yes; replace the directory as the workflow does.

mrrobot47 added 13 commits October 18, 2026 19:05

test(dash-command): cover dash command surface and validation

dc03d6f

test(dash-command): cover dash init server registration

d6bd3a7

test(dash-command): cover dash init site sync

cf5e7c2

test(dash-command): add behat coverage inventory

2eb66bd

test(dash-command): treat a valueless --api/--org as rejected, not a …

99f8a2e

…known bug The synopsis validator warns and unsets a valueless --api or --org before init() runs, so the empty-value check already rejects them (DASH-2 refuted on CI).

test(dash-command): run the DASH-6 and LOG-1 token scenarios as regul…

25b479b

…ar ones Core masks secret flag values in every log line since v4.13.2 (easyengine#1944), so the API token no longer reaches ee.log.

rtBot requested a balanced review from Copilot October 19, 2026 20:04

Copilot started reviewing on behalf of rtBot October 19, 2026 20:05

View session

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Comment thread features/bootstrap/FeatureContext.php Outdated

Comment thread features/dash-server.feature Outdated

mrrobot47 added 2 commits October 20, 2026 01:53

rtBot requested a balanced review from Copilot October 19, 2026 20:39

Copilot started reviewing on behalf of rtBot October 19, 2026 20:39

View session

Copilot AI reviewed Oct 19, 2026

View reviewed changes

mrrobot47 added 4 commits October 20, 2026 02:25

test(behat): fix the container count step and drop the unused subsite…

edf6539

… step Neither step is used yet. The count step compared a string with an int and threw on a match; the subsite step swapped the site and the subsite and still used the pre-compose-v2 container name.

docs(dash-command): describe coverage gaps without workspace references

d7e121d

ci(dash-command): run the behat lanes on every supported PHP version

128bb74

rtBot requested a balanced review from Copilot October 20, 2026 04:15

Copilot started reviewing on behalf of rtBot October 20, 2026 04:16

View session

Copilot AI reviewed Oct 20, 2026

View reviewed changes

Comment thread features/dash-server.feature Outdated

Comment thread features/dash-server.feature Outdated

rtBot requested a balanced review from Copilot October 20, 2026 04:22

Copilot started reviewing on behalf of rtBot October 20, 2026 04:22

View session

Copilot AI reviewed Oct 20, 2026

View reviewed changes

docs(dash-command): describe local runs that replace core's features

b8921cb

Copying the suite over core's features/ keeps core's cli.feature, whose scenario runs ee cli self-uninstall --yes; replace the directory as the workflow does.

Navigation Menu

Repository navigation

test(dash): add a Behat suite and CI workflow - #2

test(dash): add a Behat suite and CI workflow#2
mrrobot47 wants to merge 21 commits into
EasyEngine:developfrom
mrrobot47:test/behat-suite-r3

mrrobot47 commented Oct 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mrrobot47 commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mrrobot47 commented Oct 19, 2026 •

edited

Loading