Skip to content

test(log): add a Behat suite and CI workflow - #9

Open
mrrobot47 wants to merge 22 commits into
EasyEngine:developfrom
mrrobot47:test/behat-suite-r3
Open

mrrobot47 wants to merge 22 commits into
EasyEngine:developfrom
mrrobot47:test/behat-suite-r3

Conversation

@mrrobot47

@mrrobot47 mrrobot47 commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Adds a Behat suite for log-command (features/: 52 scenarios, outline rows counted) and a Behat workflow that runs it on every pull request, on pushes to develop and on demand.

It covers ee log show for a site (type filters, line counts, following), --cli and --global, site resolution and validation errors. features/COVERAGE.md maps each command, flag and behaviour to its scenarios.

  • The workflow runs the suite inside a fresh core checkout: core develop is cloned, every easyengine/* package is set to dev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's, not Packagist's.
  • It runs on PHP 7.4 and 8.5.
  • Scenarios written for open bugs are left out for now: they assert the fixed behaviour, so each comes back with its fix. Every scenario in this PR gates, and COVERAGE.md lists the bugs that aren't covered yet.

To run it locally (as root, on a test machine, since the suite creates and deletes *.test sites): replace a core checkout's features/ with this package's (rm -rf features && cp -R <package>/features features, as the workflow does; copying over it would keep core's own cli.feature, whose scenario runs ee cli self-uninstall --yes) and run sudo -E vendor/bin/behat from the core root.

Green run on my fork: https://github.com/mrrobot47/log-command/actions/runs/39292416544

Based on site-command's context. Cleanup only deletes sites with the bh-log-command- prefix, the unanchored I run and STDOUT regexes are anchored, and a marked section adds generic steps: lazy site setup, file fixtures confined to the suite's sites, commands bounded by timeout(1) for tail -f, stream and exit-code assertions, and HTTP checks.
proc_open's cwd resolves the app/ volume symlink, so get_site_name() saw /var/lib/docker/volumes/… and the site-detection scenario could never pass. Pass the logical path as PWD, as an interactive cd does.
Add a @known-bug scenario for the unescaped file list (LOGCMD-1), make "--cli honours --n" assert a single ee.log line instead of a check that also passed with the default, reference the issues.md IDs and state that @known-bug scenarios fail until fixed.
The shell injection and --n validation fixes are on fix/escape-log-tail. The --n injection scenario now also checks the up-front rejection.
…LOGCMD-9)

Also runs the LOGCMD-4 --global scenario as a regular one: the LOGCMD-9 fix stops reading $wp_logs_path for non-WordPress sites.
Run PHP 7.4 and 8.5, strip control characters from the junit files before building failures.txt, and cap the Behat step at 75 minutes so a hung run still uploads its logs. Diagnostics: the host addresses and routes, and the docker-compose version after the first run (MIG-20).
EE's first run downloads docker-compose 2.27.0 without --fail (MIG-10); on a GitHub error it moves the HTML error page over the binary, as on an admin-tools-command runner. Fetch 2.27.0 again with --fail and retries in that case.
The 7 scenario rows written for open bugs assert the fixed behaviour and fail today, so they stay out of the suite until their fixes land, and COVERAGE.md lists those bugs as not covered yet. Every scenario that ships now gates, and the non-gating known-bug CI lane goes with them.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Local runs delete pre-existing PHAR artifacts and fail directory-resolution scenarios when checkout paths contain spaces.

2 open issues
What changed in this PR

Adds integration coverage for ee log using Behat inside an EasyEngine core checkout.

Changes:

  • Adds scenarios for site, CLI and global logs, validation, filtering and following.
  • Introduces fixture helpers and a coverage inventory.
  • Adds PHP 7.4/8.5 CI runs against the checked-out package, with uploaded results.
File Description
features/​log.feature Command surface and site resolution scenarios.
features/​log-site.feature Site log filtering, following and safety scenarios.
features/​log-global.feature CLI and global logging scenarios.
features/​COVERAGE.md Coverage map and deferred bugs.
features/​bootstrap/​support.php Shared assertion utilities.
features/​bootstrap/​FeatureContext.php Steps, fixtures, timeouts and cleanup.
.github/​workflows/​behat.yml Core-based test matrix and result collection.

Review effort: Balanced

Comment thread features/bootstrap/FeatureContext.php Outdated
Comment thread features/bootstrap/FeatureContext.php Outdated
The AfterFeature cleanup deleted ee.phar and ee-old.phar from the core checkout after every feature, although no scenario here builds them, so a local run removed a developer's own builds. The 'ee phar is generated' step now records that it built them, and the cleanup only removes them then.
The checkout path was spliced into the command unquoted (and as a replacement string), so a path with spaces or a backslash broke the current-directory scenarios.
… step

Neither step is used yet. The count step compared a string with an int and threw on a match; the subsite step swapped the site and the subsite and still used the pre-compose-v2 container name.
…ine output

compareContents() compared list elements by position, so [1,3,5] wasn't found in [1,2,3,4,5], and a missing key or property raised a notice. The indexed exact-output step and both something-like steps only used the first line of the expected text.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The CSV assertion helper can incorrectly pass when duplicate actual rows compensate for missing expected rows.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Prevent duplicate actual rows from masking missing expected rows

features/​bootstrap/​support.php:197

The shared match counter lets duplicate actual rows compensate for missing expected rows. For example, actual CSV name\nfoo\nfoo returns true for expected rows [['name'], ['foo'], ['bar']], although bar is absent. Track a match separately for each expected row and stop after its first match to prevent false-positive assertions.

2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved blocking issues were identified in the scenarios, test harness or CI integration.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants