Repository navigation
Conversation
`and` and `or` evaluated both operands before combining them, so a guard could not protect the operand after it. With an empty optional input, `inputs.payload != '' and (inputs.payload | from_json | length)` still ran `from_json` on the empty string and failed the run with "invalid JSON". Return as soon as the left operand decides the result. The validation probes behind condition corrections (`_evaluator_rejects`, `_collect_leaves`) now pass a `_ProbeNamespace` as the outer namespace and keep evaluating both operands, so an unknown name or misused filter in a skipped operand is still reported. Assisted-by: Codex CLI (model: gpt-6-astra, autonomous) Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Contributor
There was a problem hiding this comment.
馃煝 Approval recommended
The implementation is consistent, documented, and covered by positive, negative, probe, and end-to-end regression tests.
0 open findings
What changed in this PR
Implements lazy boolean evaluation in workflow expressions while preserving exhaustive validation probes.
Changes:
- Short-circuits
and/orat runtime. - Preserves full operand traversal during validation.
- Adds unit, CLI regression, and documentation coverage.
The submitted regression evidence was reviewed; tests were not rerun in this environment.
| File | Description |
|---|---|
src/鈥媠pecify_cli/鈥媤orkflows/鈥媏xpressions.py |
Implements short-circuiting and probe behavior. |
tests/鈥媡est_workflows.py |
Covers expression semantics and failures. |
tests/鈥媢nit/鈥媡est_condition_expression_block.py |
Verifies exhaustive validation probes. |
tests/鈥媠pecify_cli/鈥媤orkflows/鈥媡est_command_run.py |
Adds end-to-end workflow regression coverage. |
docs/鈥媟eference/鈥媤orkflows.md |
Documents boolean evaluation behavior. |
馃 Review effort: Balanced
馃挕 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Workflow expressions evaluate both operands of
andandorbefore combining them (_evaluate_simple_expression,expressions.py:884-894onmain). So a guard can't protect the operand after it. The usual way to read an optional JSON input fails when the input is left empty:inputs.payload != ''is false, butfrom_jsonstill runs on the empty string and the run fails before either branch.inputs.payload == '' or (...)fails the same way.This PR returns as soon as the left operand decides the result:
andskips its right side when the left is false, andorwhen the left is true. Results are still booleans, precedence and parentheses are unchanged, and an operand that does get evaluated still fails the run on an error.The validation probes behind condition corrections (
_evaluator_rejects,_collect_leaves) rely on visiting both operands.test_both_sides_of_a_boolean_are_reportedexists to catch a short-circuit that would blind them. The probes now pass a_ProbeNamespaceas the outer namespace, and the evaluator keeps visiting both operands for it, so an unknown name or a misused filter in a skipped operand is still reported. Root lookups usedict.get(), so the wrapper doesn't change what resolves. With a plain short-circuit and no probe exception, 10 probe tests fail, including the existingtest_both_sides_of_a_boolean_are_reported.docs/reference/workflows.mdgets a paragraph on howand/orevaluate, with this guard as the example.Repro with the workflow above:
main(0443760)specify workflow run optional.ymlError: from_json: invalid JSON: Expecting value: line 1 column 1 (char 0), exit 1[absent]runs,Status: completed, exit 0specify workflow run optional.yml --input payload=[1][provided]runs, exit 0specify workflow run optional.yml --input payload=brokenfrom_json: invalid JSON, exit 1The bundled workflows don't use
and/ortoday, so this affects workflows that users write.Testing
uv run specify --helpuv sync && uv run pytestNew tests:
tests/test_workflows.py::TestExpressions: guards withand,or, mixed precedence and nesting short-circuit on an empty payload; a needed operand, on either side, still raises.tests/unit/test_condition_expression_block.py: the probes still collect leaves and reject unknown or misused filters in operands that runtime skips (true or (...),false and (...), nested).tests/specify_cli/workflows/test_command_run.py:specify workflow run --jsonin process, with real shell steps and persisted state, for both guards with an empty, a valid and an invalid payload.With only
src/reverted tomain, 5 of the 24 new or changed tests fail: 3 expression cases and the 2 empty-payload CLI cases. With the fix, all 24 pass.Full suite on Linux, Python 3.13: 11435 passed, 264 skipped (
uv sync --extra test, then pytest in that venv, 12m36s). Ruff 0.15.0 (ruff check src tests) passes, and markdownlint passes ondocs/reference/workflows.md.AI Disclosure
AI disclosure: Two agents, both autonomous. Codex CLI (model: gpt-6-astra, reasoning effort xhigh, autonomous, sandboxed and offline) found this while scouting the workflow expression evaluator, wrote the reproduction, and generated the fix, tests and docs paragraph. Claude Code (model: claude-opus-5-5, autonomous) checked that the probe marker doesn't change name resolution, moved and rewrote the docs paragraph and the
_ProbeNamespacedocstring, reran the tests againstmainand the fix (plus the plain short-circuit variant above), reran the repro with the realspecifybinary, ran the full suite and lint, and drafted this description. Commits carryAssisted-by:trailers.