Skip to content

Avoid storing MCP gateway stderr in artifacts - #50744

Closed
pelikhan wants to merge 13 commits into
mainfrom
copilot/avoid-mcp-stderr-artifact
Closed

pelikhan wants to merge 13 commits into
mainfrom
copilot/avoid-mcp-stderr-artifact

Conversation

@pelikhan

@pelikhan pelikhan commented Aug 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • stop writing MCP gateway stderr to /tmp/gh-aw/mcp-logs/stderr.log
  • emit container stderr through core.info during gateway shutdown without logging stdout configuration
  • explicitly exclude the legacy stderr path from agent artifacts, including ARC/DinD path rewriting

Validation

  • make agent-report-progress components passed (build, JavaScript lint, schema freshness, impacted Go and JavaScript tests)
  • golangci-lint run ./pkg/constants ./pkg/workflow
  • targeted MCP gateway JavaScript and workflow compiler tests

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 7.24 AIC · ⊞ 8.3K · ◷
Comment /souschef to run again

Emit gateway stderr through core.info during shutdown and explicitly exclude the legacy stderr path from artifact uploads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 1554a64b-2d03-4321-819f-4d590e44e95b
@pelikhan

pelikhan commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@copilot merge main and recompile

Copilot AI and others added 4 commits August 6, 2026 03:19
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…err-artifact

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Regenerate workflow lock files with the core.info shutdown logger and stderr artifact exclusion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 1554a64b-2d03-4321-819f-4d590e44e95b
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1554a64b-2d03-4321-819f-4d590e44e95b
@pelikhan
pelikhan marked this pull request as ready for review August 6, 2026 03:26
Copilot AI balanced review requested due to automatic review settings August 6, 2026 03:26
@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

No ADR enforcement needed: PR does not have the 'implementation' label and has ≤100 new lines of code in business logic directories (35 additions detected).

@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates MCP gateway shutdown and artifact handling to avoid retaining gateway stderr in artifacts.

Changes:

  • Captures gateway stderr through the shutdown action instead of a file.
  • Excludes the legacy stderr path, including ARC/DinD rewrites.
  • Regenerates affected workflow lock files and adds tests.
Show a summary per file
File Description
actions/setup/js/start_mcp_gateway.cjs Stops persisting gateway stderr.
actions/setup/js/stop_mcp_gateway.cjs Emits stderr during shutdown.
actions/setup/js/stop_mcp_gateway.test.cjs Tests stderr-only emission.
pkg/constants/constants.go Defines the legacy stderr path.
pkg/workflow/compiler_yaml_ai_execution.go Uses the JavaScript shutdown action.
pkg/workflow/compiler_yaml_post_agent.go Excludes stderr from artifacts.
pkg/workflow/arc_dind_artifacts.go Rewrites negated ARC/DinD paths.
pkg/workflow/arc_dind_artifacts_test.go Tests negated-path rewriting.
pkg/workflow/mcp_logs_upload_test.go Tests stderr exclusion and shutdown generation.
.github/workflows/*.lock.yml Regenerates affected workflows with the new shutdown step and artifact exclusion.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 284/284 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread actions/setup/js/stop_mcp_gateway.cjs Outdated
}

async function main() {
await printGatewayStderr(exec, core);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. main() now runs stop_mcp_gateway.sh first, then calls docker logs awmg-mcpg after the container is stopped (the EXIT trap issues docker stop, leaving the container available). The container is then explicitly removed with docker rm after the stderr has been collected.

// Container stderr is retrieved from Docker and emitted through core.info
// by stop_mcp_gateway.cjs after agent execution. Never persist it in mcp-logs,
// because it can contain credentials that must not enter the agent artifact.
stdio: ["pipe", outputFd, "ignore"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. After the stop script runs, collectGatewayStderr in stop_mcp_gateway.cjs writes the Docker stderr to /tmp/gh-aw/mcp-logs/stderr.log (the path already excluded from artifacts by !constants.TmpMcpStderrLog). parse_mcp_gateway_log.cjs still reads from that path for ai_credits_rate_limit_error and unknown_model_ai_credits detection — the parser flow is fully preserved.

Comment thread actions/setup/js/stop_mcp_gateway.cjs Outdated

const stderr = result.stderr.trim();
if (stderr) {
coreApi.info(`MCP Gateway stderr:\n${stderr}`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The core.info call that printed raw stderr has been removed. The stop step now only logs the byte count (MCP Gateway stderr written to log (N bytes).), never the content. The raw stream is written directly to the excluded file path without any emission to the Actions log.

…t' into copilot/avoid-mcp-stderr-artifact

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@copilot merge main and recompile

Addressed in 1c5203b.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

PR Review: Avoid storing MCP gateway stderr in artifacts

/diagnosing-bugs + /tdd applied. This is a clean, well-reasoned security fix.

One minor observation worth addressing in a follow-up:

In stop_mcp_gateway.test.cjs, the test for the unavailable stderr path (exitCode !== 0 and empty stderr) is missing. The current test covers "stderr present" and "exitCode === 0 with no stderr", but not the third branch (exitCode !== 0 → "MCP Gateway stderr is unavailable."). A test like:

it("reports when stderr is unavailable", async () => {
  const execApi = {
    getExecOutput: vi.fn().mockResolvedValue({ exitCode: 1, stdout: "", stderr: "" }),
  };
  const coreApi = { info: vi.fn() };
  await printGatewayStderr(execApi, coreApi);
  expect(coreApi.info).toHaveBeenCalledWith("MCP Gateway stderr is unavailable.");
});

...would complete the coverage of all three branches per /tdd. Not a blocker.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 58.6 AIC · ⊞ 7.1K · ◷
Comment /matt to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — well-structured security fix. Approving with a minor test coverage note.

Positive Highlights

  • ✅ Stderr redirected to "ignore" on spawn — root cause eliminated
  • ✅ !stderr.log exclusion protects against stale files from older runners
  • ✅ docker logs separation via getExecOutput + silent: true preserves diagnostics without leaking credentials
  • ✅ printGatewayStderr is injected (not global) — clean and testable
  • ✅ ARC/DinD path rewriting correctly handles ! prefix> 🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 58.6 AIC · ⊞ 7.1K

Comment /matt to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Review: Avoid storing MCP gateway stderr in artifacts

This is a well-scoped, security-focused change. The implementation is correct and test coverage is thorough.

What the PR does

  • Eliminates credentials leak vector: Gateway stderr was written to /tmp/gh-aw/mcp-logs/stderr.log and included in job artifacts. Since gateway stdout can contain generated client configuration (API keys, ports), routing stderr to an artifact-included file is a latent credential-leakage risk. The PR stops this.
  • Redirects stderr to core.info at shutdown: The new stop_mcp_gateway.cjs calls docker logs awmg-mcpg with silent: true and emits only result.stderr via core.info, correctly ignoring result.stdout.
  • Adds artifact exclusion as defense-in-depth: !/tmp/gh-aw/mcp-logs/stderr.log is appended to collectArtifactPaths so a stale or third-party file at the legacy path is never uploaded.
  • Fixes ARC/DinD path rewriting for negated globs: rewriteTmpGhAwPathsForArcDind now strips the leading ! before prefix substitution, preserving the negation — test asserts !${{ runner.temp }}/gh-aw/mcp-logs/stderr.log.

Quality observations

  • All five error-path branches in start_mcp_gateway.cjs are updated consistently.
  • stop_mcp_gateway.test.cjs explicitly asserts that stdout (the API key fixture) never appears in core.info calls — directly guarding the security property.
  • TmpMcpStderrLog constant is introduced so the path is defined in one place.
  • Lock files are regenerated as expected.

No blocking issues found.> 🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 59.6 AIC · ⊞ 5.3K

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🧪 Test Quality Sentinel Report

🔶 Test Quality Score: 80/100 — Excellent (failed on violation)

Analyzed 7 test(s): 5 design, 2 implementation, 1 violation.

📊 Metrics (7 tests)
Metric Value
Analyzed 7 (Go: 5, JS: 2)
✅ Design 5 (71%)
⚠️ Implementation 2 (29%)
Edge/error coverage 5 (71%)
Duplicate clusters 0
Inflation No
🚨 Violations 1
Test File Classification Issues
emits only container stderr through core.info stop_mcp_gateway.test.cjs design_test, behavioral_contract, high_value None
reports when no stderr was produced stop_mcp_gateway.test.cjs design_test, behavioral_contract, high_value None
rewrites /tmp/gh-aw/ paths to runner.temp arc_dind_artifacts_test.go design_test, behavioral_contract, high_value ⚠️ Missing build tag
preserves paths already using runner.temp arc_dind_artifacts_test.go design_test, behavioral_contract, high_value ⚠️ Missing build tag
preserves unrelated paths arc_dind_artifacts_test.go design_test, behavioral_contract, high_value ⚠️ Missing build tag
TestMCPLogsUpload mcp_logs_upload_test.go implementation_test, high_value None
TestMCPLogsUploadWithoutPlaywright mcp_logs_upload_test.go implementation_test, high_value None
⚠️ Flagged Tests (1)

TestRewriteTmpGhAwPathsForArcDind (pkg/workflow/arc_dind_artifacts_test.go) — Hard violation: missing mandatory //go:build tag on line 1.

All new Go *_test.go files must begin with either //go:build !integration (for unit tests) or //go:build integration. arc_dind_artifacts_test.go starts with package workflow on line 1 with no build constraint. Since these are unit tests, add //go:build !integration as the first line.

Test Highlights

  • stop_mcp_gateway.test.cjs is the strongest test in this PR — it verifies the security invariant that stdout (which may contain API keys) is never logged while stderr is, and uses a negative assertion (not.toHaveBeenCalledWith(stringContaining("must-not-be-logged"))). This is a high-value behavioral contract.
  • arc_dind_artifacts_test.go has excellent table-driven coverage including idempotency and unrelated-path preservation edge cases. Blocked only by the missing build tag.
  • mcp_logs_upload_test.go tests are compiler integration tests that verify generated YAML content. They are classified as implementation tests because they assert on exact string matches in generated output rather than higher-level behavioral contracts, but they provide genuine regression protection for the artifact exclusion behavior (!/tmp/gh-aw/mcp-logs/stderr.log).

Verdict

❌ Failed. 29% implementation tests (within 30% threshold). Failing reason: 1 guideline violation — arc_dind_artifacts_test.go is missing the mandatory //go:build build tag on line 1. Add //go:build !integration as the first line to fix.

🧪 Test quality analysis by Test Quality Sentinel · sonnet46 · 56.2 AIC · ⊞ 7.7K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Test Quality Sentinel: 80/100. 1 guideline violation — arc_dind_artifacts_test.go is missing the mandatory //go:build !integration build tag on line 1. Review flagged tests in the comment above.

@pelikhan

pelikhan commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@copilot Run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

REQUEST_CHANGES: gateway stderr redaction regressed

This PR removes the stderr-to-artifact leak but introduces a worse one: gateway stderr (explicitly documented as possibly containing credentials) is now piped straight to the Actions log via core.info, which is never scrubbed by the redact_secrets step. That's a regression, not a fix, for the stated goal of not exposing secrets.

Themes
  • Correctness/security: printGatewayStderr bypasses the existing secret-redaction mechanism (file-based scrubbing only), exposing potential credentials directly in the live log output — a step backward from hiding them in an excluded artifact file.
  • Minor: exclusion-glob ordering for the new !/tmp/gh-aw/mcp-logs/stderr.log pattern depends on append order across several code paths; worth double-checking there's no later insertion that could shadow it.
  • Positive: the ARC/DinD negation-prefix handling in rewriteTmpGhAwPathsForArcDind and the added tests are clean and correctly preserve the ! prefix through rewriting.
> 🔎 *Code quality review by [PR Code Quality Reviewer](https://github.com/github/gh-aw/actions/runs/31068425424)* · auto · 130.1 AIC · ⊞ 7.8K > Comment /review to run again

Comments that could not be inline-anchored

actions/setup/js/stop_mcp_gateway.cjs:22

Gateway stderr is printed straight into the Actions log via core.info, before the redaction step runs and outside its file-scan mechanism — so any credential it contains reaches unmasked, unredacted output.

<details><summary>Why this matters and a fix</summary>

The redact_secrets.cjs step (which runs after Stop MCP Gateway in the generated workflow) only scans files under /tmp/gh-aw and ${RUNNER_TEMP}/gh-aw for known secret values and replaces them in-place; it never touches text already …

pkg/workflow/compiler_yaml_post_agent.go:289

The exclusion glob is appended after the directory path it needs to negate, and upload-artifact/glob ordering for negation patterns is fragile — verify the generated YAML actually places !.../stderr.log after /tmp/gh-aw/mcp-logs/ in the final include list for every code path (including ARC/DinD rewriting), not just the common case.

<details><summary>Details</summary>

actions/upload-artifact (via @actions/glob) evaluates patterns in file order, so a negation pattern only takes effect i…

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@copilot Run pr-finisher skill

Done. Fixed the three CI failures from the prior run:

  • shim.cjs prettier format (lint-js job): ran prettier --write on shim.cjs
  • Test mock missing setSecret (extra_empty_commit.integration.test.cjs, git_patch_integration.test.cjs): added setSecret: vi.fn() to both mocks
  • WASM golden files (test-wasm-golden job): regenerated via make update-wasm-golden

All local tests pass. Note: the eslint-factory TypeScript 7 build error (moduleResolution=node10 removed) is pre-existing and unrelated to this PR — the CI lint-js job runs make lint-cjs (prettier only) and is not affected by it.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

@copilot PR still needs follow-up before a maintainer can investigate quickly.

Please address these unresolved review threads, newest first:

There is also a pending commit status on the current head SHA, so please wait for checks to settle after you push fixes.

After resolving the review feedback, run the pr-finisher skill.

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 7.24 AIC · ⊞ 8.3K · ◷
Comment /souschef to run again

…d log path, remove container

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

[!WARNING]
Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

...

Addressed all three review threads in the latest commit:

  1. Log collection order (stop_mcp_gateway.cjs:31): The shutdown script now runs first, then docker logs is read from the stopped (not yet removed) container, capturing all shutdown-time stderr. The container is explicitly removed afterward.

  2. AI credits detection (start_mcp_gateway.cjs:552): Gateway stderr is now written to /tmp/gh-aw/mcp-logs/stderr.log (the path already excluded from artifacts) after shutdown, so parse_mcp_gateway_log.cjs can still detect ai_credits_rate_limit_error and unknown_model_ai_credits from it.

  3. Raw stderr emission (stop_mcp_gateway.cjs:22): Removed the core.info call that printed raw stderr. The stop step now only logs the byte count, never the content.

Comment thread actions/setup/js/stop_mcp_gateway.cjs Outdated
Comment on lines +32 to +34
fs.mkdirSync(path.dirname(MCP_STDERR_LOG_PATH), { recursive: true });
fs.writeFileSync(MCP_STDERR_LOG_PATH, stderr, { mode: 0o600 });
coreApi.info(`MCP Gateway stderr written to log (${stderr.length} bytes).`);

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot do not write log to file, instead use core.debug. DO NOT WRITE THE LOG TO A FILE.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. collectGatewayStderr now calls core.debug instead of writing to a file. The fs dependency and all file-write logic have been removed.

@pelikhan

pelikhan commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

…ty_commit tests

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@copilot fix https://github.com/github/gh-aw/actions/runs/31071541151/job/92520581255?pr=50744

Fixed. Added setSecret: vi.fn() to the mockCore objects in push_signed_commits.test.cjs and extra_empty_commit.test.cjs. Both test files now pass locally (106 tests).

@pelikhan

pelikhan commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@copilot

@pelikhan pelikhan closed this Aug 6, 2026
@github-actions
github-actions Bot deleted the copilot/avoid-mcp-stderr-artifact branch August 14, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants