Skip to content

Slai Agent Runtime:Allow opt-in DNS servers for Linux test sandboxes - #66

Open
Slai.life (huong8373tt-beep) wants to merge 1 commit into
microsoft:mainfrom
huong8373tt-beep:slai-linux-dns-opt-in
Open

Slai.life (huong8373tt-beep) wants to merge 1 commit into
microsoft:mainfrom
huong8373tt-beep:slai-linux-dns-opt-in

Conversation

@huong8373tt-beep

Copy link
Copy Markdown

Problem

Some published Linux benchmark tests depend on public DNS but inherit Docker Engine's resolver configuration unchanged. In the kubernetes__kops-17519 replay, TestNvidia and many unrelated integration tests stopped before their assertions because lookups of example.com through the engine resolver (192.168.65.7) returned i/o timeout or no such host.

The candidate changes NVIDIA configuration, not DNS. The same resolver fault therefore turns an external network condition into a candidate verdict.

A local Docker control establishes that the runtime can use a reviewed explicit resolver when needed:

# Docker default resolver
nameserver 192.168.65.7

# docker run --dns 1.1.1.1 ...
nameserver 1.1.1.1
getent hosts example.com  # succeeds

Hard-coding a resolver for every task would change normal sandbox behavior. RepoLaunch currently lacks a narrow, explicit launch control for the evaluator to select a resolver only for a reviewed network-dependent task.

Change

  • Keep Docker Engine DNS behavior unchanged by default.
  • Add optional comma-separated SWE_BENCH_LIVE_DOCKER_DNS handling when a Linux task container is created.
  • Pass only non-empty, trimmed nameserver strings to Docker's dns argument.

For a reviewed task, an evaluator can opt in explicitly:

SWE_BENCH_LIVE_DOCKER_DNS=1.1.1.1,2606:4700:4700::1111

Validation

python -m pytest -q tests/test_linux_docker_dns.py
# 2 passed

python -m py_compile launch/core/platforms/linux.py tests/test_linux_docker_dns.py
# passed

git diff --check
# passed

Tests cover preserving the default Docker resolver and forwarding a whitespace/empty-entry-separated DNS list.

This is runtime plumbing only. It does not modify benchmark metadata, test patches, candidate patches, or scoring; choosing a resolver remains an explicit evaluator decision.

@huong8373tt-beep Slai.life (huong8373tt-beep) changed the title Allow opt-in DNS servers for Linux test sandboxes Slai Agent Runtime:Allow opt-in DNS servers for Linux test sandboxes Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant