Skip to content

tests: build the harness on os.exec so make test compiles on current V - #218

Merged
JalonSolov merged 2 commits into
vlang:mainfrom
metif12:fix/test-harness-exec
Oct 6, 2026
Merged

JalonSolov merged 2 commits into
vlang:mainfrom
metif12:fix/test-harness-exec

Conversation

@metif12

@metif12 metif12 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

tests: build the harness on os.exec so make test compiles on current V

make test is v -Wimpure-v -W test ., and on current V it could not compile 31 of
the 49 test files:

common/testing/testrig.v:211: error: function `execute` has been deprecated;
    use os.exec with an argument array; command strings can allow shell injection

-W turns a deprecation into an error, so every test file that went through the
harness, and every file that called os.execute itself, was failing to build rather
than running. That is what the three Linux workflows do, and what this unblocks.

What changed

os.exec takes an argument array, so a command line has to be split somewhere. Every
command in this repository is written as a string by the tests, so this adds the one
place that does it:

split_args.v   whitespace between words, single and double quotes, and
               nothing else

Not a shell, on purpose. A backslash is an ordinary character everywhere in it,
because on this platform a backslash is nearly always part of a path and a shell's
rule of eating it would take the volume off C:\Users\me\file.txt. The shapes it
does have to handle are the ones the tests actually write, and each is pinned by a
test taken from a real call site:

--output-delimiter="|"                  comm_test.v
sh -c "exit 42"                        timeout_test.v
tr -d '\r'                             sum_test.v
C:\Users\me\coreutils\test.txt          the reason it is not a shell

An unterminated quote is not an error: the rest of the line becomes one word, so a
test that made the mistake fails on what the program said rather than on a parse in
this file.

Numbers

124 call sites, 22 files. v -Wimpure-v -W test . on Windows:

before   31 files could not be built at all
after    2

and the suite goes from 25 files failing to 17, with those 17 being real test
failures rather than build failures. The two that still do not build are fixed
elsewhere: tail's string_to_i64_test.v is #207 and sum's is #215.

What did not change

Nothing that a test asserts. The commands are the same strings, the arguments are the
same arguments, and no test was reworded; the difference is only that the split is
done in V rather than by a shell. Twenty files under src/ and the three under
common/testing/ and no other.

@metif12 metif12 mentioned this pull request Oct 4, 2026
@JalonSolov

Copy link
Copy Markdown
Contributor

Conflicts.

`make test` is `v -Wimpure-v -W test .`, and on current V it could not compile 31 of
the 49 test files:

	common/testing/testrig.v:211: error: function `execute` has been deprecated;
	    use os.exec with an argument array; command strings can allow shell injection

-W turns a deprecation into an error, so every test file that went through the
harness, and every file that called os.execute itself, was failing to build rather
than running. That is what the three Linux workflows do, and what this unblocks.

os.exec takes an argument array, so a command line has to be split somewhere. Every
command in this repository is written as a string by the tests, so this adds the one
place that does it:

	split_args.v   whitespace between words, single and double quotes, and
	               nothing else

Not a shell, on purpose. A backslash is an ordinary character everywhere in it,
because on this platform a backslash is nearly always part of a path and a shell's
rule of eating it would take the volume off `C:\Users\me\file.txt`. The shapes it
does have to handle are the ones the tests actually write, and each is pinned by a
test taken from a real call site:

	--output-delimiter="|"                  comm_test.v
	sh -c "exit 42"                        timeout_test.v
	tr -d '\r'                             sum_test.v
	C:\Users\me\coreutils\test.txt          the reason it is not a shell

An unterminated quote is not an error: the rest of the line becomes one word, so a
test that made the mistake fails on what the program said rather than on a parse in
this file.

124 call sites, 22 files. `v -Wimpure-v -W test .` on Windows:

	before   31 files could not be built at all
	after    2

and the suite goes from 25 files failing to 17, with those 17 being real test
failures rather than build failures. The two that still do not build are fixed
elsewhere: tail's string_to_i64_test.v is vlang#207 and sum's is vlang#215.

Nothing that a test asserts. The commands are the same strings, the arguments are the
same arguments, and no test was reworded; the difference is only that the split is
done in V rather than by a shell. Twenty files under src/ and the three under
common/testing/ and no other.
@metif12
metif12 force-pushed the fix/test-harness-exec branch from 43ed842 to 2eb299a Compare October 5, 2026 18:09
@JalonSolov

Copy link
Copy Markdown
Contributor

There's a warning in the CI that appears to be related to this...

build.vsh:126:3: warning: function `execute_or_panic` has been deprecated; use os.exec_or_panic with an argument array; command strings can allow shell injection

@metif12

metif12 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Found and fixed, in #228.

build.vsh:126 called execute_or_panic with a command string. It now uses
os.exec_or_panic with an argument array:

mut cmd := [@VEXE, '-Wimpure-v']
cmd << vargs
cmd << ['-o', '${curdir}/bin/${dir}', './${dir}']
os.exec_or_panic(cmd)

Verified: the whole script builds clean under -W, and ./bin gets 68 tools in
about 60s on this machine.

Two more -W failures turned up on the way, both in numfmt, which had not been
reached before because the build stopped earlier:

So v -Wimpure-v -W test . on Windows, after #228: 41 of 52 files pass, 3 fail,
8 skip. None of the three is in a file this touches — cksum and shuf read
stdin, and sort is an ordering mismatch.

@JalonSolov

Copy link
Copy Markdown
Contributor

Still has conflicts to fix.

# Conflicts:
#	src/cksum/cksum_test.v
#	src/sum/sum_test.v
#	src/wc/wc_test.v
@JalonSolov
JalonSolov merged commit d3b212e into vlang:main Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants