Repository navigation
UnityStringArgumentMatches doesn't seem to work properly for parameterized tests #779
Description
Environment
OS: Linux/Fedora
Test env: Native (on host)
Unity version: 9578ffd (HEAD of master at the time of submitting this issue)
The problem
- Unity runner executes more parameterized tests than it should when filtering with inclusive match (option -f)
- Unity runner does not execute any parameterized tests when filtering with exact/strict match (option -n)
Given
The following test file
// Contents of example_test.c
#include "unity.h"
TEST_CASE(0,0)
TEST_CASE(1,1)
TEST_CASE(10,10)
void test_first(int a, int b)
{
TEST_FAIL();
}
TEST_CASE(0,0)
TEST_CASE(2,2)
TEST_CASE(20,20)
void test_second(int a, int b)
{
TEST_FAIL();
}AND
Unity is compiled with -DUNITY_USE_COMMAND_LINE_ARGS and -DUNITY_SUPPORT_TEST_CASES
AND
generate_test_runner.rb example_test.c config.yml runner_example_test.c is used to generate test runner, where config.yml contains
:unity:
:cmdline_args: true
:use_param_tests: trueWhen (using -f NAME option)
Test executable is invoked as ./example_test -f "test_first(0,0)"
Then
Expected output:
example_test.c:8:test_first(0,0):FAIL
-----------------------
1 Tests 1 Failures 0 Ignored
FAIL
Actual output:
example_test.c:8:test_first(0,0):FAIL
example_test.c:8:test_first(10,10):FAIL
example_test.c:16:test_second(0,0):FAIL
example_test.c:16:test_second(20,20):FAIL
-----------------------
4 Tests 4 Failures 0 Ignored
FAIL
Notice that even test_second tests are executed, even though the input string was test_first(0,0).
When (using -n NAME option)
Test executable is invoked as ./example_test -n "test_first(0,0)"
Then
Expected output:
example_test.c:8:test_first(0,0):FAIL
-----------------------
1 Tests 1 Failures 0 Ignored
FAIL
Actual output:
-----------------------
0 Tests 0 Failures 0 Ignored
OK
Additional information/observations
The problem seem to be related to how the function UnityStringArgumentMatches (in unity.c on line 2412) parses the input string. In particular when the input string includes the comma character ,. The function seem to tokenize (ish) the input string using the comma character as a delimiter. I'm not sure if this is the intended behavior, but I've noticed that it causes more tests to be run when the substring pointed to by ptr1 partially matches the current test name.
The following if-test evaluates to true for other parameterized tests (not just the exact match) because the ptr1 is pointing to substring 0).
2461: /* done if complete testname match */
2462: if (IsStringInBiggerString(Unity.CurrentTestName, ptr1) == 1)
2463: {
2464: return 1;
2465: }So for an input string "test_first(0,0)", I would expect Unity to only execute on exact match (with both -f and/or -n options). However, invoking the test executable as:
./example_test -f "test_first(0,0)"
is effectively the same as invoking it with
./example_test -n "0)"
Activity
orjangj commentedon May 1, 2025
Even
-f "test_first(0,100)", which isn't even a test case, causes a test to be triggered 😅I'm not entirely sure if I've misunderstood the purpose of
-fand-noptions, but I'm currently presuming that all of this is unintended behavior. Let me know if you need more help/info.orjangj commentedon Aug 12, 2025
Any update on this?