Skip to content

UnityStringArgumentMatches doesn't seem to work properly for parameterized tests #779

Open

Description

@orjangj

Environment

OS: Linux/Fedora
Test env: Native (on host)
Unity version: 9578ffd (HEAD of master at the time of submitting this issue)

The problem

  1. Unity runner executes more parameterized tests than it should when filtering with inclusive match (option -f)
  2. Unity runner does not execute any parameterized tests when filtering with exact/strict match (option -n)

Given

The following test file

// Contents of example_test.c

#include "unity.h"

TEST_CASE(0,0)
TEST_CASE(1,1)
TEST_CASE(10,10)
void test_first(int a, int b)
{
    TEST_FAIL();
}

TEST_CASE(0,0)
TEST_CASE(2,2)
TEST_CASE(20,20)
void test_second(int a, int b)
{
    TEST_FAIL();
}

AND

Unity is compiled with -DUNITY_USE_COMMAND_LINE_ARGS and -DUNITY_SUPPORT_TEST_CASES

AND

generate_test_runner.rb example_test.c config.yml runner_example_test.c is used to generate test runner, where config.yml contains

:unity:
  :cmdline_args: true
  :use_param_tests: true

When (using -f NAME option)

Test executable is invoked as ./example_test -f "test_first(0,0)"

Then

Expected output:

example_test.c:8:test_first(0,0):FAIL

-----------------------
1 Tests 1 Failures 0 Ignored 
FAIL

Actual output:

example_test.c:8:test_first(0,0):FAIL
example_test.c:8:test_first(10,10):FAIL
example_test.c:16:test_second(0,0):FAIL
example_test.c:16:test_second(20,20):FAIL

-----------------------
4 Tests 4 Failures 0 Ignored 
FAIL

Notice that even test_second tests are executed, even though the input string was test_first(0,0).

When (using -n NAME option)

Test executable is invoked as ./example_test -n "test_first(0,0)"

Then

Expected output:

example_test.c:8:test_first(0,0):FAIL

-----------------------
1 Tests 1 Failures 0 Ignored 
FAIL

Actual output:


-----------------------
0 Tests 0 Failures 0 Ignored 
OK

Additional information/observations

The problem seem to be related to how the function UnityStringArgumentMatches (in unity.c on line 2412) parses the input string. In particular when the input string includes the comma character ,. The function seem to tokenize (ish) the input string using the comma character as a delimiter. I'm not sure if this is the intended behavior, but I've noticed that it causes more tests to be run when the substring pointed to by ptr1 partially matches the current test name.

The following if-test evaluates to true for other parameterized tests (not just the exact match) because the ptr1 is pointing to substring 0).

2461:        /* done if complete testname match */
2462:       if (IsStringInBiggerString(Unity.CurrentTestName, ptr1) == 1)
2463:       {
2464:            return 1;
2465:       }

So for an input string "test_first(0,0)", I would expect Unity to only execute on exact match (with both -f and/or -n options). However, invoking the test executable as:

./example_test -f "test_first(0,0)"

is effectively the same as invoking it with

./example_test -n "0)"

Activity

  1. orjangj commented on May 1, 2025

    @orjangj
    Author

    Even -f "test_first(0,100)", which isn't even a test case, causes a test to be triggered 😅

    example_test.c:8:test_first(0,0):FAIL
    
    -----------------------
    1 Tests 1 Failures 0 Ignored 
    FAIL
    

    I'm not entirely sure if I've misunderstood the purpose of -f and -n options, but I'm currently presuming that all of this is unintended behavior. Let me know if you need more help/info.

  2. orjangj commented on Aug 12, 2025

    @orjangj
    Author

    Any update on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions