Skip to content

AST tests do not correctly report errors #506

Open

Description

@Mai-Lapyst

It seems that the ast checks all are counted as passed, even if one tries to write an failing check on purpose. It seems that when xmllint doesnt find what one tries to search via --xpath, it simply prints out XPath set is empty to stderr, but exits with a non-zero status.

Example:

$ ./tester --ast ast_checks/while_condition.d | xmllint --xpath "//functionDeclaration[name = 'z']" -
XPath set is empty

xmllint version:

$ xmllint --version
xmllint: using libxml version 21105-GITv2.11.5
   compiled with: Threads Tree Output Push Reader Patterns Writer SAXv1 HTTP DTDValid HTML C14N Catalog XPath XPointer XInclude Iconv ICU ISO8859X Unicode Regexps Automata Schemas Schematron Modules Debug Zlib Lzma

Activity

  1. WebFreak001 commented on Oct 18, 2023

    @WebFreak001
    Member

    Exiting with a non-zero status seems to be correct when it's not found? Anyway xmllint is not a utility we make, so for issues with that you would need to check there.

    The ci script also works properly, if you add paths that aren't found to the test txt files, the ci script properly aborts, optionally also printing the XML if you run it with environment variable VERBOSE=1.

  2. Mai-Lapyst commented on Oct 18, 2023

    @Mai-Lapyst
    ContributorAuthor

    It was a long time, atleast until like 11 months ago where this was removed: https://gitlab.gnome.org/GNOME/libxml2/-/commit/e85f9b98a5389c69167176ae6600091e719ec38f

    Which aparently was a long standing decision, since --xpath isnt intended to use for scripting: https://gitlab.gnome.org/GNOME/libxml2/-/issues/180

    For me the ci script dosnt works at all and also dosnt aborts; atleast when using a newer version of xmllint.

  3. WebFreak001 commented on Oct 29, 2023

    @WebFreak001
    Member

    for now you can work around this through this patch:

    diff --git a/test/run_tests.sh b/test/run_tests.sh
    index 1e267d2..c2d6609 100755
    --- a/test/run_tests.sh
    +++ b/test/run_tests.sh
    @@ -88,12 +88,12 @@ if [[ ${BUILDKITE:-} != "true" ]]; then
                                    expectParseFailure=1
                            elif [[ "$line" =~ ^# ]]; then
                                    true # comment line
    -                       elif echo "$AST" | xmllint --xpath "${line}" - 2>/dev/null > /dev/null; then
    -                               ((currentPasses=currentPasses+1))
    -                       else
    +                       elif echo "$AST" | xmllint --xpath "${line}" - 2>&1 | grep 'XPath set is empty' >/dev/null; then
                                    echo
                                    echo -e "    ${RED}Check on line $lineCount of $queryFile failed.${NORMAL}"
                                    ((currentFailures=currentFailures+1))
    +                       else
    +                               ((currentPasses=currentPasses+1))
                            fi
                            ((lineCount=lineCount+1))
                    done < "$queryFile"

    however we will need to find a permanent solution, although it really sucks they removed this feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions