Skip to content

Resolve backslash escapes in StringValue.getNotExcapedValue() - #2791

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
minleejae:fix/string-literal-unescape-20261013
Oct 13, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
minleejae:fix/string-literal-unescape-20261013

Conversation

@minleejae

@minleejae minleejae commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Make StringValue.getNotExcapedValue() return the characters a string literal denotes. Today it only collapses '', so:

  • A MySQL column default DEFAULT 'it\'s' yields it\'s.
  • 'a\\b' yields a\\b.
  • PostgreSQL E'it\'s' keeps its backslash.
  • The double-quoted "it""s" keeps both quotes.

Current design (As-is)

Whether a backslash is an escape depends on the parser configuration: backslash escapes, or a PostgreSQL E'' string. The StringValue node does not record it, so getNotExcapedValue() cannot resolve backslash escapes. It also always collapses single quotes, whatever the literal's quote character.

Target design (To-be)

  • Add StringValue.Escaping (STANDARD, MYSQL_BACKSLASH, POSTGRESQL_ESCAPE_STRING) with a getter, setter and with method.

  • The parser records the escaping of every literal it creates from a token:

    • MYSQL_BACKSLASH: no dialect, or MySQL/MariaDB, with backslash escapes enabled.
    • POSTGRESQL_ESCAPE_STRING: PostgreSQL E'' strings, or any literal with backslash escapes enabled.
    • STANDARD: everything else.
  • getNotExcapedValue() resolves the literal's own doubled quote and, depending on the escaping:

    • MySQL escapes: \0 \b \n \r \t \Z; \% and \_ keep their backslash; any other escaped character stands for itself.
    • PostgreSQL escapes: \b \f \n \r \t, octal, \x, \u and \U. Octal and hexadecimal escapes are bytes, decoded as UTF-8.

    It still runs in linear time.

  • These keep the standard reading: constructed values, dollar-quoted strings, and the other backslash dialects (BigQuery, Databricks, Snowflake). getValue(), toString() and equals() are unchanged.

  • Document the API in the usage guide.

Validation

  • 9 new tests cover:
    • MySQL and MariaDB escapes, including \0, \Z, \%, \_ and N'';
    • doubled quotes per quote character;
    • the default configuration with backslash escapes;
    • PostgreSQL standard, E'' and dollar-quoted strings, including UTF-8 byte sequences;
    • the other backslash dialects;
    • constructed values;
    • an ALTER TABLE ... DEFAULT column option;
    • linear time on a 1,000,000-escape literal.
  • Java 17 offline Gradle spotlessApply check: 10,103 tests, 10,078 passed, 25 skipped, zero failures/errors. The grammar choice conflict check passes.
  • Compared getNotExcapedValue() with the bytes each server produces (HEX(...) / convert_to(...)) for 35 literals on MySQL 8.4, MySQL 8.0, MariaDB 11.4 and PostgreSQL 18.3. All are identical.

This PR is independent. A literal that mixes \' and '' additionally needs the lexer fix in #2790 to be tokenized as one string.

Integration validation

Applied #2788, #2789, #2790, #2791 and #2792 together on master (166f0a27):

getNotExcapedValue() only collapsed doubled single quotes. With MySQL
backslash escapes it returned 'it\'s' as it\'s and 'a\\b' as a\\b,
PostgreSQL E'it\'s' kept its backslash, and "it""s" kept both quotes.

Record at parse time how a literal's backslashes were read: MySQL and
MariaDB backslash escapes, including the default configuration with
backslash escapes enabled, PostgreSQL escape strings, or standard SQL.
getNotExcapedValue() resolves the literal's own doubled quote and these
escapes; PostgreSQL octal and hexadecimal escapes are decoded as UTF-8
bytes. Constructed values and other dialects keep the standard reading.

Signed-off-by: minleejae <mmj9808@gmail.com>
@manticore-projects

Copy link
Copy Markdown
Contributor

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants