Repository navigation
Resolve backslash escapes in StringValue.getNotExcapedValue() - #2791
Merged
manticore-projects merged 1 commit intoOct 13, 2026
Merged
manticore-projects merged 1 commit into
manticore-projects merged 1 commit into
Conversation
getNotExcapedValue() only collapsed doubled single quotes. With MySQL backslash escapes it returned 'it\'s' as it\'s and 'a\\b' as a\\b, PostgreSQL E'it\'s' kept its backslash, and "it""s" kept both quotes. Record at parse time how a literal's backslashes were read: MySQL and MariaDB backslash escapes, including the default configuration with backslash escapes enabled, PostgreSQL escape strings, or standard SQL. getNotExcapedValue() resolves the literal's own doubled quote and these escapes; PostgreSQL octal and hexadecimal escapes are decoded as UTF-8 bytes. Constructed values and other dialects keep the standard reading. Signed-off-by: minleejae <mmj9808@gmail.com>
This was referenced Oct 13, 2026
Contributor
|
Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Make
StringValue.getNotExcapedValue()return the characters a string literal denotes. Today it only collapses'', so:DEFAULT 'it\'s'yieldsit\'s.'a\\b'yieldsa\\b.E'it\'s'keeps its backslash."it""s"keeps both quotes.Current design (As-is)
Whether a backslash is an escape depends on the parser configuration: backslash escapes, or a PostgreSQL
E''string. TheStringValuenode does not record it, sogetNotExcapedValue()cannot resolve backslash escapes. It also always collapses single quotes, whatever the literal's quote character.Target design (To-be)
Add
StringValue.Escaping(STANDARD,MYSQL_BACKSLASH,POSTGRESQL_ESCAPE_STRING) with a getter, setter andwithmethod.The parser records the escaping of every literal it creates from a token:
MYSQL_BACKSLASH: no dialect, or MySQL/MariaDB, with backslash escapes enabled.POSTGRESQL_ESCAPE_STRING: PostgreSQLE''strings, or any literal with backslash escapes enabled.STANDARD: everything else.getNotExcapedValue()resolves the literal's own doubled quote and, depending on the escaping:\0 \b \n \r \t \Z;\%and\_keep their backslash; any other escaped character stands for itself.\b \f \n \r \t, octal,\x,\uand\U. Octal and hexadecimal escapes are bytes, decoded as UTF-8.It still runs in linear time.
These keep the standard reading: constructed values, dollar-quoted strings, and the other backslash dialects (BigQuery, Databricks, Snowflake).
getValue(),toString()andequals()are unchanged.Document the API in the usage guide.
Validation
\0,\Z,\%,\_andN'';E''and dollar-quoted strings, including UTF-8 byte sequences;ALTER TABLE ... DEFAULTcolumn option;spotlessApply check: 10,103 tests, 10,078 passed, 25 skipped, zero failures/errors. The grammar choice conflict check passes.getNotExcapedValue()with the bytes each server produces (HEX(...)/convert_to(...)) for 35 literals on MySQL 8.4, MySQL 8.0, MariaDB 11.4 and PostgreSQL 18.3. All are identical.This PR is independent. A literal that mixes
\'and''additionally needs the lexer fix in #2790 to be tokenized as one string.Integration validation
Applied #2788, #2789, #2790, #2791 and #2792 together on
master(166f0a27):spotlessApply check: 10,150 tests, 10,125 passed, 25 skipped, zero failures/errors.