Let a dialect own its reserved keywords, starting with DuckDB

[guizmaii]

Purpose

Implements #2786: a dialect owns its reserved keywords, in both directions, starting with DuckDB.

Consumer<CCJSqlParser> duckDb = parser -> parser.withDialect(Dialect.DUCKDB);
CCJSqlParserUtil.parse("SELECT settings, file FROM script s WHERE s.top > 1", duckDb); // accepted, rejected without the dialect
CCJSqlParserUtil.parse("SELECT a AS from, t.where FROM t", duckDb);                     // any keyword after AS or "."
CCJSqlParserUtil.parse("SELECT desc FROM t", duckDb);                                    // rejected: DESC is reserved in DuckDB
CCJSqlParserUtil.parse("SELECT 1 name", duckDb);                                         // rejected: a bare alias cannot be a keyword

Model

• KeywordPolicy#isName(word, NamePosition): whether a word is an unquoted name at a position: NAME (column, table, CTE, ...), FUNCTION, LABEL (after AS or .), BARE_ALIAS.
• Dialect#getKeywordPolicy(): null for the default rules (nothing changes without a policy).
• Dialect.DUCKDB (dollar-quoted string tags, adjacent string literals on a new line) uses DuckDbKeywordPolicy: DuckDB 1.5.5's 489 keywords with their categories, from duckdb_keywords(), and the PostgreSQL rules DuckDB applies to them. Function names only reject reserved keywords: DuckDB's grammar special-cases functions named by other keywords (COALESCE, NULLIF, IF, ...).

Grammar

• RelObjectName() rejects the names the policy rejects at that position (position from the previous token: AS/., or a following ().
• DialectUnreservedWord() accepts the keywords reserved by default that the policy accepts as names; DialectLabel() accepts any keyword after AS or ..
• A select item's alias without AS follows the BARE_ALIAS rule; other aliases follow NAME.
• Constructs led by such words step aside when the word is a name: the XOR operator (grammar and Pratt loop), INCLUDES/EXCLUDES, EXISTS, CONNECT_BY_ROOT, PRIOR, XMLSERIALIZE, window-frame BETWEEN/UNBOUNDED, CONNECT BY [NOCYCLE], with explicit lookaheads: checkGrammarAmbiguity passes.

Validation

• Java 17 Gradle check: 13,489 tests, zero failures/errors; Spotless, Checkstyle, PMD, SpotBugs and checkGrammarAmbiguity pass. All existing tests are unchanged, apart from TaggedDollarStringTest, which now treats every dialect enabling dollar-quoted string tags (PostgreSQL and DuckDB) alike.
• DuckDbKeywordPolicyTest compares, for each of DuckDB's 489 keywords, 7 name positions (column, qualified column, AS alias, bare alias, table, table alias, CTE) against what DuckDB 1.5.5 itself parses (duckdb-keyword-names.csv, generated with json_serialize_sql()): 3,391 cells agree. 32 cells are exempted, each documented: JSqlParser parses the word as dedicated syntax at that position (MAP {...}, STRUCT(...), TRY_CAST(...), COLUMNS(...), DEFAULT, WITH RECURSIVE, UPDATE ... SET, ...) or SELECT 1 day as a date unit.
• 505 production DuckDB queries parse with and without Dialect.DUCKDB.

Limits

• MySQL's SELECT modifiers (DISTINCTROW, STRAIGHT_JOIN, SQL_CACHE, ...) cannot be names under a policy: Select()'s LOOKAHEAD(3) into PlainSelect() takes them as modifiers first.
• Other dialects keep the default rules; their policies can follow the same model.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[manticore-projects]

Thank you for your work and effort, but I am not a big fan of it:

1. SQL:2016 is very broad on reserved keywords
2. It is always good to write portable SQL and quote identifiers (as well as RDBMS specific constructs)
3. I have performance concerns
4. I have ongoing maintenance concerns
5. Our current list of reserved keywords is very very short already

[guizmaii]

Thanks for looking at it @manticore-projects. To clarify the problem this addresses:

Recent PRs added dialect-specific keywords to the reserved block for every dialect: ClickHouse's SETTINGS/PREWHERE, Exasol's CSV/FILE/SCRIPT/FBV/VERIFY, ... (and ERRORS, fixed in #2781). None of these is reserved in SQL:2016, so a query using settings or file as a column name is portable SQL: it parsed with 5.3 and is rejected by 5.4. The reserved list grew from words other dialects needed, not from the standard.

So this isn't about accepting non-portable SQL, but about one dialect's keywords not leaking into the others.

If the per-dialect model is too much, a smaller fix would be to make those dialect-only words non-reserved again, behind lookaheads where their clause starts, as #2781 did for ERRORS. Would you accept that instead?

[manticore-projects]

If the per-dialect model is too much, a smaller fix would be to make those dialect-only words non-reserved again, behind lookaheads where their clause starts, as #2781 did for ERRORS. Would you accept that instead?

I would not object that at all: exotic and scarcely used features like this Exasol stuff can take longer to parse -- as long as it does not affect the performance of the regular Oracle/Postgres/MSQL/MariaDB queries.

[guizmaii]

To make the point concrete, on current master:

Statement	no dialect	Dialect.EXASOL
IMPORT INTO t FROM LOCAL CSV FILE 'f.csv'	fails	parses
EXPORT t INTO LOCAL CSV FILE 'f.csv'	fails	parses
SELECT csv, file, script, verify, fbv FROM t	fails	fails

The Exasol IMPORT/EXPORT statements are already gated behind Dialect.EXASOL (LOOKAHEAD({ Dialect.EXASOL... }) ( stm = Import() | stm = Export() )), but the keywords they introduced are reserved for every dialect. So anyone parsing PostgreSQL, DuckDB or plain SQL with a csv or file column gets rejected because of a syntax they can't even use without opting into Exasol.

That's the inconsistency I'd like to fix: keywords introduced for a dialect-gated syntax should only be reserved when that dialect is on.

[manticore-projects]

To make the point concrete, on current master:
Statement no dialect Dialect.EXASOL
IMPORT INTO t FROM LOCAL CSV FILE 'f.csv' fails parses
EXPORT t INTO LOCAL CSV FILE 'f.csv' fails parses
SELECT csv, file, script, verify, fbv FROM t fails fails

The Exasol IMPORT/EXPORT statements are already gated behind Dialect.EXASOL (LOOKAHEAD({ Dialect.EXASOL... }) ( stm = Import() | stm = Export() )), but the keywords they introduced are reserved for every dialect. So anyone parsing PostgreSQL, DuckDB or plain SQL with a csv or file column gets rejected because of a syntax they can't even use without opting into Exasol.

That's the inconsistency I'd like to fix: keywords introduced for a dialect-gated syntax should only be reserved when that dialect is on.

Well understood, I only object separate keyword lists (because it will be a nightmare to maintain).

[guizmaii]

Opened #2793 with the smaller fix you suggested: the Exasol/ClickHouse clause words become non-reserved, with no separate keyword lists, and parse performance stays the same.