Skip to content

Parse COMMENT ON operators, operator classes, text search objects and large objects - #2822

Open
minleejae wants to merge 2 commits into
JSQLParser:masterfrom
minleejae:feat/postgresql-comment-on-operators-20261014
Open

minleejae wants to merge 2 commits into
JSQLParser:masterfrom
minleejae:feat/postgresql-comment-on-operators-20261014

Conversation

@minleejae

@minleejae minleejae commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #2816, which extends CommentTarget; please review that first.

Purpose

Parse the PostgreSQL COMMENT ON targets that still fail after #2816:

  • OPERATOR, including prefix operators such as - (NONE, integer) and schema-qualified operators such as app.+;
  • OPERATOR CLASS and OPERATOR FAMILY with USING access_method;
  • LARGE OBJECT oid;
  • TEXT SEARCH CONFIGURATION, DICTIONARY, PARSER and TEMPLATE;
  • CONVERSION and TRANSFORM FOR type LANGUAGE lang.

These statements fail in every configuration, for example COMMENT ON OPERATOR - (NONE, integer) IS 'x' with Encountered: <S_IDENTIFIER> / "OPERATOR". pg_dump writes a COMMENT ON for every commented object, so a dump with commented operators, text search objects or large objects stops at these lines. With this change, every object kind in the PostgreSQL 18 COMMENT ON synopsis parses.

Current design (As-is)

  • CommentTarget.Kind has the 29 kinds from feat: support additional PostgreSQL COMMENT targets #2639 and Extend PostgreSQL COMMENT ON targets #2816. CommentTargetKind() reads the kind words, and CommentTarget() uses the kind to choose the reference shape.
  • The kinds listed above have no Kind value and no grammar alternative.
  • PostgreSqlOperatorReference() reads [schema.]op for exclusion constraints (WITH OPERATOR(...)) and ALTER EXTENSION ... OPERATOR. It accepts 23 single tokens. JSON operators such as ->> or #>, ^, and operators that the lexer splits into several tokens, such as <<= or -|-, fail.

Target design (To-be)

  • Model: new CommentTarget.Kind values, appended at the end of the enum, with typed fields:
    • OPERATOR:
      • getOperator() is the ExclusionOperator returned by the shared operator reference (getSchemaName(), operator symbol in getName());
      • getLeftOperandType() and getRightOperandType() are ColDataTypes, and null stands for NONE.
    • OPERATOR_CLASS and OPERATOR_FAMILY: getName() and getAccessMethod(). Kind.isOwnedByAccessMethod() identifies them, as isOwnedByRelation() does for constraints.
    • LARGE_OBJECT: getLargeObjectOid() (Long). The grammar accepts an unsigned 32-bit integer.
    • TRANSFORM: getTransformType() and getLanguage().
    • TEXT_SEARCH_CONFIGURATION, TEXT_SEARCH_DICTIONARY, TEXT_SEARCH_PARSER, TEXT_SEARCH_TEMPLATE and CONVERSION: a possibly schema-qualified getName(). Each text search object type has its own kind, as in ExtensionObject.Kind.
    • appendTo() renders every kind. Rendering of the existing kinds does not change.
  • Grammar:
    • CommentTargetKind() reads the new kind words with isKeywordAhead()/ContextualKeyword() and the existing TEXT, SEARCH and PARSER tokens. There are no new tokens, and the words remain identifiers. OPERATOR class.+ is still an operator in schema class.
    • CommentTarget() has one alternative per new shape. The operand types reuse ExtensionOperatorType(). At least one operand type is required, as in PostgreSQL's oper_argtypes.
    • PostgreSqlOperatorReference() now reads the operator through PostgreSqlOperatorName():
      • it accepts the arithmetic, comparison, bitwise, string, pattern, JSON, array, range and geometric operator tokens;
      • it joins parts that touch each other, so <<=, -|-, |>>, @-@ or ~>=~ are one operator;
      • whitespace or a comment ends the operator, as in PostgreSQL, so < = and + - are rejected.
    • Exclusion constraints and ALTER EXTENSION share this production and accept the same operators.
    • No choice conflicts.
  • Not reached: ## without Dialect.POSTGRESQL, because the lexer reads it as an identifier, and catalog-qualified operators (db.schema.op).
  • Traversal: none of the new kinds name a relation. Table discovery reports nothing for them, as for routines and casts.
  • Validation: one Feature.commentOn... capability per new kind (10), all enabled in the PostgreSQL feature set.
  • Compatibility: the public API only gains additions. Exclusion constraints and ALTER EXTENSION no longer accept an operator with inner whitespace, such as OPERATOR(pg_catalog.< >). PostgreSQL reads it as two operators and rejects it.

Validation

  • 191 new test cases. CommentTargetTest goes from 240 to 417, CommentValidatorTest from 32 to 42, and the exclusion operator and type DDL tests gain 2 each. They cover:
    • every new kind: parse, then toString and deparser, then reparse, with Dialect.POSTGRESQL and without a dialect;
    • AST fields:
      • 74 operator spellings, plain and schema-qualified, with their operand types;
      • NONE on either side;
      • access methods;
      • OIDs from 0 to 4294967295;
      • transform types and languages;
      • quoted, dotted and Unicode names of text search objects and conversions;
    • 42 rejected forms, for example (NONE, NONE), (integer), < =, + -, =>, OPERATOR(pg_catalog.+), OPERATOR CLASS ops without USING, LARGE OBJECT -1, LARGE OBJECT 4294967296 and TRANSFORM int LANGUAGE sql;
    • kind words used as identifiers;
    • table discovery, the kind predicates and one validation capability per kind;
    • joined operators in exclusion constraints and ALTER EXTENSION.
  • Java 17 offline Gradle spotlessApply check: 10,741 tests, 10,716 passed, 25 skipped, zero failures/errors. The grammar conflict check, Checkstyle, PMD and SpotBugs pass.
  • Corpus checks:
  • MySQL 8.4 and PostgreSQL 18.3: every corpus statement this branch parses ran as written on one server and as rendered by toString() on a second, identical server, in the same order. All 205 MySQL and 332 PostgreSQL statements give the same result (success, or the same error).
  • The same check on the comment corpora (statements that create, alter and read MySQL and PostgreSQL comments): all 49 MySQL and 88 PostgreSQL statements this branch parses give the same results. Afterwards, every stored comment (MySQL table, column, index, partition, routine and event comments; PostgreSQL pg_description) is byte-identical on both servers.
  • On an extra corpus of 28 PostgreSQL comment cases, the 26 statements this branch parses give the same results on both servers, and the stored comments are identical. They include comments on a binary and a prefix operator, an operator class and family, a large object, the four text search object types and a conversion. The other two use U&'...' strings.

Integration validation

Applied this PR together with the other comment PRs (#2820, #2821, #2824, #2823), the DDL/DCL PRs #2806–#2819 and #2778 on master (bdf8ac31), with the stacked PRs on top of their bases (#2814 → #2821, #2816 → #2822, #2806 → #2824 and #2811 → #2812 → #2823).

  • Each branch merges cleanly into master. Between the branches, conflicts are limited to adjacent additions (README/index rows, usage sections, imports, test methods, visitor and feature lists, an enum line and an alias guard).
  • The integrated build passes Java 17 offline Gradle spotlessApply check: 11,877 tests, 11,852 passed, 25 skipped, zero failures/errors. The grammar choice conflict check passes.
  • On the corpus of 635 MySQL 8.4 and PostgreSQL 18 DDL/DCL statements, the integrated build parses 265 of 271 MySQL statements (master: 206) and 363 of 364 PostgreSQL statements (master: 328), the same as Share MySQL account names across CREATE USER and DEFINER #2806–Render single spaces in ALTER visibility, FORCE and row level security actions #2819 together. There are no regressions.
  • On the comment corpora and an extra set of comment edge cases, the integrated build returns a typed statement for all 80 MySQL statements (master: 67) and for 118 of 119 PostgreSQL statements (master: 76). The exception is a CREATE AGGREGATE setup statement. Every table, column, index, partition, routine, event, user and COMMENT ON comment literal is a StringValue, except COMMENT = _utf8mb4'x', which MySQL rejects.
  • On paired MySQL 8.4 and PostgreSQL 18.3 servers, all 263 MySQL and 362 PostgreSQL DDL/DCL statements and all 56 MySQL and 90 PostgreSQL comment statements rendered by the integrated build give the same results as the originals. The stored comments are identical on both servers.

Model the remaining common PostgreSQL COMMENT ON object kinds with the
existing CommentTarget: TRIGGER, POLICY and RULE within a table;
PROCEDURE, ROUTINE and AGGREGATE signatures through RoutineReference;
CAST; and ACCESS METHOD, COLLATION, DATABASE, EVENT TRIGGER, EXTENSION,
FOREIGN DATA WRAPPER, FOREIGN TABLE, [PROCEDURAL] LANGUAGE, PUBLICATION,
ROLE, SERVER, STATISTICS, SUBSCRIPTION and TABLESPACE.

The grammar reads the kind first and lets it select the reference
shape. Kind words without a token stay non-reserved. Shared routine
signatures now accept ordered-set aggregates without direct arguments,
such as mode(ORDER BY anyelement).

Signed-off-by: minleejae <mmj9808@gmail.com>
…ts and large objects

Extend CommentTarget with the PostgreSQL COMMENT ON kinds that still
failed: OPERATOR with its operand types (NONE for a missing operand),
OPERATOR CLASS and OPERATOR FAMILY with their access method, LARGE OBJECT
with its OID, TEXT SEARCH CONFIGURATION, DICTIONARY, PARSER and TEMPLATE,
CONVERSION, and TRANSFORM FOR type LANGUAGE lang.

COMMENT ON OPERATOR reuses the operator reference shared with exclusion
constraints and ALTER EXTENSION. That production now accepts the JSON,
pattern, bitwise and geometric operators, and joins operator parts that
the lexer splits but that touch each other, such as <<= or -|-. The new
kind words stay non-reserved.

Signed-off-by: minleejae <mmj9808@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant