Skip to content

perf: cache feature configuration lookups on the parse hot path - #2828

Draft
fudianchn wants to merge 3 commits into
JSQLParser:masterfrom
fudianchn:perf/lexer-config-overhead
Draft

fudianchn wants to merge 3 commits into
JSQLParser:masterfrom
fudianchn:perf/lexer-config-overhead

Conversation

@fudianchn

Copy link
Copy Markdown
Contributor

AI disclosure: this change was prepared with AI coding agents.

What

Remove repeated feature-configuration lookups from the per-token and per-expression paths: typed fast paths in FeatureConfiguration, identity-keyed derived-flag caches in the token manager and SimpleCharStream, lazy allocation of the DuckDB chained-sign list, a cheap first-character rejection in isPostgreSqlJsonAccess, and a hoisted flag check in captureOpaqueStatements.

Why

Every identifier, string literal and quoted identifier paid one or two EnumMap lookups per token in its lexical action and BeginToken paid up to three more; getAsBoolean / getAsLong / getAsInt also rebuilt strings on every call. The configuration is only mutated before parsing starts (it is wired through withConfiguration / getConfiguration), so the derived answers can be cached against the wired configuration's identity.

How

  • getAsBoolean / getAsLong / getAsInt / getAsInteger return stored Boolean / Long / Integer values directly and keep the string parsing for every other stored type, including null.
  • The token manager caches the dialect and flag answers its lexical actions need, refreshed whenever input_stream.featureConfiguration is a different object; SimpleCharStream.BeginToken caches its three branch answers the same way.
  • innerSigns in PrimaryExpression is allocated only when a chained sign actually appears.
  • isPostgreSqlJsonAccess rejects images that cannot be one of the four operators with a single charAt before the equals chain; this runs after every primary expression.

Root cause

Not applicable (performance).

Testing

  • Full suite: 10,344 tests, the same 35 environment failures as master on my machine, zero new failures; the dialect-sensitive suites (MySQL comments, PostgreSQL json path operators, hash line comments) pass.
  • AST dump of performance.sql byte-identical to master (54 statements).
  • Benchmark parseSQLStatements paired against master: differences stay within run-to-run noise on my machine (medians -0.6% to +0.7% in same-JVM alternation); the change removes the lookups and allocations mechanically rather than moving the needle of this corpus.

Verification of the original issue

Not applicable (no issue).

Signed-off-by: 付典 <fudianchn@gmail.com>
The token actions query the wired FeatureConfiguration for every identifier,
string literal and quoted identifier; PrimaryExpression pays new predicate
checks after every expression. Cache the derived dialect/flag answers on the
wired configuration's identity (it is only mutated before parsing starts),
lazily allocate the DuckDB chained-sign list, reject non-operator images
before the PostgreSQL json-access equals chain, and hoist the unsupported
statements flag out of the capture loop.

Signed-off-by: 付典 <fudianchn@gmail.com>
Signed-off-by: 付典 <fudianchn@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant