Description

BABEL-6037: Cross-session ANTLR parse tree caching for T-SQL routines and triggers.

Routines (stored procedures, functions, triggers) with thousands of lines take excessive time on first execution in each new session due to redundant ANTLR parsing. The PLtsql function hash table that speeds up subsequent executions is session-scoped, so every new session re-parses from scratch.

This PR introduces persistent ANTLR parse tree caching. Serialized parse trees are stored in new columns in existing catalog table sys.babelfish_function_ext replicating PostgreSQL's nodeToString()/stringToNode() framework (refer: postgresql_modified_for_babelfish/src/backend/nodes/\* and babelfish_extensions/contrib/babelfishpg_tsql/src/pltsql_serialize/\*). On first execution in a new session, cached results are deserialized to skip ANTLR re-parsing, with babelfish version validation to prevent serving stale data.

Issues Resolved

BABEL-6037

Changes

Serialization Infrastructure (contrib/babelfishpg_tsql/src/pltsql_serialize/)

• gen_pltsql_node_support.pl: Code generator (modeled after PG's gen_node_support.pl) produces pltsql_nodetags.h, pltsql_outfuncs_gen.c, pltsql_readfuncs_gen.c, pltsql_equalfuncs_gen.c and corresponding switch files from annotated PLtsql node definition headers (pltsql.h, pltsql-2.h) with pg_node_attr() annotations for serialization/deserialization control. Makefile invokes the generator with src/pltsql.h src/pltsql-2.h as input files.
• Extension-owned T_PLtsql_* NodeTag values offset from PLTSQL_NODETAG_START (10000) to avoid collision with PG engine NodeTag enum, with ABI stability check (80 serializable node types + 2 nodetag-only types = 82 total, $last_nodetag_no = 10081)
• pltsql_nodeio.c: Extension-side serialization/deserialization dispatch — pltsql_nodeToString()/pltsql_stringToNode() (duplicated from PG nodeToString/StringToNode functions) handle PLtsql nodes via generated switch files, and delegate base/PG types to PG's public outNode()/nodeRead()/parseNodeString() APIs. Uses pg_strtok_init() (engine-side addition) for tokenizer initialization
• pltsql_serialize_macros.h: Replicates engine-internal WRITE_*/READ_*/COMPARE_* macros from outfuncs.c, readfuncs.c, equalfuncs.c. Redirects WRITE_NODE_FIELD → pltsql_outNode() and READ_NODE_FIELD → pltsql_nodeRead() so generated code routes through extension dispatch
• pltsql_outfuncs_stubs.c / pltsql_readfuncs_stubs.c: Custom read/write handlers for nodes requiring special serialization logic not handled by auto-generated switch files (flexible array members, runtime-only fields, string/int arrays, unions):
  • PLtsql_expr — skips runtime-only fields (plan, func, expr_simple_*)
  • PLtsql_nsitem — handles FLEXIBLE_ARRAY_MEMBER for name[]
  • PLtsql_row — handles string/int arrays (fieldnames, varnos)
  • PLtsql_recfield — skips runtime cache fields (rectupledescid, finfo)
  • PLtsql_stmt_dbcc — handles union (PLtsql_dbcc_stmt_data) keyed by dbcc_stmt_type
• pltsql_equalfuncs.c: Parse tree comparison infrastructure for the validate_antlr_parse_cache debug GUC, with custom equality stubs for PLtsql_row, PLtsql_recfield, PLtsql_nsitem, PLtsql_stmt_dbcc

Catalog Changes (sys.babelfish_function_ext)

Four new columns:

• antlr_parse_tree_text TEXT — serialized parse tree (nodeToString(pltsql_parse_result) output)
• antlr_parse_tree_datums TEXT — serialized datum array (nodeToString(pltsql_Datums) output)
• antlr_parse_tree_bbf_version TEXT — Babelfish version at serialization time
• antlr_parse_cache_enabled BIT — per-function cache control (NULL=follow session GUC (default), true=force on, false=force off/kill switch). Updated by sys.enable_antlr_parse_cache(OID, BOOLEAN)

Upgrade SQL in babelfishpg_tsql--5.5.0--5.6.0.sql adds these columns.

Cache Lifecycle (hooks.c, pl_comp.c)

• CREATE/ALTER: Compiles routine, serializes parse tree to babelfish_function_ext (pltsql_store_func_default_positions() → pltsql_fill_antlr_parse_cache_columns()). Existing flow subsequently populates the in-session PLtsql hash table for same session executions.
• 1st EXEC in new session: Session hash table miss → pltsql_restore_antlr_parse_cache_result() looks up function/column entry in bbf_func_ext catalog and deserializes retrieved cache result, validates bbf_version, populates hash table
• 2nd+ EXEC in same session: Hash table hit (no deserialization)
• EXEC after cache miss: pltsql_update_func_antlr_parse_cache() re-serializes fresh ANTLR result to catalog (also handles triggers, which are cached at first execution rather than CREATE time)
• ALTER with GUC disabled: Sets parse tree columns to NULL
• Kill switch (sys.enable_antlr_parse_cache(oid, false)): Sets antlr_parse_cache_enabled=false, NULLs cache columns, prevents future cache reads/writes for that function regardless of session GUC
• Error handling: Serialization/deserialization errors propagate via PG_RE_THROW() with elog(LOG) for diagnostics
• TDS connection check: All cache code paths (pltsql_fill_antlr_parse_cache_columns, pltsql_restore_antlr_parse_cache_result, pltsql_update_func_antlr_parse_cache) check IS_TDS_CONN() — cache is only active for TDS connections, not psql/libpq
• m/MVU (minor/Major Version Upgrade): Version mismatch rejects cache, ANTLR re-parses, cache re-populated on next exec

GUC Configuration (guc.c)

• babelfishpg_tsql.enable_antlr_parse_cache (session-level, default true) — global toggle for enabling/disabling cache reads and writes. NOT available via sp_babelfish_configure 'enable_antlr_parse_cache', 'on'/'off' for cluster-wide configuration, currently.
• sys.enable_antlr_parse_cache(routine_id OID, use_antlr_parse_cache BOOLEAN) — per-function cache control; accepts function OID (via sys.object_id('schema.func')). true=force on, false=force off (kill switch, NULLs cache columns), NULL=reset to default (follow session GUC). Requires function ownership or sysadmin role.
• sys.antlr_parse_cache_stats() — returns session-level cache statistics: cache_hits (confirmed cache reuse), cache_misses (cache enabled but empty/stale), cache_writes (successful serialization), cache_evictions (hash table eviction, kill switch, invalidation), cache_errors (serialization/deserialization failures). Counters reset per backend session.
• babelfishpg_tsql.validate_antlr_parse_cache (session-level, default false) — debug GUC that compares cached deserialized trees against freshly ANTLR-compiled trees at CREATE/ALTER and EXEC time, logs PASS/FAIL with function name in postgres logfile.

1. Session GUC
SELECT set_config('babelfishpg_tsql.enable_antlr_parse_cache', 'on', false);
GO

2. Per-function cache control across sessions (OID-based)
-- Enable for specific function
SELECT sys.enable_antlr_parse_cache(sys.object_id('dbo.my_proc'), true);
GO
-- Disable (kill switch — NULLs cache columns, overrides session GUC)
SELECT sys.enable_antlr_parse_cache(sys.object_id('dbo.my_proc'), false);
GO
-- Reset to default (follow session GUC)
SELECT sys.enable_antlr_parse_cache(sys.object_id('dbo.my_proc'), NULL);
GO

3. Cache statistics
SELECT * FROM sys.antlr_parse_cache_stats();
GO
cache_hits  |  cache_misses  |  cache_writes  |  cache_evictions  |  cache_errors

4. Validation GUC
SELECT set_config('babelfishpg_tsql.validate_antlr_parse_cache', 'on', false);
GO

Node Allocation (pltsql.h, pltsql-2.h, pl_gram.c, tsqlIface.cpp)

• Added NodeTag type as the first field to all PLtsql statement and datum structs in pltsql.h and pltsql-2.h. This is required by PG's makeNode()/nodeTag() infrastructure which the serialization framework depends on. The existing cmd_type/dtype fields could not be reused because not all structs have them.
• Added pg_node_attr() annotations directly to pltsql.h and pltsql-2.h for serialization control:
  • read_as() / write_as(): Type conversion for serialization (e.g., read_as(PLtsql_expr*) for PLtsql_expr*[] arrays)
  • array_size(): Array length specification for pointer fields (e.g., array_size(nfields) for PLtsql_row.fieldnames)
  • equal_ignore: Skip field in equality comparison (e.g., lineno fields, runtime-only fields)
  • read_write_ignore: Skip field in serialization/deserialization (e.g., runtime caches, function pointers)
  • copy_as(): Custom copy behavior for special fields
• Consolidated serialization annotations into main headers (eliminated separate pltsql_serializable_1.h and pltsql_serializable_2.h files)
• Named previously anonymous typedef structs in pltsql-2.h: PLtsql_stmt_print, PLtsql_stmt_kill, PLtsql_stmt_init, tsql_exec_param
• Changed int32_t → int32 in PLtsql_stmt_goto.target_pc and PLtsql_stmt_save_ctx.target_pc for PostgreSQL convention consistency (both types supported in @scalar_types)
• InlineCodeBlockArgs and PLtsql_function added to @nodetag_only list in Perl script — receive NodeTags but no serialization functions (not needed for cache)
• All PLtsql statement/datum struct palloc0() calls replaced with makeNode() in pl_gram.c and tsqlIface.cpp to set proper NodeTag values
• tsqlIface.cpp: Fixed type confusion in exitExecute_body_batch — system procedures (sp_executesql etc.) return PLtsql_stmt_exec_sp, not PLtsql_stmt_exec; added cmd_type check before casting

PLtsql_function Struct Changes (pltsql.h)

• bbf_ext_xmin / bbf_ext_tid: Track the babelfish_function_ext tuple identity at compile time for cross-session cache invalidation (detects concurrent ALTER)
• from_cache: Flag indicating the function was loaded from persistent cache (skips re-serialization at EXEC time to avoid redundant catalog writes)

Documentation

• serialization-annotations-guide.md: Comprehensive reference for all pg_node_attr() annotations used in pltsql.h and pltsql-2.h, explaining the purpose and usage of each attribute type (read_as, write_as, array_size, equal_ignore, read_write_ignore, copy_as) with examples from the codebase. Includes rationale for design decisions (e.g., int32 vs int32_t type choice, lineno field handling, nodetag-only structs).

Performance Results

• Customer procedure (~1300 lines): 2031ms → 15ms first-execution time (99% reduction)
• Stress test (1000 connections): Average first-execution time ~4314ms → ~175ms

Test Scenarios Covered

• Use case based - (BABEL-6037-vu-prepare.sql, BABEL-6037-vu-verify.mix)

  • Test 1: Simple procedure without arguments — basic serialization
  • Test 2: Procedure with arguments — parameter handling and variable serialization
  • Test 3: Procedure with various statement types (WHILE, GOTO, TRY-CATCH, CASE, BREAK, CONTINUE, PRINT, RETURN)
  • Test 4: Procedure containing EXEC sp_executesql — serialization of system procedure call nodes
  • Test 5: Complex procedure — nested blocks, IF/ELSE, UNPIVOT, temp tables, RETURN value
  • Test 11: Procedure with single OUT parameter (out_param_varno re-derivation)
  • Test 12: Procedure with multiple OUT parameters (PLtsql_row datum preservation)
  • Test 13: Multi-statement table-valued function (MSTVF out_param_varno)
  • Test 13b: Inline table-valued function (ITVF itvf_query serialization)
  • Test 22: Parse cache validation GUC — cached vs ANTLR tree comparison at CREATE and EXEC
  • Test 29: DBCC CHECKIDENT procedure — custom_read_write serialization of union-based node
  • Test 30: Trigger function — cached at first execution (not CREATE time)

• Boundary conditions -

  • Cache lifecycle:
    • Test 6: Same-session CREATE → EXEC → ALTER → DROP lifecycle (samesession_proc)
    • Test 7: Cross-session procedure — EXEC, ALTER, DROP in new session (oldsession_proc)
    • Test 8: Rename procedure with GUC off, then EXEC with GUC on — cache invalidation
    • Test 9: ALTER with GUC off (cache NULLed), then EXEC with GUC on (cache re-populated)
    • Test 10: Altered dependency — drop/recreate table used by cached procedure
    • Test 23: Cache populated at EXEC time (created with GUC off, executed with GUC on)
  • Per-function cache control:
    • Test 14: Per-function cache enable/disable with OID (sys.object_id('dbo.func'))
    • Test 15: Mid-session per-func toggle (true→false→true) with hash table invalidation
    • Test 16: Default behavior (antlr_parse_cache_enabled = NULL, follows session GUC)
    • Test 17: NULL reset — explicit NULL via API, verify follows GUC
    • Test 17b: Per-func true — force cache ON even when session GUC OFF
    • Test 17c: Kill switch — per-func false blocks caching when session GUC ON
    • Test 18: Invalid OID handling, NULL OID, ownership check (non-owner denied)
    • Test 19: ALTER preserves per-function cache flag
    • Test 20: DROP removes per-function cache flag (babelfish_function_ext row gone)
  • Cache statistics:
    • sys.antlr_parse_cache_stats() calls throughout verify to track hits, misses, writes, evictions, errors progression across all test scenarios
  • Configuration:
    • Test 28: Version mismatch — cached tree with higher bbf_version, re-parses and re-caches

• Arbitrary inputs -

  • Test 24: Overloaded procs in different schemas — independent caching and per-function GUC with full signature
  • Test 25: Nested EXEC — outer cached proc calls inner cached proc, both cached independently

• Negative test cases -

  • Test 18: Invalid OID handling (non-existent OID, NULL OID)
  • Test 18b: Ownership check — non-owner denied
  • Test 24: Updating GUC with NULL OID value fails
  • Test 26: Altered/renamed dependency — ALTER TABLE, sp_rename column, sp_rename table
  • Test 27: Corrupt cache deserialization — error propagated via PG_RE_THROW, not silently swallowed

• Minor version upgrade tests -

  • Expected output updates for babelfish_function_ext schema change (BABEL-2877 upgrade cleanup files)

• Major version upgrade tests -

• Performance tests -

• Tooling impact -

• Client tests -

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.