Added ANTLR parse tree persistent caching for procedures by manisha-deshpande · Pull Request #4547 · babelfish-for-postgresql/babelfish_extensions

manisha-deshpande · 2026-02-06T03:09:44Z

Description

[TBD]

Issues Resolved

BABEL-6037

Test Scenarios Covered

[TBD]

Use case based -
Boundary conditions -
Arbitrary inputs -
Negative test cases -
Minor version upgrade tests -
Major version upgrade tests -
Performance tests -
Tooling impact -
Client tests -

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.

manisha-deshpande · 2026-02-06T19:30:12Z

contrib/babelfishpg_tsql/sql/ownership.sql

 	create_date SYS.DATETIME NOT NULL,
 	modify_date SYS.DATETIME NOT NULL,
 	definition sys.NTEXT DEFAULT NULL,
+	antlr_parse_tree JSONB DEFAULT NULL, -- JSONB serialized ANTLR parse tree for caching


babelfish_function_ext regression test fails.
Column can be queried from postgres side, not babelfish side due to unsupported datatype.
Should the column type be TEXT instead? Would that affect storage space?

--- /home/runner/work/babelfish_extensions/babelfish_extensions/test/JDBC/./expected/babelfish_function_ext-vu-cleanup.out 2026-02-06 18:16:42.761144533 +0000 +++ /home/runner/work/babelfish_extensions/babelfish_extensions/test/JDBC/./output/babelfish_function_ext-vu-cleanup.out 2026-02-06 18:42:00.808212640 +0000 @@ -52,7 +52,7 @@ -- babelfish_function_ext entry should have been removed after dropping all these functions/procedure SELECT * FROM sys.babelfish_function_ext WHERE funcname LIKE 'babel_2877_vu_prepare%'; GO -~~START~~ -varchar#!#varchar#!#nvarchar#!#text#!#text#!#bigint#!#bigint#!#datetime#!#datetime#!#ntext -~~END~~ +~~ERROR (Code: 33557097)~~ + +~~ERROR (Message: data type jsonb is not supported yet)~~

You can ignore this error for now. We can later add a TDS sender function for JSONB (which just sends it as JSON)

Should the column type be TEXT instead? Would that affect storage space?

Yes, JSONB will allow fast lookups compared to JSON/TEXT which will required deserialization of its own. (Which will become a problem for bigger procedures).

robverschoor · 2026-03-12T15:48:56Z

contrib/babelfishpg_tsql/src/pltsql_serialize_jsonb.h

+ *
+ * This header provides the interface for serializing and deserializing
+ * ANTLR PLtsql parse trees to/from JSONB format. The serialized data is
+ * stored in the cross-session cache (babelfish_func_ext) to enable faster


it's a catalog rather than a cache - even though the catalog will be cached

robverschoor · 2026-03-12T16:51:17Z

contrib/babelfishpg_tsql/src/pltsql_deserialize.c

+
+			/* Read the value for this key */
+			tok = JsonbIteratorNext(&ctx.it, &v, false);
+


why overwrite tok ?

robverschoor

addded some comments

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

…ble and type Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

…E, TRY Statements Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

…d retrieval Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Signed-off-by: Manisha Deshpande <[email protected]>

Add cross-session ANTLR parse tree caching for T-SQL stored procedures. Serialized parse trees and datums are stored in babelfish_function_ext using nodeToString/stringToNode. On procedure execution, cached results are restored to skip ANTLR re-parsing. Cache reads validate the stored bbf_version and modify_date before deserializing, skipping stale entries from different Babelfish versions or procedures modified with the GUC disabled. Changes: - Add antlr_parse_tree_text, antlr_parse_tree_datums, antlr_parse_tree_modify_date, and antlr_parse_tree_bbf_version columns to sys.babelfish_function_ext - Store serialized parse tree and version in pltsql_store_func_default_positions - Restore and validate cached parse tree in new function pltsql_restore_func_parse_result invoked prior to ANTLR parse Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

tanscorpio7 · 2026-03-19T14:52:26Z

contrib/babelfishpg_tsql/sql/upgrades/babelfishpg_tsql--5.5.0--5.6.0.sql

+DROP TABLE IF EXISTS sys.babelfish_function_ext;
+
+CREATE TABLE sys.babelfish_function_ext (


Should be ALTER TABLE ADD COLUMN

tanscorpio7 · 2026-03-19T14:57:41Z

contrib/babelfishpg_tsql/sql/upgrades/babelfishpg_tsql--5.5.0--5.6.0.sql

+ antlr_parse_tree_text TEXT DEFAULT NULL, -- Native PG nodeToString() serialized parse tree
+ antlr_parse_tree_datums TEXT DEFAULT NULL, -- Native PG nodeToString() serialized datums array
+ antlr_parse_tree_modify_date SYS.DATETIME DEFAULT NULL,
+ antlr_parse_tree_bbf_version TEXT DEFAULT NULL,


Two colums should be sufficent, no ?

pg_node_tree - the tree

bbf_version_for_tree

Also why will storing it in binary have performnace improvments ? Wouldn't it have the reverse effect

I've let it remain as two separate columns for now as the split by delimiter function call on large concatenated strings (for large procedures) might be expensive (unnecessary overhead)

tanscorpio7 · 2026-03-19T14:58:50Z

contrib/babelfishpg_tsql/src/guc.c


 bool		pltsql_enable_create_alter_view_from_pg = false;
 bool		pltsql_enable_alter_owner_from_pg = false;
+bool		pltsql_enable_procedure_parse_cache = false;


Lets use the general term routine applies to both procedures and functions.

Add bbf_version validation, exec-time cache repopulation, and rename/alter/dependency invalidation logic and tests Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

manisha-deshpande requested a review from tanscorpio7 February 6, 2026 03:10

manisha-deshpande force-pushed the jira-babel-6037 branch from 0e2dd88 to 751d3e8 Compare February 6, 2026 18:16

manisha-deshpande commented Feb 6, 2026

View reviewed changes

manisha-deshpande requested a review from forestkeeper February 9, 2026 19:24

manisha-deshpande force-pushed the jira-babel-6037 branch from 751d3e8 to 6bb6abf Compare February 23, 2026 23:26

robverschoor reviewed Mar 12, 2026

View reviewed changes

manisha-deshpande added 8 commits March 19, 2026 10:13

Added ANTLR parse tree persistent caching for procedures

bd7e742

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Extended serialization logic for PLtsql datums, namespace item, varia…

eaeb157

…ble and type Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Refactored and optimized PLtsql serialization deserialization code

0d991c8

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Extended serialization logic for PLtsql WHILE, PRINT, EXIT, GOTO, CAS…

692b2b6

…E, TRY Statements Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Added basic test cases

3d66eee

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Introduced guc for enabling and disabling antlr parse tree caching an…

495ea56

…d retrieval Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Added serialization support for PLTSQL_STMT_DECL_TABLE

6055b7b

Signed-off-by: Manisha Deshpande <[email protected]>

manisha-deshpande force-pushed the jira-babel-6037 branch from 6316c43 to ffb844e Compare March 19, 2026 10:32

tanscorpio7 requested changes Mar 19, 2026

View reviewed changes

manisha-deshpande added 2 commits March 19, 2026 22:31

Handle additional caching scenarios

dd6e334

Add bbf_version validation, exec-time cache repopulation, and rename/alter/dependency invalidation logic and tests Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Addressed PR comments and fixed regression test failures

84d3a71

Task: BABEL-6037 Signed-off-by: Manisha Deshpande <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added ANTLR parse tree persistent caching for procedures#4547

Added ANTLR parse tree persistent caching for procedures#4547
manisha-deshpande wants to merge 10 commits intobabelfish-for-postgresql:BABEL_5_X_DEVfrom
amazon-aurora:jira-babel-6037

manisha-deshpande commented Feb 6, 2026

Uh oh!

manisha-deshpande Feb 6, 2026

Uh oh!

tanscorpio7 Feb 8, 2026

Uh oh!

robverschoor Mar 12, 2026

Uh oh!

robverschoor Mar 12, 2026

Uh oh!

robverschoor left a comment

Uh oh!

tanscorpio7 Mar 19, 2026

Uh oh!

tanscorpio7 Mar 19, 2026

Uh oh!

manisha-deshpande Mar 23, 2026

Uh oh!

tanscorpio7 Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		/* Read the value for this key */
		tok = JsonbIteratorNext(&ctx.it, &v, false);

		DROP TABLE IF EXISTS sys.babelfish_function_ext;

		CREATE TABLE sys.babelfish_function_ext (

Conversation

manisha-deshpande commented Feb 6, 2026

Description

Issues Resolved

Test Scenarios Covered

Check List

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robverschoor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants