Proposal to Decouple Routing Rule Definitions from Storage and Enabling Database Storage

[Peiyingy]

Currently, the routing rule definitions and their storage are tightly coupled within Trino Gateway. The system only supports either config file-based rules (using MVEL expressions) or entirely external service-based routing rules, each introducing its own set of limitations.

Config File Based Routing Rules

• Deployment Dependency: Updating rule configurations requires a new deployment.

External Service Based Routing Rules

• Infrastructure Overhead: The use of an additional service introduces extra maintenance issues.

Proposal

This proposal suggests decoupling the abstract definition of routing rules from the storage mechanism and introducing a database for routing rules storage. (This approach is also mentioned in GitHub PR #540: "Database storage for MVEL rules will be added as a follow-up"). This decoupling allows for independent evolution of both aspects:

Decoupled Abstraction for Rule Definitions

• Common Core Fields: Every rule can share a set of common fields such as rule name, description, and priority.
• Extension for Specific Rule Types:
  • MVEL Rules: Extend with fields like condition and actions.
  • Value-Based Rules: Extend with fields like username, destination_routing_group, etc.
  • Mixed Rule Types: This abstraction enables us to mix rule types - using value-based rules for simplicity and MVEL rules for more complex logic - within the same framework.

Flexible Storage Options

• Database Storage: Rules can be stored in a database to leverage benefits such as rapid updates and enhanced validity checks.
• Config File Storage Solutions: Rules can be stored in YAML files if that better fits the use case, supporting a diverse range of storage strategies.

LinkedIn Experience

At LinkedIn, we implemented Presto-Gateway with a routing rules table stored in a database. Based on that experience, this simplifies management and introduces flexibility and agility. This change will benefit our open-source community by providing a more adaptable, maintainable, and robust routing rule system.

[Chaho12]

Yes, please to submit PR on this. @oneonestar @vishalya wdyt?

[Peiyingy]

Hey @Chaho12 @oneonestar @vishalya , we’re currently prioritizing this implementation at LinkedIn, see the comment #772 (comment) . I’ve assigned the issue to myself, and we’ll be leading the development and opening a PR for review soon.

[mosabua]

That sounds great @Peiyingy .. just fyi we had a draft implementation for routing rules storage as a PR in the past but it needs rebase and further clean up. Feel free to use it as a base or ignore and work out your own approach.

The old branch is preserved in the PR #772