Skip to content

Conversation

@smartaquarius10
Copy link

@smartaquarius10 smartaquarius10 commented Nov 17, 2025

WHAT

Introduces connection pooling capability in KEDA. This allows scalers (starting with PostgreSQL) to reuse existing DB connections rather than opening new ones for every trigger evaluation. The design is generic and can be extended to other targets (e.g., Redis, SQS, MySQL) in future, if needed.

WHY

The current PostgreSQL scaler establishes a new DB connection for every trigger query, which can lead to connection exhaustion under high ScaledObject counts. Connection pooling solves this by sharing a finite set of connections across multiple scalers, reducing DB load and improving operator stability.

HOW

  1. Library used: pgxpool

  2. New package: pkg/scalers/connectionpool/

    • A generic framework to manage connection pools across all future DB-based scalers (Postgres now, Redis or SQS later).
  3. Created Pooling Manager

    • Maintains a global map of connection pools keyed by target identifier.
    • Uses thread-safe sync.Map and reference counting to handle concurrent scaler access safely.
    • sync.Map ensures safe concurrent reads/writes when many scalers access pools to fetch their connections simultaneously.
    • Pools are keyed by unique DB identifiers (<scaler>.<host>.<db>), ensuring isolation per database target.
  4. Max connections or any other property can be sent from config map and can be live reloaded

    • This can help in providing updated values to new scaled object however, existing stay as is
    • Graceful fallback to pgxpool defaults when:
      • ConfigMap is missing,
      • Invalid values are provided, or
      • ConfigMap is deleted (operator continues using defaults).
      • Discussion point for reviewers:

      Runtime live reload only benefits new pool creation; existing pools retain prior configuration.
      Considering this, should we keep live reload or simplify to “restart on config change”?

  5. At line #214 in file pkg/scalers/postgresql_scaler.go added a small if s.connection != nil { as it was creating error logs if secret passed is wrong then we doesn't create connection object as getConnection called once

    • In case of passing wrong password via secret, KEDA does not trigger a ScaledObject reconciliation. The added condition prevents repeated connection errors by skipping pool creation until credentials are corrected. Once AWS IAM is also enabled then we can call connection creation like already covered for Azure
  6. Old behavior (new DB connection per trigger) replaced with pooled base access (via kind of factory, singleton and strategy pattern):

db, err := connectionpool.GetOrCreate(poolKey, func() (ResourcePool, error) {
    return connectionpool.NewPostgresPool(ctx, connectionString, maxConns)
})

TEST CASES(on EKS)

  1. Pool creation with defaults
  2. Created 2 scaled objects to reuse same pool for same DB.
  3. Independent pools per DB.
  4. Removed last scaled object closed the connection DB
  5. Used config map to set properties
  6. Put wrong value on config map to switch scaled object to fallback on default value
  7. If put wrong secret the remove then request for release of connection from the pool.

PENDING

  • Configmap support in KEDA helm chart

Fallback is already handled so it will not break. Can we take it separately?

Checklist

Fixes #6955

Relates to 7206

@smartaquarius10 smartaquarius10 requested a review from a team as a code owner November 17, 2025 18:02
@keda-automation keda-automation requested a review from a team November 17, 2025 18:02
@snyk-io
Copy link

snyk-io bot commented Nov 17, 2025

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@github-actions
Copy link

Thank you for your contribution! 🙏

Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected.

While you are waiting, make sure to:

  • Add an entry in our changelog in alphabetical order and link related issue
  • Update the documentation, if needed
  • Add unit & e2e tests for your changes
  • GitHub checks are passing
  • Is the DCO check failing? Here is how you can fix DCO issues

Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient.

Learn more about our contribution guide.

@smartaquarius10
Copy link
Author

smartaquarius10 commented Nov 17, 2025

Hey @JorTurFer 👋
I’ve raised the PR — could you please take a look and let me know if I’ve missed anything or made any mistakes?

This is my first contribution, so I’d really appreciate your guidance and any feedback you can share.

Thanks a lot for all your help so far!

@smartaquarius10
Copy link
Author

Hey @JorTurFer ,
Hope you’re doing great!
Wanted to check if you’ve had a chance to review the PR. No rush. Just making sure it’s on your radar. Really appreciate your time and help with this! 🙏

@rickbrouwer
Copy link
Member

Regardless of the content of the PR (which I didn't look at yet), there are definitely a number of jobs that have failed. You might want to look into those.

@smartaquarius10 smartaquarius10 force-pushed the main branch 2 times, most recently from fa88420 to d938d5b Compare November 24, 2025 14:40
bhasin tanul and others added 2 commits November 24, 2025 17:00
@smartaquarius10
Copy link
Author

smartaquarius10 commented Nov 25, 2025

Hello @rickbrouwer ,
I think its resolved. Jobs are not failing any more. Correct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Does KEDA Reuse DB Connections Across ScaledObjects

2 participants