Improve SQL query performance and metrics

[brandond]

Proposed Changes

• Reorders filter expressions and adds optimized direct key query to reduce use of subqueries and temp tables.
• Uses a prepared statement for watch polling, instead of re-evaluating it again every time.
• Uniquely names each query, and uses the query name as a label in latency metrics and log messages
• Reworks postgres queries to better align with other engines.
• Increases default max-idle-conns to 20 to avoid constant connection thrashing under even moderate load
• Adds a SQL server hostname cache to reduce delays caused by excessive DNS lookups when operating with an undersized connection pool.
• Fixes several race conditions where the reported current revision could temporarily go backwards under heavy load. No data was lost, but clients could see incorrect required revision is a future revision errors.

Unfortunately is is not possible to eliminate the filesort on name, although the query cost is still reduced by half due to elimination of the inner join.

Linked Issues:

• listSQL causes Using temporary; Using filesort on MySQL/Aurora leading to slow queries under concurrent load #651

Additional Notes

• Moving the ORDER BY name clause into the subquery (to eliminate the filesort) works some of the time, but the order of rows across an inner join is unspecified so it cannot be relied upon. In practice it does seem to work most of the time, but not when a LIMIT is in effect as the engine will limit the number of rows produced by the subquery so that later revisions are not seen, producing incorrect results.
• SELECT ... FROM kine WHERE id IN (SELECT MAX(id) FROM kine WHERE name LIKE ? ESCAPE '^' AND name >= ? GROUP BY name) is less expensive when the list of IDs is large, but more expensive when the list of IDs is small - and TERRIBLE when only a single ID is matched.
• SELECT ... FROM kine INNER JOIN (SELECT MAX(id) AS id FROM kine WHERE name LIKE ? ESCAPE '^' AND name >= ? GROUP BY name) AS mkv USING (id) is slightly more expensive when the list of IDs is large, but less expensive when the list is short.
• Queries that do not need any matching behavior - just the latest ID for a specific name - can be optimized since we know that there will be only a single ID.

This PR also enhances slow SQL warnings to show a redacted version of the query with args filled:

INFO[2026-05-20T19:31:56.421415989Z] Slow SQL: INSERT INTO kine(name, created, deleted, create_revision, prev_revision, lease, value, old_value) values('/registry/health', 1, 0, 0, 0, 0, [17]byte(...), [0]byte(...)) duration=5.868734ms name=InsertLastInsertID started="2026-05-20T19:31:56.415526682Z"
INFO[2026-05-20T19:31:58.424805073Z] Slow SQL: SELECT current_rev, compact_rev, id, name, created, deleted, create_revision, prev_revision, lease, value, old_value FROM (SELECT MAX(id) AS current_rev FROM kine) AS current, (SELECT MAX(prev_revision) AS compact_rev FROM kine WHERE name = 'compact_rev_key') AS compact, kine WHERE name LIKE '%' ESCAPE '^' AND id > 2 ORDER BY id ASC LIMIT 500 duration="644.935µs" name=AfterOldVal started="2026-05-20T19:31:58.424126367Z"

Performance Comparison

CI runs a very lightweight load test that just churns configmaps; this shows a roughly 20% improvement in average request latencies across all operation types. CPU and memory utilization are not measured.

BEFORE
[mysql-8.4] [PERF]	0.004  average mysql-8.4 request duration (seconds): create configmaps
[mysql-8.4] [PERF]	0.005  [899]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [239]  █████████████▎
[mysql-8.4] [PERF]	0.05   [  0]
--
[mysql-8.4] [PERF]	0.004  average mysql-8.4 request duration (seconds): delete configmaps
[mysql-8.4] [PERF]	0.005  [310]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [212]  ██████████████████████████████████▎
[mysql-8.4] [PERF]	0.05   [  3]  ▌
--
[mysql-8.4] [PERF]	0.002  average mysql-8.4 request duration (seconds): get configmaps
[mysql-8.4] [PERF]	0.005  [3337]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [ 140]  ██▏
[mysql-8.4] [PERF]	0.05   [   0]
--
[mysql-8.4] [PERF]	0.000  average mysql-8.4 request duration (seconds): list configmaps
[mysql-8.4] [PERF]	0.005  [1]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [0]
[mysql-8.4] [PERF]	0.05   [0]
--
[mysql-8.4] [PERF]	0.005  average mysql-8.4 request duration (seconds): update configmaps
[mysql-8.4] [PERF]	0.005  [873]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [780]  ████████████████████████████████████████████▋
[mysql-8.4] [PERF]	0.05   [  1]

AFTER
[mysql-8.4] [PERF]	0.003  average mysql-8.4 request duration (seconds): create configmaps
[mysql-8.4] [PERF]	0.005  [1009]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [ 101]  █████
[mysql-8.4] [PERF]	0.05   [   1]
--
[mysql-8.4] [PERF]	0.004  average mysql-8.4 request duration (seconds): delete configmaps
[mysql-8.4] [PERF]	0.005  [457]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [ 91]  ██████████
[mysql-8.4] [PERF]	0.05   [  1]  ▏
--
[mysql-8.4] [PERF]	0.001  average mysql-8.4 request duration (seconds): get configmaps
[mysql-8.4] [PERF]	0.005  [3462]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [  79]  █▏
[mysql-8.4] [PERF]	0.05   [   2]
--
[mysql-8.4] [PERF]	0.000  average mysql-8.4 request duration (seconds): list configmaps
[mysql-8.4] [PERF]	0.005  [0]
[mysql-8.4] [PERF]	0.025  [1]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.05   [0]
--
[mysql-8.4] [PERF]	0.004  average mysql-8.4 request duration (seconds): update configmaps
[mysql-8.4] [PERF]	0.005  [1189]  ██████████████████████████████████████████████████
[mysql-8.4] [PERF]	0.025  [ 514]  █████████████████████▋
[mysql-8.4] [PERF]	0.05   [   3]  ▏


BEFORE
[postgres-17] [PERF]	0.004  average postgres-17 request duration (seconds): create configmaps
[postgres-17] [PERF]	0.005  [842]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [288]  █████████████████▏
[postgres-17] [PERF]	0.05   [  7]  ▍
--
[postgres-17] [PERF]	0.004  average postgres-17 request duration (seconds): delete configmaps
[postgres-17] [PERF]	0.005  [324]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [201]  ███████████████████████████████
[postgres-17] [PERF]	0.05   [  3]  ▌
--
[postgres-17] [PERF]	0.003  average postgres-17 request duration (seconds): get configmaps
[postgres-17] [PERF]	0.005  [2974]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [ 518]  ████████▊
[postgres-17] [PERF]	0.05   [  20]  ▍
--
[postgres-17] [PERF]	0.000  average postgres-17 request duration (seconds): list configmaps
[postgres-17] [PERF]	0.005  [0]
[postgres-17] [PERF]	0.025  [0]
[postgres-17] [PERF]	0.05   [1]  ██████████████████████████████████████████████████
--
[postgres-17] [PERF]	0.006  average postgres-17 request duration (seconds): update configmaps
[postgres-17] [PERF]	0.005  [986]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [681]  ██████████████████████████████████▌
[postgres-17] [PERF]	0.05   [  9]  ▌

AFTER
[postgres-17] [PERF]	0.002  average postgres-17 request duration (seconds): create configmaps
[postgres-17] [PERF]	0.005  [1142]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [  36]  █▋
[postgres-17] [PERF]	0.05   [   0]
--
[postgres-17] [PERF]	0.002  average postgres-17 request duration (seconds): delete configmaps
[postgres-17] [PERF]	0.005  [500]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [ 41]  ████▏
[postgres-17] [PERF]	0.05   [  1]  ▏
--
[postgres-17] [PERF]	0.001  average postgres-17 request duration (seconds): get configmaps
[postgres-17] [PERF]	0.005  [3488]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [  96]  █▍
[postgres-17] [PERF]	0.05   [   1]
--
[postgres-17] [PERF]	0.000  average postgres-17 request duration (seconds): list configmaps
[postgres-17] [PERF]	0.005  [0]
[postgres-17] [PERF]	0.025  [1]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.05   [0]
--
[postgres-17] [PERF]	0.003  average postgres-17 request duration (seconds): update configmaps
[postgres-17] [PERF]	0.005  [1511]  ██████████████████████████████████████████████████
[postgres-17] [PERF]	0.025  [ 190]  ██████▎
[postgres-17] [PERF]	0.05   [   1]

These charts capture two runs - the first segment is Kine v0.15.0, the second is from this PR. Note in particular the drop in connection rate, query latency, and network throughput.

[manuelbuil]

lgtm! (from a non-kine expert)