Skip to content

Conversation

@blathers-crl
Copy link

@blathers-crl blathers-crl bot commented May 2, 2024

Backport 4/4 commits from #123277 on behalf of @michae2.

/cc @cockroachdb/release


See individual commits for details.

Benchmarks before and after the change:

                                                                    │ /tmp/tmp.PrzUglo6GH/bench.catch~3 │    /tmp/tmp.PrzUglo6GH/bench.catch     │
                                                                    │              sec/op               │    sec/op      vs base                 │
CatchVectorizedRuntimeError/noError-12                                                    0.4033n ± ∞ ¹   0.4224n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/expected-12                                                 40831.00n ± ∞ ¹    98.55n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/storage-12                                                  41302.00n ± ∞ ¹    99.54n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/contextCanceled-12                                          40947.00n ± ∞ ¹    99.78n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/internalWithCode-12                                          41088.0n ± ∞ ¹    474.6n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/internal-12                                                  41032.0n ± ∞ ¹    943.0n ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/runtime-12                                                    37.094µ ± ∞ ¹    1.091µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/noError-12                                         43.83µ ± ∞ ¹    43.63µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/expectedWithCode-12                              1234.24µ ± ∞ ¹    48.27µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/expectedAssertion-12                             1182.11µ ± ∞ ¹    88.12µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/internalAssertion-12                             1211.83µ ± ∞ ¹    77.58µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/internalIndexOutOfRange-12                       1227.76µ ± ∞ ¹    82.40µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/internalDivideByZero-12                          1224.45µ ± ∞ ¹    81.01µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/contextCanceled-12                               1265.30µ ± ∞ ¹    44.83µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/noError-12                                        43.78µ ± ∞ ¹    42.83µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/expectedWithCode-12                             1210.98µ ± ∞ ¹    48.41µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/expectedAssertion-12                            1187.21µ ± ∞ ¹    84.70µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/internalAssertion-12                            1224.62µ ± ∞ ¹    73.85µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/internalIndexOutOfRange-12                      1224.56µ ± ∞ ¹    81.39µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/internalDivideByZero-12                         1259.05µ ± ∞ ¹    80.63µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/contextCanceled-12                              1262.44µ ± ∞ ¹    47.13µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/noError-12                                        44.98µ ± ∞ ¹    43.53µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/expectedWithCode-12                             1231.37µ ± ∞ ¹    48.65µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/expectedAssertion-12                            1201.74µ ± ∞ ¹    97.23µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/internalAssertion-12                            1249.63µ ± ∞ ¹    80.46µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/internalIndexOutOfRange-12                      1257.86µ ± ∞ ¹    77.83µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/internalDivideByZero-12                         1251.79µ ± ∞ ¹    82.22µ ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=600/contextCanceled-12                              1271.82µ ± ∞ ¹    51.19µ ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                                                                    243.2µ          13.09µ        -94.62%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                                                    │ /tmp/tmp.PrzUglo6GH/bench.catch~3 │    /tmp/tmp.PrzUglo6GH/bench.catch     │
                                                                    │               B/op                │     B/op       vs base                 │
CatchVectorizedRuntimeError/noError-12                                                      0.000 ± ∞ ¹     0.000 ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/expected-12                                                   9834.00 ± ∞ ¹     32.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/storage-12                                                    8880.00 ± ∞ ¹     32.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/contextCanceled-12                                            8840.00 ± ∞ ¹     40.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/internalWithCode-12                                           9.586Ki ± ∞ ¹   1.511Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/internal-12                                                  10.479Ki ± ∞ ¹   2.961Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/runtime-12                                                   10.964Ki ± ∞ ¹   2.656Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/noError-12                                        81.75Ki ± ∞ ¹   81.71Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/expectedWithCode-12                              143.04Ki ± ∞ ¹   85.28Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/expectedAssertion-12                              362.7Ki ± ∞ ¹   304.5Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalAssertion-12                              304.2Ki ± ∞ ¹   245.3Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalIndexOutOfRange-12                        313.0Ki ± ∞ ¹   267.2Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalDivideByZero-12                           308.3Ki ± ∞ ¹   260.7Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/contextCanceled-12                               128.96Ki ± ∞ ¹   80.60Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/noError-12                                       81.21Ki ± ∞ ¹   81.25Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/expectedWithCode-12                             136.02Ki ± ∞ ¹   84.99Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/expectedAssertion-12                             362.0Ki ± ∞ ¹   302.9Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalAssertion-12                             291.4Ki ± ∞ ¹   244.5Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalIndexOutOfRange-12                       317.1Ki ± ∞ ¹   267.0Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalDivideByZero-12                          308.4Ki ± ∞ ¹   260.4Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/contextCanceled-12                              145.34Ki ± ∞ ¹   81.73Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/noError-12                                       81.33Ki ± ∞ ¹   81.24Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/expectedWithCode-12                             137.49Ki ± ∞ ¹   85.05Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/expectedAssertion-12                             364.7Ki ± ∞ ¹   305.8Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalAssertion-12                             318.4Ki ± ∞ ¹   244.5Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalIndexOutOfRange-12                       315.8Ki ± ∞ ¹   265.4Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalDivideByZero-12                          310.9Ki ± ∞ ¹   260.2Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/contextCanceled-12                              133.20Ki ± ∞ ¹   80.53Ki ± ∞ ¹        ~ (p=1.000 n=1) ³
geomean                                                                                               ⁴                  -61.19%               ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ need >= 4 samples to detect a difference at alpha level 0.05
⁴ summaries must be >0 to compute geomean

                                                                    │ /tmp/tmp.PrzUglo6GH/bench.catch~3 │    /tmp/tmp.PrzUglo6GH/bench.catch    │
                                                                    │             allocs/op             │  allocs/op    vs base                 │
CatchVectorizedRuntimeError/noError-12                                                      0.000 ± ∞ ¹    0.000 ± ∞ ¹        ~ (p=1.000 n=1) ²
CatchVectorizedRuntimeError/expected-12                                                    36.000 ± ∞ ¹    3.000 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/storage-12                                                     11.000 ± ∞ ¹    3.000 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/contextCanceled-12                                             11.000 ± ∞ ¹    4.000 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/internalWithCode-12                                             35.00 ± ∞ ¹    37.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/internal-12                                                     46.00 ± ∞ ¹    59.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
CatchVectorizedRuntimeError/runtime-12                                                      58.00 ± ∞ ¹    51.00 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/noError-12                                          838.0 ± ∞ ¹    838.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=12/expectedWithCode-12                                1110.0 ± ∞ ¹    950.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/expectedAssertion-12                               1.736k ± ∞ ¹   1.581k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalAssertion-12                               1.285k ± ∞ ¹   1.228k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalIndexOutOfRange-12                         1.418k ± ∞ ¹   1.425k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/internalDivideByZero-12                            1.413k ± ∞ ¹   1.411k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=12/contextCanceled-12                                  880.0 ± ∞ ¹    865.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/noError-12                                         836.0 ± ∞ ¹    836.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
SQLCatchVectorizedRuntimeError/conns=240/expectedWithCode-12                               1012.0 ± ∞ ¹    949.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/expectedAssertion-12                              1.744k ± ∞ ¹   1.572k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalAssertion-12                              1.201k ± ∞ ¹   1.228k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalIndexOutOfRange-12                        1.518k ± ∞ ¹   1.424k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/internalDivideByZero-12                           1.429k ± ∞ ¹   1.411k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=240/contextCanceled-12                                 993.0 ± ∞ ¹    865.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/noError-12                                         838.0 ± ∞ ¹    837.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/expectedWithCode-12                               1038.0 ± ∞ ¹    951.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/expectedAssertion-12                              1.782k ± ∞ ¹   1.594k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalAssertion-12                              1.471k ± ∞ ¹   1.231k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalIndexOutOfRange-12                        1.476k ± ∞ ¹   1.417k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/internalDivideByZero-12                           1.471k ± ∞ ¹   1.411k ± ∞ ¹        ~ (p=1.000 n=1) ³
SQLCatchVectorizedRuntimeError/conns=600/contextCanceled-12                                 960.0 ± ∞ ¹    867.0 ± ∞ ¹        ~ (p=1.000 n=1) ³
geomean                                                                                               ⁴                 -18.86%               ⁴
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ need >= 4 samples to detect a difference at alpha level 0.05
⁴ summaries must be >0 to compute geomean

Fixes: #123235

Release note (performance improvement): Make error handling in the vectorized execution engine much cheaper. This should help avoid bad metastable regimes perpetuated by statement timeout handling consuming all CPU time, leading to more statement timeouts.

Co-authored-by: Drew Kimball [email protected]


Release justification: fix for serious performance issue.

michae2 added 4 commits April 29, 2024 23:11
Fixes: #123235

Release note (performance improvement): Make error handling in the
vectorized execution engine much cheaper. This should help avoid bad
metastable regimes perpetuated by statement timeout handling consuming
all CPU time, leading to more statement timeouts.
@blathers-crl blathers-crl bot force-pushed the blathers/backport-release-24.1.0-rc-123277 branch from becbf38 to 879751e Compare May 2, 2024 18:27
@blathers-crl blathers-crl bot requested review from a team as code owners May 2, 2024 18:27
@blathers-crl blathers-crl bot added the blathers-backport This is a backport that Blathers created automatically. label May 2, 2024
@blathers-crl blathers-crl bot removed the request for review from a team May 2, 2024 18:27
@blathers-crl blathers-crl bot added the O-robot Originated from a bot. label May 2, 2024
@blathers-crl blathers-crl bot requested a review from michae2 May 2, 2024 18:27
@blathers-crl
Copy link
Author

blathers-crl bot commented May 2, 2024

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Backports should only be created for serious
    issues
    or test-only changes.
  • Backports should not break backwards-compatibility.
  • Backports should change as little code as possible.
  • Backports should not change on-disk formats or node communication protocols.
  • Backports should not add new functionality (except as defined
    here).
  • Backports must not add, edit, or otherwise modify cluster versions; or add version gates.
  • All backports must be reviewed by the owning areas TL and one additional
    TL. For more information as to how that review should be conducted, please consult the backport
    policy
    .
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters. State changes must be further protected such that nodes running old binaries will not be negatively impacted by the new state (with a mixed version test added).
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.
  • Your backport must be accompanied by a post to the appropriate Slack
    channel (#db-backports-point-releases or #db-backports-XX-X-release) for awareness and discussion.

Also, please add a brief release justification to the body of your PR to justify this
backport.

@blathers-crl blathers-crl bot added the backport Label PR's that are backports to older release branches label May 2, 2024
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 7 files at r1, 1 of 1 files at r2, 1 of 1 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @DrewKimball, @mgartner, @michae2, @petermattis, and @rafiss)

@michae2 michae2 removed request for a team, DrewKimball and petermattis May 2, 2024 19:08
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 7 files at r1, 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @mgartner and @michae2)

Copy link
Collaborator

@DrewKimball DrewKimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 7 of 7 files at r1, 1 of 1 files at r2, 1 of 1 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @mgartner and @michae2)

Copy link
Collaborator

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 7 of 7 files at r1, 1 of 1 files at r2.
Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @michae2)

@michae2 michae2 merged commit 0e937a6 into release-24.1.0-rc May 3, 2024
@michae2 michae2 deleted the blathers/backport-release-24.1.0-rc-123277 branch May 3, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Label PR's that are backports to older release branches blathers-backport This is a backport that Blathers created automatically. O-robot Originated from a bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants