Fix premature close of the stream when it is paused#2416
Merged
wellwelwel merged 3 commits intosidorares:masterfrom Jan 31, 2024
Merged
Fix premature close of the stream when it is paused#2416wellwelwel merged 3 commits intosidorares:masterfrom
wellwelwel merged 3 commits intosidorares:masterfrom
Conversation
Collaborator
|
Hi @jansivans, thanks! I was able to reproduce this error using your example 🙋🏻♂️ What do you you think about adding a test case for this? I tested it locally and it worked using your fix:
+ const rows4 = [];
// ...
+ const s4 = connection.query('SELECT * FROM announcements').stream();
+ for await (const row of s4) {
+ await new Promise(resolve => setTimeout(resolve, 1000));
+ rows4.push(row);
+ }
// ...
+ assert.deepEqual(rows, rows4);
|
Contributor
Author
|
@wellwelwel I have added your test to this PR. |
Collaborator
Thanks again 🙋🏻♂️ @jansivans, before merging, could you "restore" the |
Contributor
Author
|
@wellwelwel done! |
Vylpes
pushed a commit
to Vylpes/Droplet
that referenced
this pull request
Apr 10, 2024
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [mysql2](https://sidorares.github.io/node-mysql2/docs) ([source](https://github.com/sidorares/node-mysql2)) | dependencies | minor | [`3.0.1` -> `3.9.3`](https://renovatebot.com/diffs/npm/mysql2/3.0.1/3.9.3) | --- ### Release Notes <details> <summary>sidorares/node-mysql2 (mysql2)</summary> ### [`v3.9.3`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#393-2024-03-26) [Compare Source](sidorares/node-mysql2@v3.9.2...v3.9.3) ##### Bug Fixes - **security:** improve cache key formation ([#​2424](sidorares/node-mysql2#2424)) ([0d54b0c](sidorares/node-mysql2@0d54b0c)) - Fixes a potential parser cache poisoning attack vulnerability reported by Vsevolod Kokorin (Slonser) of Solidlab - update Amazon RDS SSL CA cert ([#​2131](sidorares/node-mysql2#2131)) ([d9dccfd](sidorares/node-mysql2@d9dccfd)) ### [`v3.9.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#392-2024-02-26) [Compare Source](sidorares/node-mysql2@v3.9.1...v3.9.2) ##### Bug Fixes - **stream:** premature close when it is paused ([#​2416](sidorares/node-mysql2#2416)) ([7c6bc64](sidorares/node-mysql2@7c6bc64)) - **types:** expose TypeCast types ([#​2425](sidorares/node-mysql2#2425)) ([336a7f1](sidorares/node-mysql2@336a7f1)) ### [`v3.9.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#391-2024-01-29) [Compare Source](sidorares/node-mysql2@v3.9.0...v3.9.1) ##### Bug Fixes - **types:** support encoding for string type cast ([#​2407](sidorares/node-mysql2#2407)) ([1dc2011](sidorares/node-mysql2@1dc2011)) ### [`v3.9.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#390-2024-01-26) [Compare Source](sidorares/node-mysql2@v3.8.0...v3.9.0) ##### Features - introduce typeCast for `execute` method ([#​2398](sidorares/node-mysql2#2398)) ([baaa92a](sidorares/node-mysql2@baaa92a)) ### [`v3.8.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#380-2024-01-23) [Compare Source](sidorares/node-mysql2@v3.7.1...v3.8.0) ##### Features - **perf:** cache iconv decoder ([#​2391](sidorares/node-mysql2#2391)) ([b95b3db](sidorares/node-mysql2@b95b3db)) ##### Bug Fixes - **stream:** premature close when using `for await` ([#​2389](sidorares/node-mysql2#2389)) ([af47148](sidorares/node-mysql2@af47148)) - The removeIdleTimeoutConnectionsTimer did not clean up when the … ([#​2384](sidorares/node-mysql2#2384)) ([18a44f6](sidorares/node-mysql2@18a44f6)) - **types:** add missing types to TypeCast ([#​2390](sidorares/node-mysql2#2390)) ([78ce495](sidorares/node-mysql2@78ce495)) ### [`v3.7.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#371-2024-01-17) [Compare Source](sidorares/node-mysql2@v3.7.0...v3.7.1) ##### Bug Fixes - add condition which allows code in callback to be reachable ([#​2376](sidorares/node-mysql2#2376)) ([8d5b903](sidorares/node-mysql2@8d5b903)) ### [`v3.7.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#370-2024-01-07) [Compare Source](sidorares/node-mysql2@v3.6.5...v3.7.0) ##### Features - **docs:** release documentation website ([#​2339](sidorares/node-mysql2#2339)) ([c0d77c0](sidorares/node-mysql2@c0d77c0)) ### [`v3.6.5`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#365-2023-11-22) [Compare Source](sidorares/node-mysql2@v3.6.4...v3.6.5) ##### Bug Fixes - add decodeuricomponent to parse uri encoded special characters in host, username, password and datbase keys ([#​2277](sidorares/node-mysql2#2277)) ([fe573ad](sidorares/node-mysql2@fe573ad)) ### [`v3.6.4`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#364-2023-11-21) [Compare Source](sidorares/node-mysql2@v3.6.3...v3.6.4) ##### Bug Fixes - malformed FieldPacket ([#​2280](sidorares/node-mysql2#2280)) ([8831e09](sidorares/node-mysql2@8831e09)) - move missing options to ` ConnectionOptions ` ([#​2288](sidorares/node-mysql2#2288)) ([5cd7639](sidorares/node-mysql2@5cd7639)) ### [`v3.6.3`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#363-2023-11-03) [Compare Source](sidorares/node-mysql2@v3.6.2...v3.6.3) ##### Bug Fixes - correctly pass values when used with sql-template-strings library ([#​2266](sidorares/node-mysql2#2266)) ([6444f99](sidorares/node-mysql2@6444f99)) ### [`v3.6.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#362-2023-10-15) [Compare Source](sidorares/node-mysql2@v3.6.1...v3.6.2) ##### Bug Fixes - sql-template-strings/tag compatibility ([#​2238](sidorares/node-mysql2#2238)) ([f2efe5a](sidorares/node-mysql2@f2efe5a)) ### [`v3.6.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#361-2023-09-06) [Compare Source](sidorares/node-mysql2@v3.6.0...v3.6.1) ##### Bug Fixes - EventEmitter on method signatures to use spread syntax ([#​2200](sidorares/node-mysql2#2200)) ([5d21b81](sidorares/node-mysql2@5d21b81)) ### [`v3.6.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#360-2023-08-04) [Compare Source](sidorares/node-mysql2@v3.5.2...v3.6.0) ##### Features - add conn-level `infileStreamFactory` option ([#​2159](sidorares/node-mysql2#2159)) ([5bed0f8](sidorares/node-mysql2@5bed0f8)) ### [`v3.5.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#352-2023-07-14) [Compare Source](sidorares/node-mysql2@v3.5.1...v3.5.2) ##### Bug Fixes - Update events that are propagated from pool cluster to include remove ([#​2114](sidorares/node-mysql2#2114)) ([927d209](sidorares/node-mysql2@927d209)) ### [`v3.5.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#351-2023-07-10) [Compare Source](sidorares/node-mysql2@v3.5.0...v3.5.1) ##### Bug Fixes - improvements to allow to use Bun and tls ([#​2119](sidorares/node-mysql2#2119)) ([fd44a2a](sidorares/node-mysql2@fd44a2a)) - missing `ResultSetHeader[]` to `query` and `execute` ([f649486](sidorares/node-mysql2@f649486)) ### [`v3.5.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#350-2023-07-06) [Compare Source](sidorares/node-mysql2@v3.4.5...v3.5.0) ##### Features - improved inspection of columns ([#​2112](sidorares/node-mysql2#2112)) ([69277aa](sidorares/node-mysql2@69277aa)) ### [`v3.4.5`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#345-2023-07-05) [Compare Source](sidorares/node-mysql2@v3.4.4...v3.4.5) ##### Bug Fixes - handle prepare response with actual number of parameter definition less than reported in the prepare header. Fixes [#​2052](sidorares/node-mysql2#2052) ([b658be0](sidorares/node-mysql2@b658be0)) ### [`v3.4.4`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#344-2023-07-04) [Compare Source](sidorares/node-mysql2@v3.4.3...v3.4.4) ##### Bug Fixes - add `ProcedureCallPacket` to `execute` overloads ([3566ef7](sidorares/node-mysql2@3566ef7)) - add `ProcedureCallPacket` to `query` overloads ([352c3bc](sidorares/node-mysql2@352c3bc)) - add `ProcedureCallPacket` to promise-based `execute` overloads ([8292416](sidorares/node-mysql2@8292416)) - add `ProcedureCallPacket` to promise-based `query` overloads ([0f31a41](sidorares/node-mysql2@0f31a41)) - create `ProcedureCallPacket` typings ([09ad1d2](sidorares/node-mysql2@09ad1d2)) ### [`v3.4.3`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#343-2023-06-30) [Compare Source](sidorares/node-mysql2@v3.4.2...v3.4.3) ##### Bug Fixes - remove acquireTimeout invalid option ([#​2095](sidorares/node-mysql2#2095)) ([eb311db](sidorares/node-mysql2@eb311db)) ### [`v3.4.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#342-2023-06-26) [Compare Source](sidorares/node-mysql2@v3.4.1...v3.4.2) ##### Bug Fixes - changing type files to declaration type files ([98e6f3a](sidorares/node-mysql2@98e6f3a)) ### [`v3.4.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#341-2023-06-24) [Compare Source](sidorares/node-mysql2@v3.4.0...v3.4.1) ##### Bug Fixes - `createPool` uri overload ([98623dd](sidorares/node-mysql2@98623dd)) - `PoolCluster` typings ([3902ca6](sidorares/node-mysql2@3902ca6)) - create promise-based `PoolCluster` typings ([7f38496](sidorares/node-mysql2@7f38496)) - missing `parserCache` in `promise.js` ([7f35cf5](sidorares/node-mysql2@7f35cf5)) - missing constants in `promise.js` ([4ce2c70](sidorares/node-mysql2@4ce2c70)) - missing keys for `Types` constant ([86655ec](sidorares/node-mysql2@86655ec)) - missing typings for `Charsets` constants ([01f77a0](sidorares/node-mysql2@01f77a0)) - missing typings for `CharsetToEncoding` constants ([609229a](sidorares/node-mysql2@609229a)) - missing typings for `parserCache` ([891a523](sidorares/node-mysql2@891a523)) - missing typings for `Types` constant ([04601dd](sidorares/node-mysql2@04601dd)) - rename file of typings `Charsets` constants ([51c4196](sidorares/node-mysql2@51c4196)) ### [`v3.4.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#340-2023-06-19) [Compare Source](sidorares/node-mysql2@v3.3.5...v3.4.0) ##### Features - support STATE_GTIDS session track information ([2b1520f](sidorares/node-mysql2@2b1520f)) ### [`v3.3.5`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#335-2023-06-12) [Compare Source](sidorares/node-mysql2@v3.3.4...v3.3.5) ##### Bug Fixes - `createPool` `promise` as `PromisePool` ([#​2060](sidorares/node-mysql2#2060)) ([ff3c36c](sidorares/node-mysql2@ff3c36c)) - keepAliveInitialDelay not taking effect ([#​2043](sidorares/node-mysql2#2043)) ([585911c](sidorares/node-mysql2@585911c)) ### [`v3.3.4`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#334-2023-06-11) [Compare Source](sidorares/node-mysql2@v3.3.3...v3.3.4) ##### Bug Fixes - `PromisePoolConnection` import name ([76db54a](sidorares/node-mysql2@76db54a)) - `releaseConnection` types and promise ([4aac9d6](sidorares/node-mysql2@4aac9d6)) ### [`v3.3.3`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#333-2023-05-27) [Compare Source](sidorares/node-mysql2@v3.3.2...v3.3.3) ##### Bug Fixes - add package.json to exports ([#​2026](sidorares/node-mysql2#2026)) ([09fd305](sidorares/node-mysql2@09fd305)) ### [`v3.3.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#332-2023-05-23) [Compare Source](sidorares/node-mysql2@v3.3.1...v3.3.2) ##### Bug Fixes - respect enableKeepAlive option ([#​2016](sidorares/node-mysql2#2016)) ([f465c3e](sidorares/node-mysql2@f465c3e)) ### [`v3.3.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#331-2023-05-11) [Compare Source](sidorares/node-mysql2@v3.3.0...v3.3.1) ##### Bug Fixes - LRU constructor ([#​2004](sidorares/node-mysql2#2004)) ([fd3d117](sidorares/node-mysql2@fd3d117)) - Missing types in "mysql" import ([#​1995](sidorares/node-mysql2#1995)) ([b8c79d0](sidorares/node-mysql2@b8c79d0)) ### [`v3.3.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#330-2023-05-06) [Compare Source](sidorares/node-mysql2@v3.2.4...v3.3.0) ##### Features - Added updated/new error codes gathered from MySQL 8.0 source code ([#​1990](sidorares/node-mysql2#1990)) ([85dc6e5](sidorares/node-mysql2@85dc6e5)) ### [`v3.2.4`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#324-2023-04-25) [Compare Source](sidorares/node-mysql2@v3.2.3...v3.2.4) ##### Bug Fixes - **server:** Added missing encoding argument to server-handshake ([#​1976](sidorares/node-mysql2#1976)) ([a4b6b22](sidorares/node-mysql2@a4b6b22)) ### [`v3.2.3`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#323-2023-04-16) [Compare Source](sidorares/node-mysql2@v3.2.2...v3.2.3) ##### Bug Fixes - **types:** add decimalNumbers to createConnection/createPool typings. fixes [#​1803](sidorares/node-mysql2#1803) ([#​1817](sidorares/node-mysql2#1817)) ([bb48462](sidorares/node-mysql2@bb48462)) ### [`v3.2.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#322-2023-04-16) [Compare Source](sidorares/node-mysql2@v3.2.1...v3.2.2) ##### Bug Fixes - `ConnectionOptions` conflict between `mysql` and `mysql/promise` ([#​1955](sidorares/node-mysql2#1955)) ([eca8bda](sidorares/node-mysql2@eca8bda)) ### [`v3.2.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#321-2023-04-13) [Compare Source](sidorares/node-mysql2@v3.2.0...v3.2.1) ##### Bug Fixes - Add typings for Connection.promise(). ([#​1949](sidorares/node-mysql2#1949)) ([e3ca310](sidorares/node-mysql2@e3ca310)) - PoolConnection redundancy when extending Connection interface in TypeScript ([7c62d11](sidorares/node-mysql2@7c62d11)) ### [`v3.2.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#320-2023-03-03) [Compare Source](sidorares/node-mysql2@v3.1.2...v3.2.0) ##### Features - maxVersion ssl option to tls.createSecureContext ([0c40ef9](sidorares/node-mysql2@0c40ef9)) ### [`v3.1.2`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#312-2023-02-08) [Compare Source](sidorares/node-mysql2@v3.1.1...v3.1.2) ##### Bug Fixes - update `lru-cache` reset method to clear ([114f266](sidorares/node-mysql2@114f266)) ### [`v3.1.1`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#311-2023-02-07) [Compare Source](sidorares/node-mysql2@v3.1.0...v3.1.1) ##### Bug Fixes - remove accidental log in caching_sha2\_password.js ([c1202b6](sidorares/node-mysql2@c1202b6)) ### [`v3.1.0`](https://github.com/sidorares/node-mysql2/blob/HEAD/Changelog.md#310-2023-01-30) [Compare Source](sidorares/node-mysql2@v3.0.1...v3.1.0) ##### Features - cleanup buffer/string conversions in hashing/xor helpers that were failing in Bun ([a2392e2](sidorares/node-mysql2@a2392e2)) ##### Bug Fixes - when port is pased as a string convert it to a number (Bun's net.connect does not automatically convert this) ([703ecb2](sidorares/node-mysql2@703ecb2)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4wLjAiLCJ1cGRhdGVkSW5WZXIiOiIzNy4wLjAiLCJ0YXJnZXRCcmFuY2giOiJkZXZlbG9wIn0=--> Reviewed-on: https://gitea.vylpes.xyz/RabbitLabs/Droplet/pulls/300 Co-authored-by: Renovate Bot <renovate@vylpes.com> Co-committed-by: Renovate Bot <renovate@vylpes.com>
This was referenced May 19, 2024
This was referenced Jun 1, 2024
This was referenced Jul 14, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request addresses a problem I have found with this pull request - #2389.
When using streams with a pause (introduced by a timeout or similar asynchronous operation), the stream prematurely closes, leading to an
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close.Here's a code snippet to reproduce the issue:
I have implemented a fix that handles the stream's pause state more gracefully, preventing the premature close error.
With my approach we no longer need to use
setImmediatewhich creates a risk of emittingclosebefore the stream had fully processed thenullsignal.Here is output without timeout and with timeout:
