MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 abort listeners added to [EventEmitter]

[ahmedsamir-dev]

Using the connection's emitter for undici's request causes a event emitter memory leak, for each request an event handler is attached to the abort signal and never deleted and because the max listeners by default is 10 for each event, the process shows up warning fro that.

example:

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 abort listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
    at _addListener (node:events:587:17)
    at EventEmitter.addListener (node:events:605:10)
    at addSignal (/app/server-transpiled/node_modules/undici/lib/api/abort-signal.js:35:19)

There is already unresolved issues related to the same problem:

elastic/elasticsearch-js#1741
elastic/elasticsearch-js#1733
elastic/elasticsearch-js#1716

Here is a suggested Fix for it but closed unmerged
#55

[ahmedsamir-dev]

Hi tomas, we hope it gets your team attention 🙏
@delvedor

[delvedor]

Heya, I'm no longer actively workin on this project, @JoshMock is the new maintainer.

[madispuk]

Hi Josh, thanks for working on it!

[arciisine]

Just spent a moment digging into this, and it looks like its tied to the EventEmitter in the Undici connection class. I verified that by introducing this fix (9012ddc), the errors all disappear.

[JoshMock]

Thanks for taking a look @arciisine. 🖤 I'm just onboarding as the new maintainer of this library so I'll take a look at this soon. Your solution is straightforward, so I'll play with it soon to make sure it has no undesired side effects.

[ftejeria]

Hey guys, I removed the warning by adding in the TransportRequestOptions for the signal field a new AbortController().signal;, here an example client.search( { yourQuery }, { signal: new AbortController().signal, } Any problem attached to this workaround to remove the warning?

[JoshMock]

As a note while I'm working to prioritize a possible fix for this, former maintainer @delvedor mentioned on the Kibana repo that this is just a warning about a possible memory leak for behavior we know is, in fact, just a large number of concurrent requests in progress. So, it's mostly just noise, which you can configure your project to ignore based on the concurrent request limits you'd expect to see in your app.

Hello! If you are reusing the abort signal multiple times, the client is cleaning up listeners as soon as they are no longer needed, but if you are sending more than 10 concurrent requests with the same abort signal, then you will hit that "issue" again. I'm using quotes around issue because that's just a warning. Node.js has no way to know if having dozens of listeners is ok, so it's proactive and lets you know because sometimes it can mean a memory leak.

If it's expected to share the same abort signal among multiple concurrent request, you can change the max listeners limit with:

• You can increase the max listeners globally with nodejs.org/api/events.html#eventsdefaultmaxlisteners

• While you can increase the max listeners for a single abort signal with nodejs.org/api/events.html#eventssetmaxlistenersn-eventtargets

[pocesar]

@JoshMock we are seeing a steady increase in memory until the containers die out of memory, over the course of 8-10 hours, happens literally every day. I don't think it's just noise, our baseline goes from 64MB of memory to 8GB...

[JoshMock]

@JoshMock we are seeing a steady increase in memory until the containers die out of memory, over the course of 8-10 hours, happens literally every day. I don't think it's just noise, our baseline goes from 64MB of memory to 8GB...

That's good to know. And you've narrowed it down to memory being consumed by Elasticsearch client activity? Would love any traces, usage examples, versions used (Elasticsearch and client) or other relevant details.