Provide separate sinks for signals and messages

[s0me0ne-unkn0wn]

As it was discussed, a subsystem must be able to process signals and messages separately so it can prioritize signal processing if message processing takes a long time. Two separate async methods, recv_signal() and recv_msg(), should be introduced to separate the flows.

[drahnr]

What speaks against pushing the message off to a worker pool or FuturesUnordered if they tend to block? There is no requirement of having a loop over recv() and execute all futures that are consequential from that serially in the loop body. Note that this also allows a bounded implementation, not polling the messages stream until a task is received.
Or do you require the tasks to be cancellable on a view update?

Or are you talking about the overload condition where there is an excessive amount of messages?

There is quite a bit of constraints context missing - even after reading the linked discussion - to be for or against the extension.

[s0me0ne-unkn0wn]

What speaks against pushing the message off to a worker pool or FuturesUnordered if they tend to block?

That is exactly what we are doing right now, and that causes backpressure issues. Spawning a new task on every message effectively renders the message channel unbounded, as the number of spawned tasks is unbounded. If we try to limit the number of tasks spawned (awaiting for a free slot in some second-level task pool with a limited number of concurrent tasks to create a backpressure on the message channel), it also blocks signal processing.

The behavior we want is like that: Subsys1 sends messages to Subsys2. The latter spawns tasks, limiting the number of concurrent tasks somehow. If Subsys1 starts spamming Subsys2 with a large number of messages, each spawning a task from the Subsys2 side, at some point, the task pool gets full, and Subsys2 awaits for a free slot, thus stopping recving new messages. Subsys1 has to await on send. Thus, backpressure is channeled from Subsys2 to Subsys1.

In principle, it works even now but blocks signal processing completely. If tasks from the Subsys2 side are heavy, potentially executing up to 60 sec, it's no good we're skipping leaf activations and even Conclude.

That is where the idea to separate signal and message channels comes from. Hope that helps.