Callback version of PortMidi for Windows, bad idea?

[cogled4]

I needed a callback version of PortMidi for my Windows app. I modified the library to accept a callback function when opening a midi input stream. Then, in the Windows implementation, I call this callback instead of inserting the PmEvent into the queue.

For my purposes this seems to work well, the queue stays empty and I get the PmEvent's, both short and Sysex, sent straight to my callback function. Of course this isn't portable and I'm not recommending anyone else do this :) But I did want to ask, should I anticipate any major problems with this approach? It looks like the Windows Midi callback does use EnterCriticalSection() and LeaveCriticalSection(). I need to do more research about what that entails.

Forgive me if I'm posting this question in the wrong place. I couldn't find any Portmidi forum anywhere else.

[rbdannenberg]

What do you do in your callback? The callback is not running under your application thread, so if it reads or writes anything in your application that might also be read or written by the application, then you can expect non-deterministic and probably wrong behavior, but it won't happen often, so it will be hard to detect or debug. If you use critical sections correctly, you can probably avoid wrong behavior, but it can be hard to get right and even harder to test. The safest thing you could do is probably to put incoming MIDI into a data structure to be safely processed later by the application, but that's exactly what PortMidi does already. The other problem is priority inversion: Suppose you use critical sections and your application holds the lock and some other application is running at an intermediate priority between that of the MIDI callback and the application. This unknown intermediary process can remain busy indefinitely, effectively blocking the MIDI callback that has higher priority. I don't know what priority is given to the MIDI callback or what happens if it is blocked for an unusual amount of time, and I don't think Microsoft documents this or makes any promises, which is why PortMidi takes a safer and more portable implementation strategy (it's also free from priority inversion issues).

[darkuranium]

A bit of a necro, but I can give a reason for my own use-case (albeit in Linux) --- I use PortMidi to provide USB host functionality, ultimately interfacing with a microcontroller.

I only need notes and a few other simple things, and I do not need perfect sync in my case (it's a display, of a sort).
In my case, the actual event loop simply does (pseudocode; actual code currently uses GCC intrinsics due to bugs related to C11 atomics):

if(keyPressed)
    atomic_fetch_or_explicit(&keyState[KEY_STATE_BYTE(key)], (keystate_t)1 << KEY_STATE_BIT(key), memory_order_release);
else if(keyReleased)
    atomic_fetch_and_explicit(&keyState[KEY_STATE_BYTE(key)], ~((keystate_t)1 << KEY_STATE_BIT(key)), memory_order_release);

And the protocol-handling code on the other end does atomic reads of states (atomic_load_explicit with memory_order_acquire) and sends updates via the network. Yes, it will happen that the device gets half the state (e.g. if keyState[1] & keyState[2] were just updated, and we're reading keyState[2] on the read thread), but for this use-case it's perfectly fine, as the display will only be behind by a few frames (and since I'm reading from a physical piano, it's impossible for a human to tell this "mid-update read" case from simply releasing the piano key a millisecond later).

Since the approach is wait-free, I avoid the usual priority-inversion issues --- and (in a hypothetical callback-based API) I avoid the possibility of an event buffer overflow. It would also presumably consume less memory due to the lack of a need for such a buffer.

I know this is a very niche case, and at least in my case, I have enough memory & spare performance for the buffer to not be a problem, but it's just an example of where a callback-based approach would be preferred.

I 100% understand why it's not present, but maybe there would be value in some "advanced" APIs being available, with documented warnings about their pitfalls.

For what it's worth, there is a lot of precedent for audio-related callbacks to run in arbitrary (possibly real-time) threads, or even in interrupt code, as opposed to the main thread. SDL & PortAudio immediately come to mind.