MSGS-style drum note cut causes unnatural cymbals, SFX, & more

[mrbumpy409]

I find the new drum note cutting behavior problematic for a few reasons:

Reason no. 1

The note cutoff prevents natural sound overlap with cymbal playing and many sound effects.

Here is a zip file containing two recordings and a MIDI file. The FluidSynth recording uses GeneralUser GS 2.0.1 as the General MIDI bank, since it has all of the drum kits duplicated as melodic presets on bank 120. This allows for a direct comparison between the melodic preset and percussive preset behavior using the same samples.

In the FluidSynth recording, you will hear the following musical examples, in order, played using the Standard 1 drum kit:

1. Ride cymbal playing a jazzy pattern on channel 10.
2. Ride cymbal playing a jazzy pattern on channel 1.
3. Cymbal roll crescendo on channel 10.
4. Cymbal roll crescendo on channel 1.

In the Roland SC-8820 recording, you will only hear examples 1 and 3 above.

The FluidSynth channel 10 tests result in a very unnatural cutting off of the previous sound. The channel 1 tests behave much more like real cymbals.

The Roland SC-8820 matches FluidSynth's old behavior, and the current channel 1 behavior.

This also has a major effect on the sound effects kit. With the old behavior, it was possible to have the seashore waves overlap by repeating the same note when you wanted a new wave. Now, the new wave abruptly cuts off the previous one. Same with the thunder. One can find many other types of sounds in percussion kits that are negatively affected by this change, such as the belltree or orchestra kit timpani. This is not to mention sounds in non-GM percussion kits that can now no longer overlap.

Reason no. 2

The vast majority of other MIDI devices do not self-mute re-struck notes in the percussion kit. None of the digital keyboards I've ever played do this, nor any virtual instruments with a GM-style drum kit. The Roland Sound Canvas devices don't do this (at least as emulated by the Roland SOUND Canvas VA VSTi—see the the Roland SC-8820 recording included in the above linked zip file), and no hardware or software SoundFont synth to my knowledge has ever defaulted to this behavior. The fact that Microsoft's terrible GM synth self-mutes should not be used as a model for anything.

The only exception I have found is that select GM percussion sounds are supposed to self-mute, such as hi-hats, surdos, record scratches and triangle (off the top of my head). However, that is handled per-preset and brings me to...

Reason no. 3

There is already a way to make percussion sounds self-mute built into the SoundFont standard. "Exclusive class" exists for this very purpose, and this is how GeneralUser GS makes the aforementioned GM sounds self-mute. I believe self-muting of percussion sounds should be left up to the SoundFont designer as it always has been.

I regret not finding out about this change earlier to share my thoughts and experience. At the very least, this new muting behavior should be an opt-in feature. There is nothing in the SoundFont spec—nor the long history of hardware and software SoundFont implementations—that supports this change as default behavior.

[mrbumpy409]

Here is audio comparing the behavior of the GeneralUser GS SFX kit seashore sound between the percussive and melodic versions of the preset. You can hear the melodic preset version's samples naturally overlap whereas the percussive preset version cuts off the previously-playing sample whenever a new ocean wave is triggered.

[derselbst]

At the beginning of discussion #1196 I was also skeptical, but the feedback I got afterwards as well in the PullRequest convinced me. Unfortunately I completely forgot about exclusive class, sry.

Exclusive class is the killer argument. I'm torn between reverting this change completely or introducing an opt-in setting.

Pinging @vladkosi, @defensem3ch and @fdelapena who gave positive feedback before and possibly want to share their thoughts.

[defensem3ch]

As far as I'm aware, exclusive class has nothing to do with this particular issue in terms of how MSGS renders using gm.dls. Yes there are some percussion sounds in the gm.dls drum kits that use it (hi hats, cuica, ...?) but since there is only one exclusive class per sound, it's not possible to implement MSGS-style note cuts for a sound that already uses exclusive class. Please correct me if I'm wrong, or request clarification as needed (I'm extremely busy these days but will make every attempt to reply) - this is why in my original request I implored that this be optional. It's great to be able to render MSGS MIDI files in the same style as MSGS, and it would be an utter shame to lose that functionality (again the original feedback was extremely positive for this use case), yet I understand that not every MIDI module behaves the same and I didn't originally intend for this feature to be something that was applied globally across all soundfonts.

[mrbumpy409]

Since the code already exists for emulating MSGS behavior and there are those who would find it useful, I would vote for opt-in via a setting.

[mrbumpy409]

As far as I'm aware, exclusive class has nothing to do with this particular issue in terms of how MSGS renders using gm.dls. Yes there are some percussion sounds in the gm.dls drum kits that use it (hi hats, cuica, ...?) but since there is only one exclusive class per sound, it's not possible to implement MSGS-style note cuts for a sound that already uses exclusive class.

Sounds that use exclusive class already function in the manner you describe. Play an open hi-hat and then play it again at velocity=1 and it will self-mute. The only difference might be the smooth cutoff ramping that FluidSynth applies in this situation, but on Sound Blaster hardware, this cutoff is quite abrupt. You can have up to 127 different exclusive classes, as I recall—enough for each MIDI key to uniquely cut itself off. Basically, sample assignments that share the same exclusive class number will cut eachother off but also cut themselves off.

Regardless, having the MSGS behavior as an option would certainly be easier than editing a SoundFont's drum kits in the above manner.

[vladkosi]

I'm not a MIDI expert, I've simply compared the audio. In the example I used, the SC-55 is cutting the hi-hat sound. It's subtle, but audible.

e1m1.zip

The SC-55 recording is from https://sc55.duke4.net/games.php#doom

[mrbumpy409]

Yes, that is because the SC-55 uses the equivalent of "exclusive class" so that the three hi-hat samples mute eachother and themselves. Most GM SoundFonts are set up the same way (e.g., GeneralUser GS).

[vladkosi]

Even with GeneralUser GS soundfont, the hi-hat isn't cut in FluidSynth 2.3.7.

fluidsynth -R 0 -C 0 -r 44100 -F output.wav GeneralUser-GS.sf2 e1m1.mid

e1m1-generaluser.zip

[mrbumpy409]

Thank you for sharing this MIDI. Analyzing it, I was able to find out what is happening here:

1. It is not hi-hat but crash cymbal (both crash cymbals 1 and 2) that are being used in this section of the MIDI file.
2. The MIDI file is essentially "choking" the crash cymbals by playing a quiet cymbal note after the loud crash. The expectation is that doing this should cause the cymbal sound to cut off.
3. Analyzing the Roland SC-55 ROM SoundFont conversion, I was able to ascertain that the SC-55 indeed does use exclusive class on the crash cymbals, which is a behavior I had not previously been aware of.

Now I have to consider if I want to emulate this behavior in GeneralUser GS. If I do, it will make cymbal rolls sound worse, but will be more compatible with MIDI files written for the Roland SC-55. I will play around with this and post back.

[mrbumpy409]

@vladkosi, Here is a beta version of GeneralUser GS featuring the following changes:

• Added exclusive class (self-muting) behavior to the following percussion kit sounds to better match Roland SC-55 behavior:
  • Crash Cymbals 1 and 2 – You can now "choke" a ringing cymbal by playing the cymbal again at a very low velocity, as is done in the famous level 1 tune from Doom.
  • Hi / Low Whistle
  • Guiro Short / Long
  • Cuica Mute / Open

Please test and let me know if the percussion now behaves as you expect when using FluidSynth 2.3.7.

[vladkosi]

Thanks for the clarification @mrbumpy409. I've tested the beta version with FluidSynth 2.3.7, and the crash cymbals are cut as expected.

I've found out the Roland SC-55 soundfont by EmperorGrieferus from my original example also uses exclusive class on the crash cymbals. FluidSynth 2.3.7 is cutting the sound, but not fast enough compared to version 2.4.2 and the SC-55.

"2.3.7-no-exclusive-class.flac" is using edited version of the soundfont with exclusive class removed.

e1m1.zip

[mrbumpy409]

FluidSynth 2.3.7 is cutting the sound, but not fast enough compared to version 2.4.2 and the SC-55.

That is because FluidSynth "softens" the exclusive class cutoff time to make for more natural cymbal and hi-hat cuts. It does so by reducing the volume and modulation envelope release times (by -200 in the code, but I'm not sure what time or percentage this translates to).

If I understand the code correctly, this means that the quickness of the cut is determined largely by the length of the volume envelope release. Often, the release in percussion sounds is set very long or even to maximum time in the SoundFont for samples that are not supposed to decay. This can lead to FluidSynth's exclusive class cutoff being far too long. In the GeneralUser GS beta, for example, playing the long whistle quickly followed by the short whistle results is the long whistle continuing to be heard. Though it does fade out, the fade is way too long. Since the cymbals and open hi-hat have shorter release times, they get cut at a more natural speed, but I feel the cut could still be shorter.

I might play with the code and see if I can find a faster value that still sounds natural in most cases. The code in question can be found in /src/synth/fluid_voice.c lines 1401–1408.