voice with high filter Q killed too early, causing audible click

[mrbumpy409]

FluidSynth version

Affects 2.3 (and presumably older) and 2.4 versions of FluidSynth including current master and iir-tests branches.

Describe the bug

By default FluidSynth cuts off a voice after the volume envelope release has reached -96 dB (as I recall... I'm going off of memory here). Normally the volume envelope being at -96 dB would be an inaudible cutoff, but when high filter Q is used, the filter hasn't stopped "ringing" at this point, and so the still audible sound is abruptly cut off, resulting in a click.

Steps to reproduce

1. Download and extract the following zip file:
   high_Q_note_cutoff_test.zip
2. Load the .sf2 file into FluidSynth and play the .mid file. I have also included an audio file fluidsynth_iir-tests.flac showing the result of this test using FluidSynth's current iir-tests branch.

Current behavior

The voice cuts off while the filter is still resonating, causing an audible click. You can see this quite clearly by viewing the waveform in Audacity:

Expected behavior

The voice shouldn't be cut off until the filter output has also reached -96 dB.

Additional context

Related to bug #1415.

[spessasus]

This is strange. It seems that the filter is not being affected by the final gain amplifier and instead is being sent directly to the final output.

According to Section 9.1.8 of the sf2 specification:

The gain should come after the filtered output, so the ringing should ramp to -96dB too. This happens when playing the Whistlewind in Polyphone.

[derselbst]

Thanks again for the report. I didn't found time to look into this, but here are two thoughts:

By default FluidSynth cuts off a voice after the volume envelope release has reached -96 dB (as I recall... I'm going off of memory here)

We wanted to in #319, but it didn't happen. On the other hand, there is that logic around "noise floor" that might explain the effect observed here:

fluidsynth/src/rvoice/fluid_rvoice.c

Line 52 in 168183c

fluid_real_t amplitude_that_reaches_noise_floor;

It seems that the filter is not being affected by the final gain amplifier and instead is being sent directly to the final output.

Yes, because the SF spec asks for a gain compensation dependent on Q, which we currently do here:

fluidsynth/src/rvoice/fluid_iir_filter.c

Lines 204 to 215 in 168183c

	/* SF 2.01 page 59:
	*
	* The SoundFont specs ask for a gain reduction equal to half the
	* height of the resonance peak (Q). For example, for a 10 dB
	* resonance peak, the gain is reduced by 5 dB. This is done by
	* multiplying the total gain with sqrt(1/Q). `Sqrt' divides dB
	* by 2 (100 lin = 40 dB, 10 lin = 20 dB, 3.16 lin = 10 dB etc)
	* The gain is later factored into the 'b' coefficients
	* (numerator of the filter equation). This gain factor depends
	* only on Q, so this is the right place to calculate it.
	*/
	iir_filter->filter_gain /= FLUID_SQRT(q);

But it's amount is only half of Q's peaks.

I need to investigate how both are related to explain the observed effect here.

[spessasus]

Yes, because the SF spec asks for a gain compensation dependent on Q, which we currently do here

That's an additional gain. it doesn't mention anywhere that only the Q Gain should be applied to the filter, but according to the diagram, it shows that the filter is affected both by Q and volEnv. It honestly makes sense and doing that does not cause any clicks since the ringing gets attenuated too.

Not to mention that:

9.1.4 Final Gain Amplifier
The final gain amplifier is a multiplier on the filter output, which is controlled by an initial gain in dB. This is added to the volume envelope. Additional modulation can also be added. The gain is always specified in dB.

It does explicitly state that the final gain amplifier is the Q Gain added to the volEnv gain from the filtered input.

[derselbst]

I looked into this. The voice is being OFFed because it has reached the end of its volume env, here:

fluidsynth/src/rvoice/fluid_rvoice.c

Line 339 in d814a62

return 0;

I had a look at the unfiltered voice to make sure the amplitude becomes reasonably small just before it finishes playing. Looking at the waveform this does seem to be the case, i.e. the amplitude drops below the quantization noise floor:

When the voice finishes, its dsp amplitude has a factor of 0.000007 which corresponds to approx. 103 dB of attenuation. That should be enough.

So it does look like the abrupt cutoff is being caused by the filters resonance amplitude. To fix this, I'll try to feed the filter with a bunch of zero samples to drain its state.

[derselbst]

I made a quick and dirty implementation of my previous idea by just keep feeding the filter with zero samples forever after the voice has finished. The result looks like this:

On the bottom: the previous cutoff behavior, on top the new "filter-drained" behavior. Take a listen yourself:

1427_high-Q-note-cutoff.zip

Obviously, this approach will completely ruin the release envelope. Yet, if you consider it an acceptable solution, I have to figure out how long we're going to "drain" the filter this way, before we can truly turn off the voice.

[spessasus]

Why not just feed the filter into the gain amplifier like I've suggested earlier?
Even the section 9.1.4 does specify that the filtered output should go into the volEnv gain input.

[derselbst]

The gain of the volEnv is applied when fluidsynth interpolates the voice buffers to the output sample rate in fluid_rvoice_dsp*().

Therefore and from my understanding, you're suggesting to move the call to fluid_iir_filter_apply here

fluidsynth/src/rvoice/fluid_rvoice.c

Line 491 in d814a62

fluid_iir_filter_apply(&voice->resonant_filter, dsp_buf, count, voice->dsp.output_rate);

a few lines up before the fluid_rvoice_dsp*() calls. Yes, this would also attenuate the filtered output in accordance to the volEnv. I see two possible issues here:

The IIR filter has a dependency to the "output_rate", here

fluidsynth/src/rvoice/fluid_iir_filter.c

Lines 271 to 272 in d814a62

	fluid_real_t omega = (fluid_real_t)(2.0 * M_PI) *
	(iir_filter->last_fres / output_rate);

This is currently set to the synth's output_rate. We would need to access the sample's original sample_rate via voice->sample->samplerate. This might change the behavior of the filter and the filter cutoff in particular, as it affects omega inside the filter.

Also note that fluid_rvoice_dsp*() is responsible for pitching. So instead of applying the pitch before the filter, it would be applied on the filtered signal. That should be fine though, since it's a linear system.

Ok, I'll try that.

[spessasus]

I'm suggesting the same thing my synth is doing.

I.E.:

1. wavetable oscillator (with pitch control) fills the buffer
2. filter applied to the buffer
3. volume envelope applied to the buffer
4. the buffer goes wherever it needs to go (probably the panning node if fluid doesn't do any custom processing after that)

https://github.com/spessasus/SpessaSynth/blob/23d4bd96c04ac6f5e95d0d45052a7059107e7afe/src/spessasynth_lib/synthetizer/worklet_system/worklet_methods/voice_control.js#L158-L185

I hope this clears things up.

[derselbst]

Ok, I found a solution that integrates quite well. Here is the result:

1427_high-Q-note-cutoff_filter-before-volenv.zip

Essentially, the IIR filter is now run on-the-fly inside the sample interpolation. Though this means that only a single sample can ever be processed by the IIR filter at a time (rather than previously 64 samples in one go), it also eliminates the need to re-read the samples from memory. I do like that solution, thanks for your persistence! I'll open a PR shortly.