Hosting external LV2 Plugins

[dbrgn]

I'm used to digital audio mixing consoles from Yamaha, and I miss the visual feedback when using the non-graphical equalizer in easyeffects. Here's what the Yamaha CL series EQ looks like:

In the Linux world, I'm a fan of some of the Calf virtual effect devices, for example the graphical multi-band equalizer that is quite similar:

Specifically the combination of variable-frequency bands and combination with the audio level visualization in the background are really useful, because it allows you to see how your settings are influencing the mix. It also allows you to see and immediately fix issues like audio feedback.

I searched through the issue tracker regarding graphical equalizers, and found #634, #515 and #663, and it seems that you're using some APIs of existing effect plugins, but that graphing them is difficult, correct?

Might it be an alternative to allow hosting external LV2 plugins directly, similar to the way calfjackhost does it? Would that be possible? (I'm not familiar with the APIs involved, so I don't know what effort this would involve. Calfjackhost _looks_simple, but maybe it isn't.)

[wwmm]

and it seems that you're using some APIs of existing effect plugins, but that graphing them is difficult, correct?

Yes. The plugins can be used like a library. The built-in graphical interface is optional.

Might it be an alternative to allow hosting external LV2 plugins directly, similar to the way calfjackhost does it? Would that be possible?

It is hard to say. I looked into this situation a long time ago and it is a tough thing to handle. The main problem is that you can't under normal circumstances load a plugin that uses one version of a toolkit inside a window that uses a different version. For example you can not load a plugin that uses gtk2 inside a program that uses gtk3. And if you want to embed a plugin that uses QT inside GTK or vice versa things probably get worse.

In most cases plugins hosts seem to workaround that by using this http://drobilla.net/software/suil.html. But there are some disadvantages

• As far as I could see it does not support gtk4
• We would now be at the mercy of Suil development pace. What means we would not be able to "just move" to newer toolkits versions when we wanted. For example if I had used it while we were in gtk3 I would not have been able to move to gtk4. What sucks.
• Suil won't solve the problems with translation handling. We would probably depend on them being submitted to each plugin author.

It is tough. I would really like to have the graphs. But the disadvantages are huge. If we were a general purpose like Ardour or Carla it would make sense. But we are not.

[wwmm]

But the disadvantages are huge.

And I did not even talked about the possible difficulties in synchronizing what happens in the plugin window with our preset system. So it is far from being something simple to do. Unfortunately.

[dbrgn]

Ah, I see. Since calfjackhost is calf-specific, they can sync their toolkit versions across all plugins, I guess.

Then the best way to use these graphical plugins with PipeWire is probably to go through Jack, along with some setup script, right?

[Bleuzen]

I do agree that having proper interfaces for the plugins would be very useful.

For the downsides:

I looked into this situation a long time ago and it is a tough thing to handle. The main problem is that you can't under normal circumstances load a plugin that uses one version of a toolkit inside a window that uses a different version. For example you can not load a plugin that uses gtk2 inside a program that uses gtk3. And if you want to embed a plugin that uses QT inside GTK or vice versa things probably get worse.

This is indeed a problem. I did also face it while packaging the Calf plugins as flatpak (had to disable their GUI so that they can run in GTK hosts).

However, under all the plugins packaged on flathub, afaik Calf is the only one where we had this issue.

Also there are two more I could think of (just not packaged and not very useful in EasyEffects anyway):

• Carla
• samplv1, synthv1, drumkv1, padthv1

However these are not many. The very most plugins do not suffer from this problem because nowadays they use non-desktop UI toolkits which work perfectly fine alongside Qt and GTK hosts.

So yeah, while some plugins may not work, I wouldn't see this as a blocker.
Most plugins will work just fine. This would already be a great benefit. (even without bothering with suil)

For example the LSP plugins already in use by EasyEffects do also use their independent UI toolkit, so EasyEffects could display their custom interfaces without a toolkit collision.

And for the rest...
maybe EasyEffects could detect if a plugin is linked to GTK or QT and in that case disallow the "show external UI" button?
Or alternatively some kind of allow/exlude-list approach could work.

And I did not even talked about the possible difficulties in synchronizing what happens in the plugin window with our preset system.

Possibly save the plugin states base64 encoded?
Ardour seems to do this for VST3s. Not sure if LV2 also supports reading/setting full plugin states easily.

[wwmm]

The very most plugins do not suffer from this problem because nowadays they use non-desktop UI toolkits which work perfectly fine alongside Qt and GTK hosts.

I am not sure it is so straightforward. Based on the LV2 specifications https://lv2plug.in/ns/extensions/ui#X11UI when not using gtk or QT the plugin will create its window drawing directly to X11. A long time ago I saw a super old gtk code that showed a X11 window inside a container specifically designed for this. There was X11 only code everywhere... Besides the obvious problem of having something like this in Wayland times I think that the gtk container that did this isn't available even in gtk3. This may be one of the reasons why Ardour still uses the gtkmm version that wraps gtk2. I seriously doubt gtk4 has this kind of container. Its approach to handle drawings is very different.

For example the LSP plugins already in use by EasyEffects do also use their independent UI toolkit, so EasyEffects could display their custom interfaces without a toolkit collision.

It is not going to be this easy for the reasons I said above. If I am not mistaken this "independent UI toolkit" draws directly to X11 as expected from what the LV2 specs recommend. I doubt it is easy to put a X11 window inside a gtk4 container that is designed to work both on X11 and Wayland. I have never seen something like this in the gtk4 code examples or in its API.

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like... This kind of workflow may be fine for people that are used to do music production in the usual hosts. But I think it would be super weird in EasyEffects to have the plugins controls in other windows.

maybe EasyEffects could detect if a plugin is linked to GTK or QT and in that case disallow the "show external UI" button?

There is no need to disallow anything. It is just a matter of not accessing the window port. We already ignore it because my simple LV2 wrapper does not support the plugin window anyway.

Possibly save the plugin states base64 encoded?

I am not sure it would help. As I said above the LV2 standard has a port(a C pointer) that allows us to interact with the controls in the window. The problem is loading the window and embedding it in a gtk container so it is shown inside EasyEffects window like any other widget.

[wwmm]

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window.

What would still require code capable of displaying a X11 object in a standalone window that works both on X11 and Wayland. But I have the feeling this will require X11 code in our sources. It does not feel the right thing to do....

[polarathene]

In regards to the graphic EQ support, I got a bit lost with the discussion of toolkit concerns (and in related linked issues with how EasyEffects communicates with LSP and the difficulties there for supporting in EE UI).

Does any of that apply to making a separate standalone GUI (be that GTK/Qt or a browser tab web app) that can communicate with EasyEffects (or PipeWire if that makes more sense) to provide controls that update the EasyEffects EQ?

From what I've seen elsewhere, it was suggested to do the EQ in a different program that can export an APO preset?

Is it difficult to support an API (be that over a socket, dbus, or C FFI) to manipulate EasyEffects EQ from the external program? That way EasyEffects can run with the full chain of effects in place and have the EQ adjusted in real-time by an external program that provides different GUI / controls.

---

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like...
This kind of workflow may be fine for people that are used to do music production in the usual hosts. But I think it would be super weird in EasyEffects to have the plugins controls in other windows.

Perhaps that's what I'm asking for, a way that third-party app could hook into EasyEffects either to control the existing EQ effect or implement it's own EQ effect (apparently this is what an LV2 plugin is suitable for?).

Is the toolkit (and X11 vs Wayland) issue only regarding embedded UI into EasyEffects? Or a problem specific to LV2 support?

Is another option (if doing EQ externally of EasyEffects), to have an effect that has PipeWire routing to the external program and back to EasyEffects? I am still new to PipeWire, but that seemed viable from what I saw inspecting via Helvum.

The EQ visualization would be quite helpful for me, and if there are limitations that prevent EasyEffects supporting that, but don't apply if deferring to an external application, that'd be great to leverage.

[wwmm]

Is it difficult to support an API (be that over a socket, dbus, or C FFI) to manipulate EasyEffects EQ from the external program?

You can configure almost all EasyEffects parameters through any program that uses the GSettings API. Even through command line tools. Take a look at #916. The discussion was for PulseEffects but the same idea applies to EasyEffects.

What you are not going to be able to get at this moment are the level meter values. GSettings is not designed for this kind of task so this would require the development of an API.

Is the toolkit (and X11 vs Wayland) issue only regarding embedded UI into EasyEffects? Or a problem specific to LV2 support?

This is way beyond just EasyEffects. Audio hosts like Ardour or Carla also have to deal with this problem. Most of the DAWs running on Linux make use of the suil library to load the custom plugins interface. But besides the fact it probably does not work with gtk4 right now the translation handling will definitely be more complicated when using the plugin custom interface.

Is another option (if doing EQ externally of EasyEffects), to have an effect that has PipeWire routing to the external program and back to EasyEffects? I am still new to PipeWire, but that seemed viable from what I saw inspecting via Helvum.

You can use Helvum to insert a PipeWire filter in the middle of our pipeline. But as EasyEffects isn't designed assuming such level of customization you may have to remake these links quite often.

The EQ visualization would be quite helpful for me, and if there are limitations that prevent EasyEffects supporting that, but don't apply if deferring to an external application, that'd be great to leverage.

Unfortunately this is is impossible to do at this moment unless we write our own equalizer. More details at #515. But besides the fact I do not know how to write one matching the flexibility and quality of the one from Linux Studio Plugins is not going to be a trivial task.

I think that what some users do is loading the Linux Studio Plugins standalone interface to set the parameters and view the graphs. Once the curves are like they want they export an APO preset and import it in EasyEffects.

[wwmm]

Unfortunately this is is impossible to do at this moment unless we write our own equalizer. More details at #515.

At the time we used GStreamer and it was totally impossible. Nowadays we have our own simple LV2 wrapper and in theory we now have the possibility of adding what was missing in the GStreamer wrapper... So maybe it isn't as impossible as it was in the past. I have to take some time to investigate if dealing with the custom plugin interface port would also be required to access the curves. In any case I doubt it is standard. It is probably necessary to read the plugin code to see what it is doing and pray for its developers do not change how the information is being coded/decoded.

[polarathene]

But as EasyEffects isn't designed assuming such level of customization you may have to remake these links quite often.

But an "effect" that could configure the routing link would resolve that right?

But besides the fact I do not know how to write one matching the flexibility and quality of the one from Linux Studio Plugins is not going to be a trivial task.

I thought that all the settings exposed in the current EQ effect would be the same, and the only difference being requested was regarding the visualization of those settings or alternative UI controls?

Hence if that was an issue for EasyEffects to handle in it's UI, a separate project could have a GUI that manipulates the same settings but can visualize the curve/graph.

It'd just need to update the same settings that EasyEffects handles via the EQ effect, which you've said can mostly be done via GSettings API, while the missing functionality would need some more development to support an API from EasyEffects? That seems alright :)

---

I have to take some time to investigate if dealing with the custom plugin interface port would also be required to access the curves. In any case I doubt it is standard.
It is probably necessary to read the plugin code to see what it is doing and pray for its developers do not change how the information is being coded/decoded.

No worries, I don't expect you to sink time into this 👍

I was just curious if EasyEffects had the flexibility for supporting it (either via direct contribution to EasyEffects, or a separate app integrating with EasyEffects). I might take a shot at it myself some time :)

It is a feature I often see on videos or guides that teach how to EQ vocals and for me was much easier to interpret than sliders / numbers evenly spaced apart (my ears aren't audiophile grade, but I can read the curves much better while I make adjustments):

[p-chan5]

I don't know much about these libraries but I think that feature would have a lot of potential! It could unlock the limits of audio effects creativity ✨

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like...

Maybe you could create a new effect called Wrapper (bridge) that allows to manage external plugins (lv2/vst/vst3). There are some projects that are based on this feature (but for Windows plugins):
https://github.com/robbert-vdh/yabridge
https://github.com/osxmidi/LinVst

[wwmm]

Maybe you could create a new effect called Wrapper (bridge) that allows to manage external plugins (lv2/vst/vst3)

From a technical point of view I imagine it is possible. But the amount of work required to integrate this in EasyEffects is really huge. Even if we talk about only native Linux plugins. Not only there is the annoyance involved in showing the plugin window or creating a generic one for it but there is also the old problem of integrating everything with our presets infrastructure.

[p-chan5]

It would be cool to see a feature of that level in EasyEffects! Don't throw this idea away, there must be some way to integrate this. I hope one day it will be possible. 😥️