Progress on UI_TYPE=external

[lucianoiam]

Great progress there, not sure these are issues or pending features. Thought it is still useful for tracking:

(All macOS for now)

• Running make on EmbedExternalUI does not create the .vst bundle
• With a custom plugin, VST2 works as expected on REAPER, but there is no plugin window at all on Live. Checked that getParentWindowHandle() != 0.
• Any chance of having something like getApp().isStandalone() without DGL?

[falkTX]

* Running make on EmbedExternalUI does not create the .vst bundle

this is normal, the vst generation is part of the main makefile, not the individual plugins. until the embed-external-ui is more complete, I wont add it as default part of the build

• With a custom plugin, VST2 works as expected on REAPER, but there is no plugin window at all on Live. Checked that getParentWindowHandle() != 0

no idea sorry, but this is all still very new. I need to have a working example plugin so I can begin check on the issues with the implementation

• Any chance of having something like getApp().isStandalone() without DGL?

Sure, will add this now, pretty easy.

[lucianoiam]

All makes sense, thanks. Will test external on the three platforms as it evolves.

[falkTX]

Reorganized and added documentation at e59b5a5

[falkTX]

I found something I can use for testing, at least on Linux...

mpv has a --wid= switch haha.

[falkTX]

Just found out something for the X11 case, if the UI is embed a call to XMapWindow (or XMapRaised) is required.
The host must be able to access the plugin X11 window after the constructor, which at least on the DPF-provided example plugin, it was not the case.
Now the VST2 version loads fine.

[osch]

I have questions how to use the methods of UI::ExternalWindow in the case of external embedded UI.

As far as I understood the plugin UI implementation has to call ExternalWindow::setSize() in the UI constructor. Otherwise I will get:

The program 'ardour-6.9.0' received an X Window System error.
The error was 'BadValue (integer parameter out of range for operation)'.

So when should the plugin UI implemenation call ExternalWindow::setSize()? Only in the constructor? Every time if the plugin UI implementation receives a configure event from the operating system? If the plugin UI wants to change the window size: should it call operating system methods or should it call ExternalWindow::setSize()?

BTW: I'm calling XMapRaised in the UI constructor.

[falkTX]

Placing the size on the UI constructor call like MyCustomUI : UI(128, 128) or with a setSize during the constructor ends up being the same.
You do need to size your X11 window yourself though, that is the crucial part. The size of the X11 window must match what you have in the external window in order to keep things nicely in sync.
You can even add some extra flags to set min size, max size, aspect ratio etc.

Plugins wanting to change window size is a next step to do and verify, I dont think this works as-is yet.
The callbacks have a note mentioning this. That said, sizing from the plugin X11 window should work correctly in carla already, because carla detects child window size changes.
We are in a weird situation for LV2 where we want to get rid of the LV2 UI resize extension, but there is lack of proper support from hosts at the moment..

[falkTX]

BTW: I'm calling XMapRaised in the UI constructor.

FYI you should use XMapWindow instead, as if non-embed the XMapRaised will make the window visible without the host ever asking for such a thing. Maybe not an issue atm, but it will be if there is ever any lv2 host without direct raw x11 ui support.

[osch]

I'm already setting the size, minSize and maxSize in the native window construction. So it's strange to also set the size in the UI constructor.

So my question is: when should the plugin UI implementation call setSize? Should this be called if it receives a configure event? AFAIK this question is related to when getWidth / getHeight is called in in UIExporter. It seems to be called only in the vst2 case.

[osch]

FYI you should use XMapWindow instead, as if non-embed the XMapRaised will make the window visible without the host ever asking for such a thing. Maybe not an issue atm, but it will be if there is ever any lv2 host without direct raw x11 ui support.

I'm sorry, but I don't understand "will make the window visible without the host ever asking for such a thing".

[falkTX]

your constructor should have the correct size matching the native x11 window size, I thought this was obvious.

what the calls that end up being made in dpf, that is irrelevant and not for plugins to care.
because it really depends on the plugin format. vst2 hosts asks for size of a plugin ui, sometimes before, sometimes after, sometimes both cases. lv2 hosts dont care, they inspect the child window to find its properties. on jack/dssi you dont get a parent ui, and however you setup the x11 window is how it will be shown in the WM.

[osch]

So I should never call setSize only give the size in the constructor?

[osch]

I juist want to understand this: if I receive a configure event I should not call setSize, right?

[falkTX]

I'm sorry, but I don't understand "will make the window visible without the host ever asking for such a thing".

I mean that the host might not give you a parent window to embed into, on that case calling XMapRaised will place that window on the desktop before the host has asked for it to become visible.

So I should never call setSize only give the size in the constructor?

err whatever you prefer? as long as the size at the end is correct.
so pass it early or late on the constructor, doesnt matter

I juist want to understand this: if I receive a configure event I should not call setSize, right?

You can but this will do nothing to what the host is concerned.
Nothing is in place yet to take the new size and inform the host about it.
Some are smart enough to detect the change, others need help. It is only carla so far that will detect such things.

[lucianoiam]

Curious to know if Windows support is planned? just saw this:

#ifndef DISTRHO_EXTERNAL_WINDOW_HPP_INCLUDED
#define DISTRHO_EXTERNAL_WINDOW_HPP_INCLUDED

#include "String.hpp"

#ifdef DISTRHO_OS_WINDOWS
# error Unsupported platform!

Btw already ported my project to external UI on Linux and it is working great on REAPER and Carla. It works on Bitwig as well but there is an issue with the keyboard input.

This also happened with the previous SubWidget-based approach, and the workaround was relying on SubWidget::onKeyboard(const KeyboardEvent& ev) to route the keystrokes into the webview. Kind of ugly but worked. For some reason the webview does not receive keyboard focus only on Bitwig.

Wnere do the keystrokes for SubWidget::onKeyboard() come from in DGL? so I can replicate the behavior.

[lucianoiam]

I think I found it:

DistrhoPluginVST2.cpp

# if !DISTRHO_PLUGIN_HAS_EXTERNAL_UI
    int handlePluginKeyEvent(const bool down, int32_t index, const intptr_t value)

DistrhoUIInternal.hpp

#if !DISTRHO_PLUGIN_HAS_EXTERNAL_UI
    bool handlePluginKeyboard(const bool press, const uint key, const uint16_t mods)

Still puzzled why the webview on REAPER/Carla catches keystrokes by itself, and why on Bitwig they need to be read from the host, despite forcing keyboard focus to the webview. I am using code stolen from puglX11GrabFocus() to set focus.

[lucianoiam]

The PR still does not fix my issue but thought it could be useful to have.

[falkTX]

Curious to know if Windows support is planned?

Yes of course. It is just the code to start the external tool that is missing at the moment.
I can import some already written and working code I have from carla.

One thing we need to try is to do external process embed ui, like the linux mpv test. afaik windows also allows to embed into windows of another process, it is just macOS that blocks this.

[lucianoiam]

Yes unfortunately there is no solution for Mac at least for what I've researched during 2021.

[lucianoiam]

I have been updating all my stuff that runs on top of DPF and external is working great so far. On Linux, initial sizing works but then subsequent resizing does not work on Carla and Bitwig (floating windows). On REAPER it works as expected (embedded window). Same binary with configuration:

#define DISTRHO_PLUGIN_HAS_UI          1
#define DISTRHO_PLUGIN_HAS_EMBED_UI    1
#define DISTRHO_PLUGIN_HAS_EXTERNAL_UI 1

[falkTX]

for the resize on vst2, we need to put in place something to report that change to the host.
reaper requests the vst2 plugin window size every second and adjusts to it, but hosts dont typically do this.

[lucianoiam]

Makes sense, that perfectly explains what I'm seeing on REAPER when resizing, it works but takes the host a little while to catch up.

[lucianoiam]

I tried something like this, inspired on the DGL version:

DistrhoUI.cpp

#if DISTRHO_PLUGIN_HAS_EXTERNAL_UI

void UI::sizeChanged(uint width, uint height)
{
    UIWidget::sizeChanged(width, height);

    uiData->setSizeCallback(width, height);
}

#endif // DISTRHO_PLUGIN_HAS_EXTERNAL_UI

Here UIWidget is typedef'd to ExternalWindow in DistrhoUI.hpp, and uiData->setSizeCallback() results in the host calling effEditGetRect. Inside effEditGetRect:

                fVstRect.right  = fVstUI->getWidth();
                fVstRect.bottom = fVstUI->getHeight();

fVstUI->getWidth() and fVstUI->getHeight() both return 0. Setting fVstRect.right and fVstRect.bottom to a constant also does not work, the plugin view just disappears.

I feel this is totally wrong and the fix is actually deeper lol. This issue and the plugin not showing on Live (maybe both things are related?) is the only thing that is preventing me from completely ditching DGL. Any hint? thanks.

[falkTX]

That seems like the correct approach to me.
Basically plugging into https://github.com/DISTRHO/DPF/blob/main/distrho/src/DistrhoUI.cpp#L216 ? alike the DGL onResize correct?

Do you override this method as well?
Because on your code you will need UI::sizeChanged, instead of the perhaps expected ExternalWindow::sizeChanged.

[lucianoiam]

That seems like the correct approach to me.
Basically plugging into https://github.com/DISTRHO/DPF/blob/main/distrho/src/DistrhoUI.cpp#L216 ? alike the DGL onResize correct?

Yes, I think. Since UI::onResize(const ResizeEvent& ev) is not defined when DISTRHO_PLUGIN_HAS_EXTERNAL_UI is true, I created a version of it that does not depend on DGL and has the same signature as the existing ExternalWindow::sizeChanged() method: void UI::sizeChanged(uint width, uint height).

Then adapted DGL void UI::onResize(const ResizeEvent& ev) for this new method definition, like shown in the previous comment.

Do you override this method as well?
Because on your code you will need UI::sizeChanged, instead of the perhaps expected ExternalWindow::sizeChanged.

Yes, my UI subclass looks like this:

void AbstractWebHostUI::sizeChanged(uint width, uint height)
{
    UI::sizeChanged(width, height);
    ...
}

[falkTX]

Yeah that seems correct. I added that in d3bd0ec just now.
I think I need to build your stuff as I dont have a real external ui that can be freely resizable.

[lucianoiam]

Great. The only resizable example I have is this (branch nodgl). I am not sure it is the best way to debug the issue because of the complexity, maybe I'd suggest adding a thread to EmbedExternalUI that polls for user entered width,height from a file or something like that. Also that would rule out any mistake in my own code lol.

[falkTX]

I have an idea. I can make the example UI already in dpf use width/height as parameters. so changing them from the generic host editor should change the native/custom-ui side.

[lucianoiam]

Right, it is expected fWindow to be null when running embedded, because the host window/view is provided by getParentWindowHandle() instead. All initialization code for the embedded case looks good.

Isn't polling events in uiIdle() unnecessary for all platforms, not just macOS ?

[lucianoiam]

Then tried setting the w/h parameters and after reopening the UI there is no change but that is maybe because the example plugin does not read the w/h parameters on start

This is wrong... immediately after opening the UI parameterChanged() is called for width and height, making REAPER suitable for testing. Still seems that setSize() does not reach that host.

[falkTX]

Isn't polling events in uiIdle() unnecessary for all platforms, not just macOS ?

incorrect.

I mean on x11 for example if we want to repaint, handle keyboard, etc we need to fetch the events for our respective window.
If you mean that driving the event loop is unnecessary, that would be true on macOS and windows if DPF itself did it, but it does not while in external-ui mode.
the event-loop might be something custom, like qt or glib, so best to not interfere.

on linux/x11 we always need to drive the events ourselves, and there is no global event loop to hook into.

[falkTX]

not that it matters too much, but I added some initial win32 code on the example plugin now.
I was thinking I didnt want to deal with this custom native code, but on 2nd thought it seems best to keep it and try to improve it.
Reasoning is that others will try to plug DPF into native code, so having an example that is known to work is great, as they can then copy a potential good workflow/setup of native windows. Not that we are there now though.

[lucianoiam]

Thanks for your time having a look at this issue, and the clarification regarding X11.

I agree it is great to have this example native code because it is likely what a plugin will end up doing. Moreover I propose having a generic version of EmbedExternalUI that could be used as the base for non-DGL plugins, ie. abstract the hosted and standalone cases. Likely you want a main window for the standalone like JUCE does. For the Linux the default would be X11, then it is up to the user to create GTK or whatever versions at their own risk.

Not that I dislike DGL, I only think the framework would be more flexible if it was an option at the same abstraction level as 3rd party UIs. This is starting to manifest with ExternalWindow replicating some interfaces, if I correctly understand. But this is really a design choice... things work pretty good the way they are now.

So I retried EmbedExternalUI on the three platforms, this is the current state:

1. Linux: works flawlessly, both for initial size and resizing
2. Windows: works for resizing, initial size is wrong.
3. Mac: Initial size correct, resizing does not work (ie, host never notified it seems)

[falkTX]

Moreover I propose having a generic version of EmbedExternalUI that could be used as the base for non-DGL plugins

I am heavily against this. this would basically replicate what pugl already does, I have no intentions to provide such a thing.

Not that I dislike DGL, I only think the framework would be more flexible if it was an option at the same abstraction level as 3rd party UIs. This is starting to manifest with ExternalWindow replicating some interfaces, if I correctly understand. But this is really a design choice... things work pretty good the way they are now.

Well you typically would not even be allowed to use 3rd party GUIs on frameworks like DPF. This external ui/window access is an extra we are trying to make work as if native, but obviously it wont be the default choice.

So I retried EmbedExternalUI on the three platforms, this is the current state:

1. Linux: works flawlessly, both for initial size and resizing

This is expected, I have more experience with X11 than the other 2 :)

2. Windows: works for resizing, initial size is wrong.

I just pushed the code, so I guess I forgot something. PRs welcome.

3. Mac: Initial size correct, resizing does not work (ie, host never notified it seems)

That will be because sizeChanged has nothing on macOS.
We need to resize the view frame, and also the window if not null.
Again, PRs welcome.

[lucianoiam]

All points valid

2. Windows: works for resizing, initial size is wrong.

I just pushed the code, so I guess I forgot something. PRs welcome.

3. Mac: Initial size correct, resizing does not work (ie, host never notified it seems)

That will be because sizeChanged has nothing on macOS.

What I don't understand is, shouldn't all these things expected to be empty for the embedded case? I mean, handling a custom window dimensions is only needed for standalone. The issue is not the content view itself not being resized ( yes, that is the plugin dev responsibility ), but the host not correctly being notified of a size change, for example REAPER never gets a chance to tightly wrap the contents in Mac.

This is backed by my observation that on Windows, there are no calls at all to Win32 in the EmbedExternalUI example, but still REAPER is still able to resize its container after tweaking w/h parameters.

When running as a plugin, shouldn't this little new code be enough for all platforms?

DistrhoUI.cpp

#if DISTRHO_PLUGIN_HAS_EXTERNAL_UI
void UI::sizeChanged(const uint width, const uint height)
{
    UIWidget::sizeChanged(width, height);

    uiData->setSizeCallback(width, height);
}
#else

EmbedExternalExampleUI.cpp

    void sizeChanged(uint width, uint height) override
    {
        d_stdout("sizeChanged %u %u", width, height);
        UI::sizeChanged(width, height);
        ...

I am happy to work on these on my own but I frankly hit a wall. I feel either I am missing something, or the issue is deeper than we think, hidden in some layer.

[falkTX]

The issue is not the content view itself not being resized ( yes, that is the plugin dev responsibility ), but the host not correctly being notified of a size change, for example REAPER never gets a chance to tightly wrap the contents in Mac.

well the code is the same for all platforms for this case, so it should just work.
are you sure the host is not notified?

seems more likely that the host expects the plugin view to be resized too.

When running as a plugin, shouldn't this little new code be enough for all platforms?
...

yes...ish, except those that try to be a bit smart and look into the child window to see if it is doing things incorrectly.
if you ask the host to resize its window but your native view/window remains unchanged, well this is undefined behaviour.

for LV2 we are even trying to get this to work without calling any host features, so that the host detects the child window change all on its own.
so the child window (the plugin one) needs to be correct regardless.

[lucianoiam]

The issue is not the content view itself not being resized ( yes, that is the plugin dev responsibility ), but the host not correctly being notified of a size change, for example REAPER never gets a chance to tightly wrap the contents in Mac.

well the code is the same for all platforms for this case, so it should just work.
are you sure the host is not notified?

So far only visual inspection. What conclusive check to use? checking effEditGetRect is called?

seems more likely that the host expects the plugin view to be resized too.

When running as a plugin, shouldn't this little new code be enough for all platforms?
...

yes...ish, except those that try to be a bit smart and look into the child window to see if it is doing things incorrectly.

Oh my... considered something like that at first then thought, it would be a very dirty thing to do. But it is definitely worth looking, if the Mac/Win children are never resized (or don't exist?) that could explain why the hosts are always getting the same dimensions.

if you ask the host to resize its window but your native view/window remains unchanged, well this is undefined behaviour.

for LV2 we are even trying to get this to work without calling any host features, so that the host detects the child window change all on its own.
so the child window (the plugin one) needs to be correct regardless.

Will write some code and get back.

[falkTX]

So far only visual inspection. What conclusive check to use? checking effEditGetRect is called?

effEditGetRect is for host->plugin call, the setSizeCallback on the VST2 DPF code is the important part.

the plugin->host call is at https://github.com/DISTRHO/DPF/blob/main/distrho/src/DistrhoPluginVST2.cpp#L400-L408

[lucianoiam]

Well I found the cause why EmbedExternalUI is not showing on Live/Mac:

DPF/distrho/src/DistrhoUIInternal.hpp

Line 127 in dbbfef6

return uiData->window->getScaleFactor();

That always returns 0, I am surprised the host does not crash after dividing w/h by 0 later on. Changing that function to return 1 fixes the issue. Could you point me to a proper solution? I got lost in the layers, not sure where that scale factor should be set for the ExternalWindow. Thanks.

[falkTX]

huh very interesting.
yeah the return 0 was a way I changed to have this possibly fallback to desktop scaling if the host doesnt provide a value.
but we removed the check for != 0 since then (the whole macos auto-scaling thing).

just switch on the WindowPrivateData to have 1 as fallback instead of 0, that should do the trick.

[falkTX]

actually I can do this here hang on...

[falkTX]

I was wrong, the 0 value is fine. issue is that we never replace it with something else for the external-ui case, but always do for the regular dgl stuff.
Fixed in 37e6922 but I didnt actually test it, should build on macOS at least.
(though I have an issue somewhere else, that I will fix soon)

[lucianoiam]

I left a comment in 37e6922#diff-98d4a2bee9607ee4512f666bf07dd845a0a4abc56588b67c3f15d4a35b4f727fR57 , a stopper and also a picky one that I can address on my own.

[falkTX]

To comment about the progress on this, everything seems to work now.
We just need to fix Windows usage for non-embed scenarios.

[lucianoiam]

Do you mean the EmbedExternalUI example for Windows? which is lacking standalone support.

Another issue is getDesktopScaleFactor() not working for standalones because it is called with the parent window handle which is 0 in such case.

[falkTX]

Another issue is getDesktopScaleFactor() not working for standalones because it is called with the parent window handle which is 0 in such case.

This is what I mean for the windows case, when non-embed we do not get a parent window. so we need to grab whatever scale factor is the default on the desktop (in case host does not provide this info)

The parent window being 0 in non-embed cases is normal and expected. That is why there is a check for this on the getDesktopScaleFactor

[lucianoiam]

How about a virtual ExternalWindow::getExternalWindowHandle() ? it could be then passed to getDesktopScaleFactor().

I first thought about making getParentWindowHandle() virtual but that would not work if called from the UI constructor, which is likely ( embedextui does it in fact ).

[falkTX]

How about a virtual ExternalWindow::getExternalWindowHandle() ? it could be then passed to getDesktopScaleFactor().

you mean the transient window parent? I think that is already in place there.
but that will only be valid for LV2, on JACK/Standalones there is no parent window of any kind.

[lucianoiam]

DistrhoUI.cpp does this:

UI::PrivateData::createNextWindow(UI* const ui, const uint width, const uint height)
{
    UI::PrivateData* const pData = s_nextPrivateData;
#if DISTRHO_PLUGIN_HAS_EXTERNAL_UI
    ...
    ewData.scaleFactor = pData->scaleFactor != 0.0 ? pData->scaleFactor : getDesktopScaleFactor(pData->winId);
    ewData.title = DISTRHO_PLUGIN_NAME;
    ewData.isStandalone = DISTRHO_UI_IS_STANDALONE;

That pData->winId will be 0 for standalone if I correctly understand. I am thinking how to pass a user-created window to that getDesktopScaleFactor(), it would be either the parent handle (if available) or the custom (standalone) one.

[falkTX]

That pData->winId will be 0 for standalone if I correctly understand.

Yes correct. On LV2 we can still get a transient window id, but that won't be a much used scenario.

I am thinking how to pass a user-created window to that getDesktopScaleFactor(), it would be either the parent handle (if available) or the custom (standalone) one.

I don't think you can do that, since it would required creating the window first.
The order of initialization is not the best here for external uis, and not sure if it can reasonably be done without getting super confusing.
For such cases you can just get the scale factor yourself anyway, you do not have to take the hint from DPF, you are free to ignore it. So that for standalones, figure out the scale factor yourself after you create your own window(s).

[lucianoiam]

I agree, initialization order would make all this complicated.

[falkTX]

I plan now to close this ticket. the parent-less situation with win32 was handled in b2e683e (turns out 0,0 always refers to the default screen/monitor)

Is there anything still to do, besides perhaps documentation and examples?

[lucianoiam]

On my side there’s nothing I can think of. External has been working great on all platform/host combinations tried so far, including resizing.

…

On 8. Sep 2021, at 20:13, Filipe Coelho ***@***.***> wrote:  I plan now to close this ticket. the parent-less situation with win32 was handled in b2e683e (turns out 0,0 always refers to the default screen/monitor) Is there anything still to do, besides perhaps documentation and examples? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[falkTX]

Nothing added or to be said from my side in 10 days, so I am closing this ticket.
Any particular issues still remaining can be dealt with in their own tickets.