Support arbitrary surface shape + input region by X11 clients

[rjhorniii]

Add to roadmap document that certain features will not be ported: the circular and other non-rectangular windows will not be supported by wlmaker and note that screengrab will be limited. This is because there is no agreed protocol extension for Wayland that supports them.

The particular example that I use is a simple clock. It has a circular window with no window decorations on Windowmaker (X11) and Plasma (X11). On wlmaker and Plasma (Wayland) it becomes a rectangular window with window decorations. Mouse and keyboard behavior is similarly different. On X11 mouse and keyboard assignment are controlled by the circular boundary, and Wayland they are controlled by the rectangular window and decoration. You can't see it in these static images, but the fraction of the background in the area of the rectangle that is not inside the circle is from the background when the program started, not the current background. It is also not stable. It changes independently of the screen background. Examples:

On windowmaker (x11)

On Plasma (Wayland)

While doing these captures I was also reminded that Wayland does not support arbitrary screen area capture. It's either full single window or full screen. As I recall this is part of the security restrictions so that compositors, clip board managers, etc., can enforce access restrictions easily.

[phkaeser]

Hm... native Wayland applications can use transparency in surfaces, paired with an arbitrarily fine-grained set of rectangular input regions (https://wayland.app/protocols/wayland#wl_surface:request:set_input_region) to define arbitrary surfaces (the input region would remain a circle, but that should be a good-enough approximation).

Do I understand correctly that the concern here is for an X11 client application, where the compositor(s) currently lack interpretation of the X11 client's non-rectangular shape?

[rjhorniii]

I'm concerned about the extent and complexity of the application changes needed to make this work. I haven't read the code yet, but I know that it works that same way on most of the X11 window managers. This is because it's primarily interactions with the X11 server over the X11 protocol.

I was hoping that there would be an extension protocol by now that would allow a compositor independent common solution. That clock application works on all the different X11 windowmanagers that I have. There is no code modification or different code paths for different window managers. It's the same executable for all the different window managers.

[phkaeser]

Apologies for not being more clear.

1. For Wayland clients, the set_input_region is an element of the core Wayland surface protocol, and allows them to define arbitrary surface shapes, for routing mouse interactions. And using transparency on the surface itself for visualization purpose. As a core Wayland protocol, that should (*) be supported by all compositors. That should address your goal.

2. For X11 clients, the X11 protocol provides XShape. I suspect this is currently not translated by XWayland, and would explain the optics you observed for the clock application.

At this point, I'd like to have full understanding of what you're seeing with the simple clock app:

• Does it have a native Wayland client mode? If so, I'd love to understand why (1) above doesn't apply.
• Or, is it using the X11 client protocol? If so, this becomes a question of if/how XShape can be forwarded through XWayland.

(*) That might yet be missing from wlmaker... but we're still early in development.

[rjhorniii]

It's an old application (first version 2007, major revision 2019) and is using X11. https://www.theknight.co.uk/ A very quick look indicates it's using cairo and gobject to access X11 based on the library references in the Makefile. I'll have to dig further into source to see exactly how it is working.

[phkaeser]

Thanks for clarifying!

For scope of this bug, I'll update the title to "Support arbitrary surface shape + input region by X11 clients", and mark it as enhancement. Not sure XWayland will permit doing that; but it's a good thing to keep track.

Regarding tzclock: The example client (https://github.com/phkaeser/wlmaker/blob/main/apps/example_toplevel.c#L74) from the wlmaker
source demonstrates using cairo with a double-buffered toplevel surface. It might be possible to update the code into using a Wayland client.

[rjhorniii]

Based on documentation for Cairo and gtk it looks like the recommended path is about what you suggested. There are cairo equivalents that let you specify an arbitrarily shaped input region and an arbitrarily shaped transparency mask. You also have to disable window decoration. If done carefully the combined result should work on both X11 and Wayland.

It turns out that the clock program uses gtk_widget_shape_combine_region to manage this. That's a gtk3 function that was dropped from gtk4 due to lack of Wayland support.

When I've got some spare time I'll see what it takes.

[phkaeser]

For applications with gtk4, the following functions may be worth exploring:

• https://docs.gtk.org/gdk4/method.Surface.set_input_region.html
• https://docs.gtk.org/gdk4/method.Display.supports_input_shapes.html

[rjhorniii]

Converting an app from gtk3 to gtk4 is reported to be a big change, so it may take a while. I'll get it working first on windowmaker (x11), then Plasma (wayland), and then see whether there are any bugs for wlmaker.