Skip to content

Input triggers spike#4410

Open
tarek-y-ismail wants to merge 5 commits intomainfrom
input-triggers-spike
Open

Input triggers spike#4410
tarek-y-ismail wants to merge 5 commits intomainfrom
input-triggers-spike

Conversation

@tarek-y-ismail
Copy link
Contributor

@tarek-y-ismail tarek-y-ismail commented Nov 13, 2025

What's new?

Quick and dirty spike to feel out how the input trigger protocol would be to work with

How to test

Direct communication with privileged client:

  • Run mir with --add-wayland-extensions=ext_input_trigger_action_manager_v1:ext_input_trigger_registration_manager_v1
  • Inside the running compositor, launch a terminal
  • Run ./build/bin/trigger_client
  • At the moment, it's hardcoded to request Ctrl + Shift + C and ALT + x for key syms, and Alt + z for scan codes. If you press any of them. "Hey from " will be printed
  • Letting go will result in "Bye from " being printed.

To test Alt + z, run mir with --keymap fr. This should use the azerty layout and make the z key print w, and the q key print z. Even in the azerty layout, Alt + z with the azerty z will not work, but the same combination with the qwerty z will. Note: Right alt is used for special letters, so use left alt :)


Gatekeeper:

  • Run mir with --add-wayland-extensions=ext_input_trigger_action_manager_v1:ext_input_trigger_registration_manager_v1
  • Inside the running compositor, launch a terminal
  • Run python3 examples/gatekeeper/gatekeeper.py --show-ui
  • Run python3 examples/gatekeeper/test_client.py
  • The gatekeeper should request to register two shortcuts. You're free to reject, modify them, or accept them as is.
  • Pressing the shortcuts with the test client not focused will print different messages when begin and end are received.

Checklist

  • Tests added and pass WLCS tests require the addition of keyboard access. Probably for another PR.

@tarek-y-ismail tarek-y-ismail self-assigned this Nov 13, 2025
@tarek-y-ismail tarek-y-ismail force-pushed the input-triggers-spike branch 2 times, most recently from 76a9c52 to 29f6da5 Compare November 14, 2025 12:32
@tarek-y-ismail
Copy link
Contributor Author

tarek-y-ismail commented Nov 14, 2025

  • The alphabetic key specified in the client request has to be the last key to be pressed. Pressing C + Shift + Ctrl or other combinations where C isn't the last character will not work.

We could solve this in two ways: Explicitly require users to press the alpha key at the end, or manually keep track of the pressed keysyms, consuming only the event where all the keys the user specified are down, which seems to be what i3 does on my machine. For example:

  1. ctrl is pressed, goes to the focused client
  2. c is pressed, goes to the focused client. We record that c is down
  3. shift is pressed. We check if all modifiers and alpha keys are down and if so, we consume this event and call begin.

Edit: I'm opting to implement the second solution

@tarek-y-ismail tarek-y-ismail force-pushed the MIRENG-1493-input-triggers branch from 41c94a9 to 3599e06 Compare November 18, 2025 08:01
@tarek-y-ismail tarek-y-ismail force-pushed the input-triggers-spike branch 2 times, most recently from f488d7b to 5e01560 Compare December 4, 2025 17:17
@tarek-y-ismail
Copy link
Contributor Author

Big TODO: The token authority should be modified to create a new token type (issue_trigger_token?) to allow multiple triggers to be registered.

Right now, tokens are used to store the relationship between a trigger, the input filter, and the corresponding action (that's what I remember, the architecture is a bit hazy in my head right now). So having tokens that are invalidated after a short time would require replacing this with something else.

The first idea that pops up into my head is using the token to just establish the relationship between the three objects, but not holding onto it as I do right now.

@AlanGriffiths AlanGriffiths force-pushed the MIRENG-1493-input-triggers branch from 7c58613 to 90f9c87 Compare January 16, 2026 11:09
Base automatically changed from MIRENG-1493-input-triggers to main January 20, 2026 11:07
@tarek-y-ismail tarek-y-ismail force-pushed the input-triggers-spike branch 4 times, most recently from 1cf3f74 to f8c1c47 Compare January 20, 2026 14:41
Comment on lines 70 to 71
input_trigger_registration_v1.cpp input_trigger_registration_v1.h
input_trigger_action_v1.cpp input_trigger_action_v1.h
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugly indentation. But probably something for another PR

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a spike/prototype for an input trigger protocol that allows Wayland clients to register global keyboard shortcuts. The protocol consists of two parts: trigger registration (defining shortcuts) and action management (handling shortcut events).

Changes:

  • Implements Wayland protocol extensions for input trigger registration and action management
  • Adds support infrastructure including token-based action tracking and event filtering
  • Includes example client and Python-based gatekeeper for testing the protocol

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/server/frontend_wayland/wayland_default_configuration.cpp Registers the two new input trigger protocol extensions in the Wayland extension system
src/server/frontend_wayland/wayland_connector.h Adds InputTriggerData and Clock dependencies to WaylandExtensions context
src/server/frontend_wayland/wayland_connector.cpp Instantiates InputTriggerData and passes it to extension context
src/server/frontend_wayland/input_trigger_registration_v1.h Defines protocol interfaces for trigger registration and KeyboardSymTrigger class
src/server/frontend_wayland/input_trigger_registration_v1.cpp Implements registration manager, action controls, and modifier mapping logic
src/server/frontend_wayland/input_trigger_data.h Defines shared data structure for tracking pending trigger actions
src/server/frontend_wayland/input_trigger_action_v1.h Defines protocol interfaces for action management
src/server/frontend_wayland/input_trigger_action_v1.cpp Implements action manager, keyboard event filtering, and modifier matching
src/server/frontend_wayland/CMakeLists.txt Adds new source files to build system
src/common/input/xkb_mapper.cpp Moves expand_modifiers function to namespace scope for external use
include/common/mir/input/xkb_mapper.h Exports expand_modifiers function for use in trigger protocol
examples/trigger_client/main.cpp Implements standalone C++ Wayland client to test trigger registration
examples/trigger_client/CMakeLists.txt Build configuration for trigger_client example
examples/gatekeeper/test_client.py Python test client using D-Bus portal interface
examples/gatekeeper/generate_protocols.py Script to generate Python bindings for Wayland protocols
examples/gatekeeper/gatekeeper.py Python-based UI for managing global shortcuts via D-Bus portal
examples/CMakeLists.txt Includes trigger_client subdirectory in build
debian/mir-demos.install Adds trigger_client to package install list
rpm/mir.spec Adds trigger_client to RPM package
CMakeLists.txt Adds coverage build configuration and fixes platform path in LGPL test

@github-actions
Copy link

TICS Quality Gate

✔️ Passed

mir

Coding Standards: ✔️ Passed

✔️ Condition “No new Coding Standard Violations for level 1, 2, 3 with respect to Previous analysis” passed.

See the results in the TICS Viewer

The following files have been checked for this project
  • include/common/mir/input/xkb_mapper.h
  • src/common/input/xkb_mapper.cpp
  • src/server/frontend_wayland/input_trigger_action_v1.cpp
  • src/server/frontend_wayland/input_trigger_action_v1.h
  • src/server/frontend_wayland/input_trigger_data.h
  • src/server/frontend_wayland/input_trigger_registration_v1.cpp
  • src/server/frontend_wayland/input_trigger_registration_v1.h
  • src/server/frontend_wayland/wayland_connector.cpp
  • src/server/frontend_wayland/wayland_connector.h
  • src/server/frontend_wayland/wayland_default_configuration.cpp

TICS / TICS / Run TICS analysis

Copy link
Contributor Author

@tarek-y-ismail tarek-y-ismail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick review for the gatekeeper and test client

Gatekeeper is a bit spaghettified, test client is alright.

Comment on lines 13 to 14
# Add current directory to path to find generated protocols
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary?

Comment on lines 62 to 69
# Minimal Keysym definitions (XKB)
KEY_A = 0x0061
# ... add more as needed

MOD_SHIFT = 0x08
MOD_CTRL = 0x100
MOD_ALT = 0x01
MOD_META = 0x800
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, maybe there's a library to handle this?

Comment on lines 100 to 101
lbl = Gtk.Label(label="An application wants to register the following shortcuts:")
box.append(lbl)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary variable.

Suggested change
lbl = Gtk.Label(label="An application wants to register the following shortcuts:")
box.append(lbl)
box.append(Gtk.Label(label="An application wants to register the following shortcuts:"))

class GlobalShortcutsPortal:
def __init__(self, app):
self.app = app
self.sessions = {} # session_handle -> { 'app_id': str, 'shortcuts': dict, 'sender': str }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we need a comment with the structure, might as well use type hints.

Comment on lines 206 to 212
# [CHANGE] Added sender argument
def CreateSession(self, request_handle, session_handle, app_id, options, sender):
logger.info(f"CreateSession: {session_handle} for {app_id} (Sender: {sender})")
self.sessions[session_handle] = {
'app_id': app_id,
'shortcuts': {},
'sender': sender # [CHANGE] Store the sender
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftovers from when I was vibing this

Suggested change
# [CHANGE] Added sender argument
def CreateSession(self, request_handle, session_handle, app_id, options, sender):
logger.info(f"CreateSession: {session_handle} for {app_id} (Sender: {sender})")
self.sessions[session_handle] = {
'app_id': app_id,
'shortcuts': {},
'sender': sender # [CHANGE] Store the sender
def CreateSession(self, request_handle, session_handle, app_id, options, sender):
logger.info(f"CreateSession: {session_handle} for {app_id} (Sender: {sender})")
self.sessions[session_handle] = {
'app_id': app_id,
'shortcuts': {},
'sender': sender

from gi.repository import Gio, GLib

# XML for the interface we (the client) will expose to receive events
# [FIXED] Removed session_handle argument to match Gatekeeper's call signature (sua{sv})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# [FIXED] Removed session_handle argument to match Gatekeeper's call signature (sua{sv})

def on_client_method_call(self, connection, sender, object_path, interface_name, method_name, parameters, invocation):
# Handle the callbacks from Gatekeeper
if method_name == "Activated":
# [FIXED] Unpack 3 arguments: shortcut_id, timestamp, options
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# [FIXED] Unpack 3 arguments: shortcut_id, timestamp, options

invocation.return_value(None)

elif method_name == "Deactivated":
# [FIXED] Unpack 3 arguments
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# [FIXED] Unpack 3 arguments

def setup_client_listener(self):
# We simply register the object. Gatekeeper knows our bus unique name (e.g. :1.99)
# because it captured it when we called CreateSession.
session_path = "/org/freedesktop/portal/desktop/session/test_client/1"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably means we can't have multiple test clients?

Comment on lines 85 to 124
/// Tracks keyboard state shared among all keyboard event filters
class KeyboardStateTracker
{
public:
KeyboardStateTracker() = default;

/// Update pressed keysyms on key down
void on_key_down(uint32_t keysym);

/// Update pressed keysyms on key up
void on_key_up(uint32_t keysym);

/// Check if a keysym exists in the pressed set
auto keysym_is_pressed(uint32_t keysym, bool case_insensitive) const -> bool;

/// Check if protocol modifiers match event modifiers
static auto protocol_and_event_modifiers_match(uint32_t protocol_modifiers, MirInputEventModifiers event_mods)
-> bool;

private:
std::unordered_set<uint32_t> pressed_keysyms;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if the comments being in a different style is reason enough to change them...

@tarek-y-ismail
Copy link
Contributor Author

Ok, I think this is ready to review now. One thing that could be improved is that the data inside InputTriggerData could be encapsulated better instead of just being public, but that's a minor thing in my opinion.

@tarek-y-ismail tarek-y-ismail marked this pull request as ready for review January 27, 2026 16:25
@tarek-y-ismail tarek-y-ismail requested a review from a team as a code owner January 27, 2026 16:25
Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK as a throwaway spike: it works without too much pain.

But for maintainable code that we want to land:

  1. There are apparently spurious changes to xkb_mapper;
  2. poor encapsulation of logic; and,
  3. a failure to leverage the threading mode.

Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is called "a spike", and spike are intended for learning, not for production. In my experience (and that of others), it is usually better to start afresh with the knowledge learnt rather than try to remove all trace of the learning process. You usually end up with better results faster.

Practically speaking, 76 commits will be a PITA if ever bisecting a problem with these changes. Can we, at least, rework the history to remove dead experiments? (It doesn't really need more than three, and one wouldn't be terrible.)

There are no tests of any of this: There are clearly elements that can be tested in isolation (KeyboardStateTracker & BoundedQueue for a start).

PS

I didn't get my head fully around this yet, so don't assume I'm content with things I've not commented on

@tarek-y-ismail
Copy link
Contributor Author

This branch is called "a spike", and spike are intended for learning, not for production. In my experience (and that of others), it is usually better to start afresh with the knowledge learnt rather than try to remove all trace of the learning process. You usually end up with better results faster.

I may have gotten slightly attached to the code. But I believe that even if I started fresh, the final outcome would be very similar. (This is my inexperience speaking, I know)

Practically speaking, 76 commits will be a PITA if ever bisecting a problem with these changes. Can we, at least, rework the history to remove dead experiments? (It doesn't really need more than three, and one wouldn't be terrible.)

That was always my intention once I got the thumbs up 👍, as usual, I stop rebasing once others start reviewing my code.

There are no tests of any of this: There are clearly elements that can be tested in isolation (KeyboardStateTracker & BoundedQueue for a start).

Fair point. Just making sure the overall picture is okay. I originally intended on testing this mainly in WLCS in an upcoming PR, but as you pointed out, there are things I could also test here.

PS

I didn't get my head fully around this yet, so don't assume I'm content with things I've not commented on

Understandable. Will fix the things you pointed out and wait for more feedback.

@AlanGriffiths
Copy link
Contributor

Fair point. Just making sure the overall picture is okay. I originally intended on testing this mainly in WLCS in an upcoming PR, but as you pointed out, there are things I could also test here.

That's the difference in mindset between writing a spike and writing production code: a spike explores the solutions space and clarifies the picture, for production code we write tests of the behaviour required to complete that picture and ensure we write code to pass them.

/// Check if this trigger matches the current keyboard state
virtual auto matches(std::shared_ptr<KeyboardStateTracker> const& keyboard_state, MirInputEventModifiers event_mods) const -> bool = 0;

MirInputEventModifiers const modifiers;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's worth going through the code and making this keysym, and scancode private. They are const so I don't see a strong reason to make them private. Opinions welcome.

Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More nitpicking

keyboard_trigger->send_done_event();
}

// TODO: Store the description string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the TODO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a comment earlier here asking whether or not we actually want to store the description. So far I can't think of a use.

@AlanGriffiths
Copy link
Contributor

AlanGriffiths commented Feb 4, 2026

I wonder why this PR is failing mir_performance_tests in TICS

Copy link
Contributor Author

@tarek-y-ismail tarek-y-ismail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too much cruft. Will continue reviewing tomorrow.

@tarek-y-ismail tarek-y-ismail marked this pull request as draft February 4, 2026 16:18
@tarek-y-ismail tarek-y-ismail force-pushed the input-triggers-spike branch 2 times, most recently from 6090fca to 2cc77c9 Compare February 5, 2026 16:39
@tarek-y-ismail
Copy link
Contributor Author

tarek-y-ismail commented Feb 5, 2026

I know why the tests fail. It's because of the protocol and mir modifiers being out of sync and being implicitly convertible between each other. Will push some code tomorrow to add new types to guard against this.

Edit: I may have slipped and "accidentally" slapped it together real quick 🤫

@tarek-y-ismail tarek-y-ismail force-pushed the input-triggers-spike branch 2 times, most recently from ad39cbb to 61feb6e Compare February 5, 2026 18:12
@tarek-y-ismail tarek-y-ismail marked this pull request as ready for review February 5, 2026 19:18
Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking a lot cleaner. Some high level nitpicking, will look in more detail next week

if (!case_insensitive)
return pressed_keysyms.contains(keysym);

// Only perform case mapping for ASCII letters. For other keysyms, fall back to exact match.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually know the character set is ASCII? What if we're running on EBCDIC or Unicode16?

Maybe the intent was "Latin 1"?

Comment on lines +8 to +14
# Override the parent's fallback component with our own
# Use COMPILE_OPTIONS to undefine first, then redefine
target_compile_options(mirfrontend-wayland-input-trigger
PRIVATE
-UMIR_LOG_COMPONENT_FALLBACK # Undefine the definition we get from mirserver
-DMIR_LOG_COMPONENT_FALLBACK="input-triggers"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems very specific naming.

Elsewhere we have:

  • MIR_LOG_COMPONENT_FALLBACK="mircommon";
  • MIR_LOG_COMPONENT_FALLBACK="mirplatform"; and,
  • MIR_LOG_COMPONENT_FALLBACK="mirserver"

Why does this one protocol within the Wayland frontend qualify as a component in its own right?

I could see MIR_LOG_COMPONENT_FALLBACK="mirserver/frontend", or similar, for ALL the frontend code, but this seems unnecessary granularity.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a separate folder for each protocol? We've not done that before

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[non blocking] I wonder why this shows as a file move? It is just a file delete & new.

Comment on lines +161 to +168
if (keysym >= XKB_KEY_A && keysym <= XKB_KEY_Z)
{
lower = keysym + (XKB_KEY_a - XKB_KEY_A);
}
else if (keysym >= XKB_KEY_a && keysym <= XKB_KEY_z)
{
upper = keysym - (XKB_KEY_a - XKB_KEY_A);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arithmetic like this is heavily dependent on the character representation: e.g. in EBCDIC this will not work as intended

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants