feat(web): accepting a suggestion then adding punctuation should delete automatically added space 🚂

[mcdurdin]

Is your feature request related to a problem? Please describe.

From a team review of the Keyman for Android UX (#7161)

When using a lexical model that inserts spaces after an accepted suggestion, it's annoying to have to press backspace before adding punctuation such as full stop.

Describe the solution you'd like

Ideally, delete the space before inserting the punctuation, then re-insert the space after the punctuation.

Unfortunately, this is probably not trivial to solve -- the keyboard itself does not know that the LM inserted the space, so it's not really possible right now to do a special case for punctuation in the keyboard code. Likewise, the LM has no control when the punctuation is typed, so it can't automatically delete the space in this situation.

Furthermore, this is language-specific behaviour: not all LMs insert spaces; some insert ZWSP, or nothing at all after a suggestion.

Per @jahorton: Will need predictive-text design work to properly addressed - pretty sure it’ll need language-specific (or script-specific) data to be defined on the lexical model.

Related issues

---

Keyman for Windows/macOS/Linux/iPhone/iPad/Android:

• Device: Samsung A90 5G
• OS: Android 12
• Keyman version: 15.0.269-stable

[jahorton]

I think some of the adjustments (and notes for future work) I have as part of #7205 may be helpful for this. See my notes starting here:

keyman/common/web/lm-worker/src/model-compositor.ts

Lines 159 to 177 in 6c50de9

	/* TODO: consider/implement: the second half of the comment above.
	* For example: on input of a `'`, predict new words instead of replacing the `'`.
	* (since after a letter, the `'` will be ignored, anyway)
	*
	* Idea: if the model's most likely prediction (with no root) would make a new
	* token if appended to the current token, that's probably a good case.
	* Keeps the check simple & quick.
	*
	* Might need a mixed mode, though: ';' is close enough that `l` is a reasonable
	* fat-finger guess. So yeah, we're not addressing this idea right now.
	* - so... consider multiple context behavior angles when building prediction roots?
	*
	* May need something similar to help handle contractions during their construction,
	* but that'd be within `ContextTracker`.
	* can' => [`can`, `'`]
	* can't => [`can't`] (WB6, 7 of https://unicode.org/reports/tr29/#Word_Boundary_Rules)
	*
	* (Would also helps WB7b+c for Hebrew text)
	*/

A similar strategy could be used to detect if we'd tokenize without the space there - "if so, why add a word-breaking space"?

[rowbory]

In NaijaType I built in rules to my keyboard to address this with the most common punctuation: ,.!? Could it be a suggested block of code that keyboard-makers could add to their keyboards and customise as needed?

[mcdurdin]

@rowbory, yes this is a good idea. We'll consider this as part of any future solution.

[ermshiperete]

What if the LM would insert a kind of dead-key that gets ignored when a punctuation key follows and turns into space or whatever when any other key is typed?

[mcdurdin]

Yeah, some special marker might be a way forward? I'd include the space with the suggestion, but then have the following keyboard pseudocode:

accept() > 'word' dk(__suggestion) ' '    c matching what is inserted by the LM

dk(__suggestion) ' ' + any(punct) > index(punct, 3)

[MakaraSok]

This looks like a small issue, but, from a user point of view, it's a turn off. It is funny to press a backspace every single time a punctuation is added after a word is picked of the banner.

If there is a voting system for this to be fixed/added, I'd like to vote for this one. Can we have this done soon as it was first posted two years ago?