bug(common/models): LM stops working on a fresh install

[MakaraSok]

Describe the bug

With a fresh installed of Keyman for Android, after installing Khmer Angkor keyboard and NC Khmer Wordlist LM, start typing in the text area ស្រ to see no suggestion on the banner.

Expected behavior

The LM should give the expected/relevant suggestions on the banner.

Screenshots

---

Keyman for Android:

• Device: Pixel 2 (AS emulator)
• OS: Android 10
• Keyman version 14.0.271
• Target application: Keyman

---

Keyboard

• Keyboard name: khmer_angkor
• Keyboard version: 1.0.9
• Language name: Khmer
• LM source: [sil.km.nckw] NC Khmer Wordlist lexical-models#131

[MakaraSok]

The behavior is rather unpredictable. I experienced that the error would not happen right of the gate. The suggestions might keep showing after typing ស្រ . The LM would crash or stop giving the suggestions if the same sequence is typed over and again after choosing from the suggestions.

i.e. keep typing ស្រ and choose ស្រី (the first option) until the banner turns blank. See below. It turns blank after three rounds for me.

[jahorton]

It's worth noting that errors will generally be a lot more likely for models targeting languages without proper word-breaking implementations. Our support for them is simply lacking until we're able to implement #5025.

If this can be shown to also affect models for languages with space-based wordbreaking, that would give this issue higher priority.