[FR] Spaceless scripts word tokenization #6
Description
I propose using the Intl.Segmenter API, as it is now supported in most modern browsers. This approach has several advantages:
- Fewer dependencies: It eliminates the need for limited APIs like Google or Azure.
- No NLP setup required: Simplifies our development process by avoiding complex natural language processing configurations.
related to #4