change(web): remove computeAlignment and older transform-tokenization methods 🚂#14882
Merged
jahorton merged 4 commits intoepic/autocorrectfrom Oct 20, 2025
Conversation
User Test ResultsTest specification and instructions User tests are not required |
github-actions
bot
added
web/
web/predictive-text/
change
keymanapp-test-bot
bot
changed the title
jahorton
force-pushed
the
refactor/web/delayed-reversion-via-tokenization-analysis
branch
from
3541216 to
79abb65
Compare
jahorton
force-pushed
the
change/web/remove-old-alignment-and-transform-tokenization
branch
from
5c35290 to
dc48191
Compare
jahorton
force-pushed
the
refactor/web/delayed-reversion-via-tokenization-analysis
branch
from
79abb65 to
aa477a9
Compare
The removed methods did not properly support cases adjacent to 'split' / 'merge' edits.
While handy, this method did not adequately account for 'split' / 'merge' edit cases and also required special handling to avoid certain degenerate edit-path cases. The newer method put in place (see #14874) handles such edits and avoids the degeneracy problem that resulted when relying on edit paths for the edited portion of context. Build-bot: skip build:web Test-bot: skip
jahorton
force-pushed
the
change/web/remove-old-alignment-and-transform-tokenization
branch
from
824de55 to
a3f1a89
Compare
…s' into change/web/remove-old-alignment-and-transform-tokenization
ermshiperete
approved these changes
Oct 17, 2025
Base automatically changed from
refactor/web/delayed-reversion-via-tokenization-analysis
to
epic/autocorrect
October 20, 2025 13:36
jahorton
deleted the
change/web/remove-old-alignment-and-transform-tokenization
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For all removed methods, note that they did not properly support cases adjacent to 'split' / 'merge' edits. New pathways heavily supporting the same functionality in a more future-forward manner were added in the following PRs:
.insertstrings on the retokenized contextcomputeAlignmentandtokenizeTransformin one, partly through use of helpers in PRs listed aboveWhile handy, especially while transitioning to the new context-management pattern, the
computeAlignmentmethod (refactored during epic/autocorrect, #14430 and relocated in #14573) did not adequately account for 'split' / 'merge' edit cases and also required special handling to avoid certain degenerate edit-path cases. The newer method put in place (by #14874) addresses both issues at once and looks to provide a more robust way forward while also optimizing the range of context that is retokenized for each potential correction, as well as the number of retokenization calls needed per correction.Relevant unit-test cases have largely been migrated and found new homes within the test suites for methods added in the PRs above. It wouldn't hurt to review the cases and ensure this holds true, though. (I personally should take an extra pass through them before undrafting this PR.)
Build-bot: skip build:web
Test-bot: skip