Skip to content

Fix Unicode in string decoding#22

Open
darthwalsh wants to merge 2 commits intodownforacross:masterfrom
darthwalsh:utf8
Open

Fix Unicode in string decoding#22
darthwalsh wants to merge 2 commits intodownforacross:masterfrom
darthwalsh:utf8

Conversation

@darthwalsh
Copy link

@darthwalsh darthwalsh commented Jul 15, 2025

Current code works only for ASCII, but creates garbled bytes for any unicode.
Use TextDecoder and TextEncoder which are broadly supported: "Baseline Widely available."
Sniff the copyright string to determine correct encoding when decoding.

For some more context, I asked ChatGPT to explain what was wrong.

Validation

With e.g. https://www.nytimes.com/crosswords/game/daily/2025/07/13
64 Across should be: "Phrase cooed en español"
But with the bug puzjs returns: "Phrase cooed en español"

My PUZ file worked fine at https://squares.io/s/jy5a5wyg

Now correctly parses both:
- BrandonKoppyWillShortz-NYTimesSundayJuly132025TunnelVision.puz
- DerrickNiedermanWillShortz-NYTimesSundaySeptember72025TheNameIsTheGame.puz

Current code works only for ASCII, but creates garbled bytes for any unicode.

Today with e.g. https://www.nytimes.com/crosswords/game/daily/2025/07/13
64 Across should be: "Phrase cooed en español"
But with the bug it shows: "Phrase cooed en español"

My PUZ file worked fine at https://squares.io/s/jy5a5wyg
darthwalsh added a commit to darthwalsh/CrossWordy that referenced this pull request Jul 24, 2025
Using jsdelivr github CDN until downforacross/puzjs#22 is merged
@darthwalsh darthwalsh marked this pull request as draft November 12, 2025 20:36
@darthwalsh
Copy link
Author

darthwalsh commented Nov 12, 2025

Changing the status to draft: The PUZ spec shouldn't allow UTF-8, and really should be ISO-8859-1. Instead, the code could sniff the copyright string for the © symbol which is encoded differently in ISO-8859-1

Validation: Correctly parses both:
- BrandonKoppyWillShortz-NYTimesSundayJuly132025TunnelVision.puz
- DerrickNiedermanWillShortz-NYTimesSundaySeptember72025TheNameIsTheGame.puz
@darthwalsh darthwalsh marked this pull request as ready for review November 25, 2025 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant