UTF-8 transliteration for unsupported characters (e.g., Latin Extended) #307

New Issue

2026-10-27T15:48:46+01:00

adam commented

2026-10-27 15:48:46 +01:00

I would like to suggest a lightweight workaround for an issue regarding messages containing UTF-8 special characters (like European diacritics, e.g., Polish ą, ć, ł, ż). Currently, when a message contains these characters, they are rendered as question marks or artifacts on the screen (for example, "żółty" renders as "?o?ty") because the default font lacks extended Unicode I suppose.

Instead of implementing heavy multi-byte parsing and expanding the font arrays, a very practical will be to implement a simple UTF-8 transliteration (sanitization) function just before the text is sent to the display, by doing a simple string replacement of 2-byte UTF-8 sequences into their closest 1-byte standard ASCII equivalents (e.g., ą -> a, ł -> l).

I think this quite simple implementation would be a massive quality-of-life improvement for international users. :)

I would like to suggest a lightweight workaround for an issue regarding messages containing UTF-8 special characters (like European diacritics, e.g., Polish ą, ć, ł, ż). Currently, when a message contains these characters, they are rendered as question marks or artifacts on the screen (for example, "żółty" renders as "?o?ty") because the default font lacks extended Unicode I suppose. Instead of implementing heavy multi-byte parsing and expanding the font arrays, a very practical will be to implement a simple UTF-8 transliteration (sanitization) function just before the text is sent to the display, by doing a simple string replacement of 2-byte UTF-8 sequences into their closest 1-byte standard ASCII equivalents (e.g., `ą` -> `a`,` ł` -> `l`). I think this quite simple implementation would be a massive quality-of-life improvement for international users. :)

adam commented

2026-10-27 15:48:47 +01:00

I strongly second this. Transliterate Latin characters with diacritics to their non-latin counterparts as long as thei are not in the font. Except polish, there are other languages which use (more or less heavily) decorated latin characters.

And extend the font to contain more and more non-ASCII characters over time.

Some actually are in the font already (e.g. ä, Ä, ö, Ö, ü, Ü, ß). And cyrillic seems(?) also to be in place.

Regards!

I strongly second this. Transliterate Latin characters with diacritics to their non-latin counterparts as long as thei are not in the font. Except polish, there are other languages which use (more or less heavily) decorated latin characters. And extend the font to contain more and more non-ASCII characters over time. Some actually _are_ in the font already (e.g. `ä`, `Ä`, `ö`, `Ö`, `ü`, `Ü`, `ß`). And cyrillic seems(?) also to be in place. Regards!

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/meshcore#307