Message compression for non-latin languages #161

New Issue

2026-10-05T02:00:55+02:00

adam commented

2026-10-05 02:00:55 +02:00

When using non-latin encodings one char eats 2x of bytes. So even not that large messages in, for example, Cyrillic rarely fits in one message.
Can we have optional live-replacement for chars while typing? This should be user-adjustable.

for example this transcoding table will achieve about 20% compression by replacing cyrillic letters with same looking latin letters:

{
 "А": "A",
 "В": "B",
 "Е": "E",
 "Ё": "E",
 "З": "3",
 "К": "K",
 "М": "M",
 "Н": "H",
 "О": "O",
 "Р": "P",
 "С": "C",
 "Т": "T",
 "Х": "X",
 "Ь": "b",
 "а": "a",
 "е": "e",
 "ё": "e",
 "о": "o",
 "р": "p",
 "с": "c",
 "у": "y",
 "х": "x"
}

When using non-latin encodings one char eats 2x of bytes. So even not that large messages in, for example, Cyrillic rarely fits in one message. Can we have optional live-replacement for chars while typing? This should be user-adjustable. for example this transcoding table will achieve about 20% compression by replacing cyrillic letters with same looking latin letters: ``` { "А": "A", "В": "B", "Е": "E", "Ё": "E", "З": "3", "К": "K", "М": "M", "Н": "H", "О": "O", "Р": "P", "С": "C", "Т": "T", "Х": "X", "Ь": "b", "а": "a", "е": "e", "ё": "e", "о": "o", "р": "p", "с": "c", "у": "y", "х": "x" } ```

adam closed this issue

2026-10-05 02:00:55 +02:00

adam commented

2026-10-05 02:00:56 +02:00

I don't think that's something that belongs within a radio operations terminal, no, sorry. Input management or glyph mutation should be the responsibility of the user; I'm sure that there exists keyboard layouts or remappings that accomplish this, and inline compression that relies on munging language or characters is not really a feature that's a good fit here. Apologies!

I don't think that's something that belongs within a radio operations terminal, no, sorry. Input management or glyph mutation should be the responsibility of the user; I'm sure that there exists keyboard layouts or remappings that accomplish this, and inline compression that relies on munging language or characters is not really a feature that's a good fit here. Apologies!

adam commented

2026-10-05 02:00:57 +02:00

see also same feature in another app with more explanation and sample realization https://github.com/zjs81/meshcore-open/pull/400

see also same feature in another app with more explanation and sample realization https://github.com/zjs81/meshcore-open/pull/400

adam commented

2026-10-05 02:00:57 +02:00

i can try to code this myself with agents, maybe we can have some plugin system to not poison main repository with features that not fit for most people?
this is really big problem for non-english locales, actual message is two times shorter than for english speaking users in the worst case.

i can try to code this myself with agents, maybe we can have some plugin system to not poison main repository with features that not fit for most people? this is really big problem for non-english locales, actual message is two times shorter than for english speaking users in the worst case.

adam commented

2026-10-05 02:00:58 +02:00

i can try to code this myself with agents, maybe we can have some plugin system to not poison main repository with features that not fit for most people? this is really big problem for non-english locales, actual message is two times shorter than for english speaking users in the worst case.

i will try to vibecode too. I think we need new settings page with json upload, and new choose vocab button for every channel.
Or maybe Plugin page, ooh
IDK

> i can try to code this myself with agents, maybe we can have some plugin system to not poison main repository with features that not fit for most people? this is really big problem for non-english locales, actual message is two times shorter than for english speaking users in the worst case. i will try to vibecode too. I think we need new settings page with json upload, and new choose vocab button for every channel. Or maybe Plugin page, ooh IDK

adam commented

2026-10-05 02:00:59 +02:00

I get where you're coming from -- I totally get how it would be useful. However, I don't want input management to be part of a radio messaging application, ya know? There are Chrome extensions and keyboard remaps and a bunch of other solutions that don't involve managing input optimization and all the headaches that come with that. It's the same reason I'm not adding an emoji picker; that's not a job that a radio app should be doing.

I get where you're coming from -- I totally get how it would be useful. However, I don't want input management to be part of a radio messaging application, ya know? There are Chrome extensions and keyboard remaps and a bunch of other solutions that don't involve managing input optimization and all the headaches that come with that. It's the same reason I'm not adding an emoji picker; that's not a job that a radio app should be doing.

adam commented

2026-10-05 02:00:59 +02:00

I get where you're coming from -- I totally get how it would be useful. However, I don't want input management to be part of a radio messaging application, ya know? There are Chrome extensions and keyboard remaps and a bunch of other solutions that don't involve managing input optimization and all the headaches that come with that. It's the same reason I'm not adding an emoji picker; that's not a job that a radio app should be doing.

its okay, thanks u for this project)

> I get where you're coming from -- I totally get how it would be useful. However, I don't want input management to be part of a radio messaging application, ya know? There are Chrome extensions and keyboard remaps and a bunch of other solutions that don't involve managing input optimization and all the headaches that come with that. It's the same reason I'm not adding an emoji picker; that's not a job that a radio app should be doing. its okay, thanks u for this project)

adam commented

2026-10-05 02:01:00 +02:00

Sorry that's probably not the answer you wanted 🙁 Every feature built incurs a debt to the future for upkeep and troubleshooting, and as this is growing ever nearer to feature completeness, it's important to me to build things that have a sustainable maintenance load. Thanks for understanding!

Sorry that's probably not the answer you wanted 🙁 Every feature built incurs a debt to the future for upkeep and troubleshooting, and as this is growing ever nearer to feature completeness, it's important to me to build things that have a sustainable maintenance load. Thanks for understanding!

adam commented

2026-10-05 02:01:01 +02:00

i'm not sure if i understand your position completely, so i will try to figure this out:

i proposed to add this as input hook because message size calculation occurs while input is ongoing, so i thought this is the best place for such replacement. but if message's maximum byte calculation can keep in mind future replacement on send this will be very nice too. or, last resort, maybe we can just add dumb standalone pre-send external hook if we can have user plugins in the future?

i'm not sure if i understand your position completely, so i will try to figure this out: i proposed to add this as input hook because message size calculation occurs while input is ongoing, so i thought this is the best place for such replacement. but if message's maximum byte calculation can keep in mind future replacement on send this will be very nice too. or, last resort, maybe we can just add dumb standalone pre-send external hook if we can have user plugins in the future?

adam commented

2026-10-05 02:01:01 +02:00

So I was GOING to say: my objection is not that it's technically difficult (simple character replace in the text box as typed when the setting is enabled), but that the fundamental idea (language alteration/input management) is not something that I think should be within the purview of a radio messaging app in the first place.

However, I've had an additional person reach out privately to ask for this since, so against my better judgement, I'll add it haha. Gimme a bit.

So I was GOING to say: my objection is not that it's technically difficult (simple character replace in the text box as typed when the setting is enabled), but that the fundamental idea (language alteration/input management) is not something that I think should be within the purview of a radio messaging app in the first place. However, I've had an additional person reach out privately to ask for this since, so against my better judgement, I'll add it haha. Gimme a bit.

adam commented

2026-10-05 02:01:02 +02:00

adam commented

2026-10-05 02:01:03 +02:00

it actually works much better than even expected. very nice! thanks! 🥇 👍

it actually works much better than even expected. very nice! thanks! 🥇 👍

adam commented

2026-10-05 02:01:03 +02:00

really, the replacement experience is so smooth and natural, it almost feels absolutely natural, thanks again, very appreciated

really, the replacement experience is so smooth and natural, it almost feels absolutely natural, thanks again, very appreciated

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/Remote-Terminal-for-MeshCore#161