Connection issues with the node and the WebSocket #164

New Issue

2026-10-05T02:01:09+02:00

adam commented

2026-10-05 02:01:09 +02:00

The system has a hard time connecting the first time, and I get the feeling that it only connects if I set MESHCORE_DISABLE_BOTS=true. Here's what I see in the log:

2026-04-26 14:38:44 - app.radio - INFO - Attempting to reconnect to radio...
2026-04-26 14:38:44 - app.keystore - INFO - Cleared in-memory keystore
2026-04-26 14:38:44 - app.radio - INFO - Radio reconnected successfully at TCP: 192.168.1.123:5000
2026-04-26 14:38:44 - app.event_handlers - INFO - Event handlers registered
2026-04-26 14:38:44 - app.keystore - INFO - Exporting private key from radio...
2026-04-26 14:38:44 - app.keystore - INFO - Private key stored in keystore (public key: nnnnnnn...)
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Applied flood_scope='(disabled)'
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Path hash mode: 0 (supported)
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Radio device info: model=Heltec V3 build=19 Apr 2026 version=v1.15.0 max_contacts=350 max_channels=40
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Radio clock at connect: epoch=1777214324 utc=2026-04-26 14:38:44 UTC
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Max channel slots: 40
2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Syncing and offloading radio data...
2026-04-26 14:38:54 - app.radio_sync - ERROR - Failed to get contacts from radio: None. If you see this repeatedly, the radio may be visible on the serial/TCP/BLE port but not responding to commands. Check for another process with the serial port open (other RemoteTerm instances, serial monitors, etc.), verify the firmware is up-to-date and in client mode (not repeater), or try a power cycle.

What strikes me as odd is that it can reach the node just fine, it’s able to retrieve its public key, model, and version. And if you manage to connect, it seems to work just fine.

Another issue I've encountered is that when I put nginx in front of it, I see log lines like this:

2026-04-26 14:59:13 - uvicorn.access - INFO - 10.88.0.1:49488 - "GET /api/ws HTTP/1.1" 404 Not Found

The system has a hard time connecting the first time, and I get the feeling that it only connects if I set `MESHCORE_DISABLE_BOTS=true`. Here's what I see in the log: ``` 2026-04-26 14:38:44 - app.radio - INFO - Attempting to reconnect to radio... 2026-04-26 14:38:44 - app.keystore - INFO - Cleared in-memory keystore 2026-04-26 14:38:44 - app.radio - INFO - Radio reconnected successfully at TCP: 192.168.1.123:5000 2026-04-26 14:38:44 - app.event_handlers - INFO - Event handlers registered 2026-04-26 14:38:44 - app.keystore - INFO - Exporting private key from radio... 2026-04-26 14:38:44 - app.keystore - INFO - Private key stored in keystore (public key: nnnnnnn...) 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Applied flood_scope='(disabled)' 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Path hash mode: 0 (supported) 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Radio device info: model=Heltec V3 build=19 Apr 2026 version=v1.15.0 max_contacts=350 max_channels=40 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Radio clock at connect: epoch=1777214324 utc=2026-04-26 14:38:44 UTC 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Max channel slots: 40 2026-04-26 14:38:44 - app.services.radio_lifecycle - INFO - Syncing and offloading radio data... 2026-04-26 14:38:54 - app.radio_sync - ERROR - Failed to get contacts from radio: None. If you see this repeatedly, the radio may be visible on the serial/TCP/BLE port but not responding to commands. Check for another process with the serial port open (other RemoteTerm instances, serial monitors, etc.), verify the firmware is up-to-date and in client mode (not repeater), or try a power cycle. ``` What strikes me as odd is that it can reach the node just fine, it’s able to retrieve its public key, model, and version. And if you manage to connect, it seems to work just fine. Another issue I've encountered is that when I put nginx in front of it, I see log lines like this: ``` 2026-04-26 14:59:13 - uvicorn.access - INFO - 10.88.0.1:49488 - "GET /api/ws HTTP/1.1" 404 Not Found ```

adam closed this issue

2026-10-05 02:01:10 +02:00

adam commented

2026-10-05 02:01:10 +02:00

@ogarcia probably you need to use upgrade for ws endpoint https://nginx.org/en/docs/http/websocket.html

@ogarcia probably you need to use upgrade for ws endpoint https://nginx.org/en/docs/http/websocket.html

adam commented

2026-10-05 02:01:11 +02:00

@ogarcia probably you need to use upgrade for ws endpoint https://nginx.org/en/docs/http/websocket.html

Yeah, I was just checking that part, and I hadn't even realized that setting was there. And sure enough, it's working fine now.

As for the connection issues, the problem is still there, but I set MESHCORE_LOAD_WITH_AUTOEVICT=true, and it seems to be going faster; the log shows the same thing.

2026-04-26 15:40:11 - uvicorn.access - INFO - 10.88.0.1:47448 - "GET /api/radio/config HTTP/1.1" 503 Service Unavailable
2026-04-26 15:40:16 - app.radio_sync - ERROR - Failed to get contacts from radio: None. If you see this repeatedly, the radio may be visible on the serial/TCP/BLE port but not responding to commands. Check for another process with the serial port open (other RemoteTerm instances, serial monitors, etc.), verify the firmware is up-to-date and in client mode (not repeater), or try a power cycle.

A little while later, when it finally manages to connect:

2026-04-26 15:41:06 - uvicorn.access - INFO - 10.88.0.1:48082 - "GET /api/radio/config HTTP/1.1" 200 OK
2026-04-26 15:41:10 - app.radio_sync - INFO - Background contact blind fill complete: refreshed 40 desired contacts
2026-04-26 15:41:10 - app.radio_sync - INFO - Background contact reconcile summary: removed 0, loaded 40, failed 0

> [@ogarcia](https://github.com/ogarcia) probably you need to use upgrade for ws endpoint https://nginx.org/en/docs/http/websocket.html Yeah, I was just checking that part, and I hadn't even realized that setting was there. And sure enough, it's working fine now. As for the connection issues, the problem is still there, but I set `MESHCORE_LOAD_WITH_AUTOEVICT=true`, and it seems to be going faster; the log shows the same thing. ``` 2026-04-26 15:40:11 - uvicorn.access - INFO - 10.88.0.1:47448 - "GET /api/radio/config HTTP/1.1" 503 Service Unavailable 2026-04-26 15:40:16 - app.radio_sync - ERROR - Failed to get contacts from radio: None. If you see this repeatedly, the radio may be visible on the serial/TCP/BLE port but not responding to commands. Check for another process with the serial port open (other RemoteTerm instances, serial monitors, etc.), verify the firmware is up-to-date and in client mode (not repeater), or try a power cycle. ``` A little while later, when it finally manages to connect: ``` 2026-04-26 15:41:06 - uvicorn.access - INFO - 10.88.0.1:48082 - "GET /api/radio/config HTTP/1.1" 200 OK 2026-04-26 15:41:10 - app.radio_sync - INFO - Background contact blind fill complete: refreshed 40 desired contacts 2026-04-26 15:41:10 - app.radio_sync - INFO - Background contact reconcile summary: removed 0, loaded 40, failed 0 ```

adam commented

2026-10-05 02:01:12 +02:00

Hmm disabling bots shouldn't have any impact on radio connection. I'm not sure how radios handle plural TCP connections but I would imagine not gracefully; are you sure there are no other clients attempting to connect to it? load with autoevict is maybe correlated; that does reduce the number of transactions needed to load contacts so if the issue is connection contention with another device (which would be my leading theory), reducing transaction volume would be consistent with a better success rate

Hmm disabling bots shouldn't have any impact on radio connection. I'm not sure how radios handle plural TCP connections but I would imagine not gracefully; are you sure there are no other clients attempting to connect to it? load with autoevict is maybe correlated; that does reduce the number of transactions needed to load contacts so if the issue is connection contention with another device (which would be my leading theory), reducing transaction volume would be consistent with a better success rate

adam commented

2026-10-05 02:01:12 +02:00

No, disabling the bots was more of a hunch on my part than anything else; after testing it more thoroughly, I saw that it wasn't having any effect. As for the load with auto-evict, it does seem to improve things somewhat.

Ultimately, it seems to have a hard time getting started, but then it runs smoothly. Every now and then it disconnects like this:

2026-04-26 19:12:59 - app.services.radio_lifecycle - WARNING - Radio connection lost, broadcasting status change
2026-04-26 19:12:59 - app.radio - INFO - Attempting to reconnect to radio...
2026-04-26 19:12:59 - app.radio - INFO - Radio reconnected successfully at TCP: 192.168.1.161:5000

but it reconnects and runs fine.

I don't know, maybe it's all due to the nature of the project and the stability of the firmware with Wi-Fi (which is still in its early stages).

By the way, before I forget, congratulations! This project is wonderful.

No, disabling the bots was more of a hunch on my part than anything else; after testing it more thoroughly, I saw that it wasn't having any effect. As for the load with auto-evict, it does seem to improve things somewhat. Ultimately, it seems to have a hard time getting started, but then it runs smoothly. Every now and then it disconnects like this: ``` 2026-04-26 19:12:59 - app.services.radio_lifecycle - WARNING - Radio connection lost, broadcasting status change 2026-04-26 19:12:59 - app.radio - INFO - Attempting to reconnect to radio... 2026-04-26 19:12:59 - app.radio - INFO - Radio reconnected successfully at TCP: 192.168.1.161:5000 ``` but it reconnects and runs fine. I don't know, maybe it's all due to the nature of the project and the stability of the firmware with Wi-Fi (which is still in its early stages). By the way, before I forget, congratulations! This project is wonderful.

adam commented

2026-10-05 02:01:13 +02:00

I don't know, maybe it's all due to the nature of the project and the stability of the firmware with Wi-Fi

I really hate handwaving/blame-the-upstream but most of the issues I've seen and been able to replicate have been with TCP connections. I do wonder if it's just asking a bit much of the ESP32 (I could be wildly wrong here; I have no load data to back that up with). But I know in personal projects, wifi + TCP stack was grindy.

I'm glad to hear at least once it gets past a rocky start, it works reasonably well. I wish I had more I could offer in terms of troubleshooting/diagnostics, I'm sorry 🙁

congratulations! This project is wonderful.

you're too kind, especially to say so in a "hey this is just generally unstable" support request haha ♥ thank you though

> I don't know, maybe it's all due to the nature of the project and the stability of the firmware with Wi-Fi I really hate handwaving/blame-the-upstream but most of the issues I've seen and been able to replicate have been with TCP connections. I do wonder if it's just asking a bit much of the ESP32 (I could be wildly wrong here; I have no load data to back that up with). But I know in personal projects, wifi + TCP stack was grindy. I'm glad to hear at least once it gets past a rocky start, it works reasonably well. I wish I had more I could offer in terms of troubleshooting/diagnostics, I'm sorry 🙁 > congratulations! This project is wonderful. you're too kind, especially to say so in a "hey this is just generally unstable" support request haha ♥ thank you though

adam commented

2026-10-05 02:01:14 +02:00

you're too kind, especially to say so in a "hey this is just generally unstable" support request haha ♥ thank you though

Not at all, the project is actually really cool.

As for the issue, I was mostly wondering if it was something already known or if there was some kind of configuration “trick” that made it work one way or another. But as I said, it probably all comes down to the fact that we're dealing with firmware in its early stages that still has a long way to go.

As for the ESP32, you shouldn't have any problems. Keep in mind that it works well in Meshtastic (which is more or less similar). Obviously, you have the same limitations (one node, one client) but it's quite stable.

All in all, I think the best thing to do is give it some time; it will surely get better little by little.

I'm closing this issue because, ultimately, the problem seems to be more of an upstream issue, and there's not much we can do about it here. Thank you very much!

> you're too kind, especially to say so in a "hey this is just generally unstable" support request haha ♥ thank you though Not at all, the project is actually really cool. As for the issue, I was mostly wondering if it was something already known or if there was some kind of configuration “trick” that made it work one way or another. But as I said, it probably all comes down to the fact that we're dealing with firmware in its early stages that still has a long way to go. As for the ESP32, you shouldn't have any problems. Keep in mind that it works well in Meshtastic (which is more or less similar). Obviously, you have the same limitations (one node, one client) but it's quite stable. All in all, I think the best thing to do is give it some time; it will surely get better little by little. I'm closing this issue because, ultimately, the problem seems to be more of an upstream issue, and there's not much we can do about it here. Thank you very much!

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/Remote-Terminal-for-MeshCore#164