adam/meshview
1
0 Fork 0
mirror of https://github.com/pablorevilla-meshtastic/meshview.git synced 2026-10-31 03:01:58 +01:00
Code Issues 20 Packages Projects Releases Wiki Activity

Crashing for last two days #14

New Issue
Closed
opened 2026-10-05 06:01:58 +02:00 by adam · 15 comments
adam commented 2026-10-05 06:01:58 +02:00
Owner
Copy link
Copy source

Hi,
For some reason my instance started to crash with this error. First time I experience this issue I was using docker container. To mitigate docker related issues I decided to run it directly both on my user account and root. Every time I try I get this error. Has anyone had such issue?

──(format㉿ubuntu-dom)-[~/meshview]
└─$ ./env/bin/python startdb.py
2025-10-06 11:45:23 startdb.py:55 [pid:2587064] INFO - Next cleanup scheduled at 2025-10-07 02:00:00
2025-10-06 11:45:23 mqtt_reader.py:53 [pid:2587064] INFO - Connected to MQTT broker at loranet.pl:1883
2025-10-06 11:45:23 mqtt_reader.py:55 [pid:2587064] INFO - Subscribing to: msh/PL/#
2025-10-06 11:45:23 mqtt_reader.py:55 [pid:2587064] INFO - Subscribing to: msh/#

  • Exception Group Traceback (most recent call last):
    | File "/home/format/meshview/startdb.py", line 172, in
    | asyncio.run(main())
    | File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    | return runner.run(main)
    | ^^^^^^^^^^^^^^^^
    | File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    | return self._loop.run_until_complete(task)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    | return future.result()
    | ^^^^^^^^^^^^^^^
    | File "/home/format/meshview/startdb.py", line 151, in main
    | async with asyncio.TaskGroup() as tg:
    | File "/usr/lib/python3.12/asyncio/taskgroups.py", line 145, in aexit
    | raise me from None
    | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
    +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    | File "/home/format/meshview/startdb.py", line 130, in load_database_from_mqtt
    | await mqtt_store.process_envelope(topic, env)
    | File "/home/format/meshview/meshview/mqtt_store.py", line 102, in process_envelope
    | PacketSeen.node_id == int(env.gateway_id[1:], 16),
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | ValueError: invalid literal for int() with base 16: ''
    +------------------------------------
Hi, For some reason my instance started to crash with this error. First time I experience this issue I was using docker container. To mitigate docker related issues I decided to run it directly both on my user account and root. Every time I try I get this error. Has anyone had such issue? > ──(format㉿ubuntu-dom)-[~/meshview] > └─$ ./env/bin/python startdb.py > 2025-10-06 11:45:23 startdb.py:55 [pid:2587064] INFO - Next cleanup scheduled at 2025-10-07 02:00:00 > 2025-10-06 11:45:23 mqtt_reader.py:53 [pid:2587064] INFO - Connected to MQTT broker at loranet.pl:1883 > 2025-10-06 11:45:23 mqtt_reader.py:55 [pid:2587064] INFO - Subscribing to: msh/PL/# > 2025-10-06 11:45:23 mqtt_reader.py:55 [pid:2587064] INFO - Subscribing to: msh/# > + Exception Group Traceback (most recent call last): > | File "/home/format/meshview/startdb.py", line 172, in <module> > | asyncio.run(main()) > | File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run > | return runner.run(main) > | ^^^^^^^^^^^^^^^^ > | File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run > | return self._loop.run_until_complete(task) > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > | File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete > | return future.result() > | ^^^^^^^^^^^^^^^ > | File "/home/format/meshview/startdb.py", line 151, in main > | async with asyncio.TaskGroup() as tg: > | File "/usr/lib/python3.12/asyncio/taskgroups.py", line 145, in __aexit__ > | raise me from None > | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) > +-+---------------- 1 ---------------- > | Traceback (most recent call last): > | File "/home/format/meshview/startdb.py", line 130, in load_database_from_mqtt > | await mqtt_store.process_envelope(topic, env) > | File "/home/format/meshview/meshview/mqtt_store.py", line 102, in process_envelope > | PacketSeen.node_id == int(env.gateway_id[1:], 16), > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > | ValueError: invalid literal for int() with base 16: '' > +------------------------------------ > <!-- github-sync-id:3486463811 -->
adam closed this issue 2026-10-05 06:01:58 +02:00
adam commented 2026-10-05 06:01:59 +02:00
Author
Owner
Copy link
Copy source

can you download the latest and test again... we have been making changes to mqtt_strore in the last couple of days.

can you download the latest and test again... we have been making changes to mqtt_strore in the last couple of days. <!-- github-comment-id:3374253722 -->
adam commented 2026-10-05 06:01:59 +02:00
Author
Owner
Copy link
Copy source

I did it yesterday but double-checked this right now:

┌──(format㉿ubuntu-dom)-[~/meshview]
└─$ git pull
Already up to date.

Maybe there are some new modules required?

I did it yesterday but double-checked this right now: > ┌──(format㉿ubuntu-dom)-[~/meshview] > └─$ git pull > Already up to date. Maybe there are some new modules required? <!-- github-comment-id:3375376371 -->
adam commented 2026-10-05 06:02:00 +02:00
Author
Owner
Copy link
Copy source

@formiification can you share your config? it's having trouble reading a gatewayid on your mqtt.

we ought to be able to recreate.

@formiification can you share your config? it's having trouble reading a gatewayid on your mqtt. we ought to be able to recreate. <!-- github-comment-id:3379382492 -->
adam commented 2026-10-05 06:02:01 +02:00
Author
Owner
Copy link
Copy source

I was able to recreate using these settings:
https://loranet.pl/mqtt

we are getting empty gateway_id packets.

It looks like an invalid MQTT sender somewhere.

Gateway ID: !849ade64
Gateway ID: !db29e764
Gateway ID: !3369245c
Gateway ID: !1fa05360
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
Gateway ID: !db2b1c94
WARNING: Missing gateway_id, skipping PacketSeen entry
Gateway ID: !9ea19fa0
Gateway ID: !ea8f42b8
Gateway ID: !db296b84

I was able to recreate using these settings: https://loranet.pl/mqtt we are getting empty gateway_id packets. It looks like an invalid MQTT sender somewhere. Gateway ID: !849ade64 Gateway ID: !db29e764 Gateway ID: !3369245c Gateway ID: !1fa05360 WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry Gateway ID: !db2b1c94 WARNING: Missing gateway_id, skipping PacketSeen entry Gateway ID: !9ea19fa0 Gateway ID: !ea8f42b8 Gateway ID: !db296b84 <!-- github-comment-id:3379414490 -->
adam commented 2026-10-05 06:02:01 +02:00
Author
Owner
Copy link
Copy source

See https://discord.com/channels/1215705285159817236/1291089743903457364/1424551871120543745 and the messages after for the cause (already "fixed")

See https://discord.com/channels/1215705285159817236/1291089743903457364/1424551871120543745 and the messages after for the cause (already "fixed") <!-- github-comment-id:3379587095 -->
adam commented 2026-10-05 06:02:02 +02:00
Author
Owner
Copy link
Copy source

@prokrypt unsure. I was able to re-pro the crash when looking at nodes on the polish mqtt server.
we want to gracefully handle missing data anyway, so I think this PR still stands as useful.

@prokrypt unsure. I was able to re-pro the crash when looking at nodes on the polish mqtt server. we want to gracefully handle missing data anyway, so I think this PR still stands as useful. <!-- github-comment-id:3379732889 -->
adam commented 2026-10-05 06:02:03 +02:00
Author
Owner
Copy link
Copy source

We just merge PR#69 that should solve the problem with a missing gatewayID on a packet.

We just merge PR#69 that should solve the problem with a missing gatewayID on a packet. <!-- github-comment-id:3383736969 -->
adam commented 2026-10-05 06:02:03 +02:00
Author
Owner
Copy link
Copy source

Thank you Joel!!

Thank you Joel!! <!-- github-comment-id:3383739503 -->
adam commented 2026-10-05 06:02:04 +02:00
Author
Owner
Copy link
Copy source

@formiification
Can you validate that it is working for you now?

@formiification Can you validate that it is working for you now? <!-- github-comment-id:3387053809 -->
adam commented 2026-10-05 06:02:05 +02:00
Author
Owner
Copy link
Copy source

@formiification Can you validate that it is working for you now?

Looking much better. Thanks!

┌──(format㉿ubuntu-dom)-[~/meshview]
└─$ git pull 1 ⨯
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 5 (delta 3), reused 4 (delta 3), pack-reused 0 (from 0)
Unpacking objects: 100% (5/5), 1.39 KiB | 355.00 KiB/s, done.
From https://github.com/pablorevilla-meshtastic/meshview
30ba603..d7b830e master -> origin/master
Updating 30ba603..d7b830e
Fast-forward
meshview/mqtt_store.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

┌──(format㉿ubuntu-dom)-[~/meshview]
└─$ ./env/bin/python startdb.py
2025-10-09 20:29:15 startdb.py:55 [pid:586677] INFO - Next cleanup scheduled at 2025-10-10 02:00:00
2025-10-09 20:29:15 mqtt_reader.py:53 [pid:586677] INFO - Connected to MQTT broker at loranet.pl:1883
2025-10-09 20:29:15 mqtt_reader.py:55 [pid:586677] INFO - Subscribing to: msh/PL/#
2025-10-09 20:29:15 mqtt_reader.py:55 [pid:586677] INFO - Subscribing to: msh/#
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry
WARNING: Missing gateway_id, skipping PacketSeen entry

> [@formiification](https://github.com/formiification) Can you validate that it is working for you now? Looking much better. Thanks! ┌──(format㉿ubuntu-dom)-[~/meshview] └─$ git pull 1 ⨯ remote: Enumerating objects: 8, done. remote: Counting objects: 100% (8/8), done. remote: Compressing objects: 100% (2/2), done. remote: Total 5 (delta 3), reused 4 (delta 3), pack-reused 0 (from 0) Unpacking objects: 100% (5/5), 1.39 KiB | 355.00 KiB/s, done. From https://github.com/pablorevilla-meshtastic/meshview 30ba603..d7b830e master -> origin/master Updating 30ba603..d7b830e Fast-forward meshview/mqtt_store.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) ┌──(format㉿ubuntu-dom)-[~/meshview] └─$ ./env/bin/python startdb.py 2025-10-09 20:29:15 startdb.py:55 [pid:586677] INFO - Next cleanup scheduled at 2025-10-10 02:00:00 2025-10-09 20:29:15 mqtt_reader.py:53 [pid:586677] INFO - Connected to MQTT broker at loranet.pl:1883 2025-10-09 20:29:15 mqtt_reader.py:55 [pid:586677] INFO - Subscribing to: msh/PL/# 2025-10-09 20:29:15 mqtt_reader.py:55 [pid:586677] INFO - Subscribing to: msh/# WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry WARNING: Missing gateway_id, skipping PacketSeen entry <!-- github-comment-id:3387059748 -->
adam commented 2026-10-05 06:02:05 +02:00
Author
Owner
Copy link
Copy source

should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not.

should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not. <!-- github-comment-id:3387098733 -->
adam commented 2026-10-05 06:02:06 +02:00
Author
Owner
Copy link
Copy source

Doesn't matter in my opinion. In my setup it goes to /dev/null anyway

Doesn't matter in my opinion. In my setup it goes to /dev/null anyway <!-- github-comment-id:3387107334 -->
adam commented 2026-10-05 06:02:07 +02:00
Author
Owner
Copy link
Copy source

should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not.

Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s)

> should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not. Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s) <!-- github-comment-id:3387109265 -->
adam commented 2026-10-05 06:02:07 +02:00
Author
Owner
Copy link
Copy source

should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not.

Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s)

I think they're masking the main useful information (the gateway id is /who/ sent it, right?)

> > should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not. > > Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s) I think they're masking the main useful information (the gateway id is /who/ sent it, right?) <!-- github-comment-id:3387149709 -->
adam commented 2026-10-05 06:02:08 +02:00
Author
Owner
Copy link
Copy source

should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not.

Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s)

I think they're masking the main useful information (the gateway id is /who/ sent it, right?)

Indeed. https://github.com/meshtastic/firmware/issues/8228

> > > should we remove that warning? I think it means someone publishing to your MQTT is sending "bad" data. not sure if it helps to log it or not. > > > > > > Would be best to include a timestamp as well as the packet id and/or other details (from? hops?), to help track down the offender(s) > > I think they're masking the main useful information (the gateway id is /who/ sent it, right?) Indeed. https://github.com/meshtastic/firmware/issues/8228 <!-- github-comment-id:3387278098 -->
Sign in to join this conversation.
No Branch/Tag Specified
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
adam (Adam Wojcieszonek)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: adam/meshview#14