Unclean shutdown when running as systemd service #112

Closed
opened 2026-10-05 01:23:17 +02:00 by adam · 2 comments
Owner

While investigating why my leaderboard was never saved, it appears when running as a service the process is simply killed and the exit_handler never runs.

https://github.com/SpudGunMan/meshing-around/blob/3273e57f0bb88e70ac5dd84999b5ce5578cdc61e/etc/mesh_bot.tmp#L16C10-L16C30

ExecStop=pkill -f mesh_bot.py
systemd[1]: Stopping mesh_bot.service - MESH-BOT...
systemd[1]: mesh_bot.service: Killing process 9470 (python3) with signal SIGKILL.
systemd[1]: mesh_bot.service: Deactivated successfully.
systemd[1]: Stopped mesh_bot.service - MESH-BOT.
systemd[1]: mesh_bot.service: Consumed 13.923s CPU time.

Replacing the ExecStop line with KillSignal=SIGINT allowed the handler to run, but there may be more preferable ways to achieve this.

bash[10479]: 2025-11-06 09:26:58,425 |    DEBUG | System: Scheduler loop cancelled, shutting down.
bash[10479]: 2025-11-06 09:26:58,426 |    DEBUG | System: Cleaning up async tasks
systemd[1]: Stopping mesh_bot.service - MESH-BOT...
bash[10479]: 2025-11-06 09:26:58,427 |    DEBUG | System: Closing Autoresponder
bash[10479]: 2025-11-06 09:26:58,428 |    DEBUG | System: Closing Interface1
bash[10479]: 2025-11-06 09:26:59,445 |  WARNING | System: Abrupt Disconnection of Interface detected
bash[10479]: Unexpected error in deferred execution <class 'BrokenPipeError'>
bash[10479]: 2025-11-06 09:26:59,446 |    DEBUG | System: Saving data/bbsdb.pkl
bash[10479]: 2025-11-06 09:26:59,449 |    DEBUG | System: Saving Updated BBS Direct Messages data/bbsdm.pkl
bash[10479]: 2025-11-06 09:26:59,451 |    DEBUG | System: BBS Messages Saved
bash[10479]: 2025-11-06 09:26:59,453 |    DEBUG | System: Mesh Leaderboard saved to leaderboard.pkl
bash[10479]: 2025-11-06 09:26:59,453 |    DEBUG | System: Exiting
systemd[1]: mesh_bot.service: Deactivated successfully.
systemd[1]: Stopped mesh_bot.service - MESH-BOT.
systemd[1]: mesh_bot.service: Consumed 2.784s CPU time.
While investigating why my leaderboard was never saved, it appears when running as a service the process is simply killed and the [exit_handler](https://github.com/SpudGunMan/meshing-around/blob/main/modules/system.py#L2380-L2402) never runs. https://github.com/SpudGunMan/meshing-around/blob/3273e57f0bb88e70ac5dd84999b5ce5578cdc61e/etc/mesh_bot.tmp#L16C10-L16C30 ``` ExecStop=pkill -f mesh_bot.py ``` ``` systemd[1]: Stopping mesh_bot.service - MESH-BOT... systemd[1]: mesh_bot.service: Killing process 9470 (python3) with signal SIGKILL. systemd[1]: mesh_bot.service: Deactivated successfully. systemd[1]: Stopped mesh_bot.service - MESH-BOT. systemd[1]: mesh_bot.service: Consumed 13.923s CPU time. ``` Replacing the ExecStop line with `KillSignal=SIGINT` allowed the handler to run, but there may be more preferable ways to achieve this. ``` bash[10479]: 2025-11-06 09:26:58,425 | DEBUG | System: Scheduler loop cancelled, shutting down. bash[10479]: 2025-11-06 09:26:58,426 | DEBUG | System: Cleaning up async tasks systemd[1]: Stopping mesh_bot.service - MESH-BOT... bash[10479]: 2025-11-06 09:26:58,427 | DEBUG | System: Closing Autoresponder bash[10479]: 2025-11-06 09:26:58,428 | DEBUG | System: Closing Interface1 bash[10479]: 2025-11-06 09:26:59,445 | WARNING | System: Abrupt Disconnection of Interface detected bash[10479]: Unexpected error in deferred execution <class 'BrokenPipeError'> bash[10479]: 2025-11-06 09:26:59,446 | DEBUG | System: Saving data/bbsdb.pkl bash[10479]: 2025-11-06 09:26:59,449 | DEBUG | System: Saving Updated BBS Direct Messages data/bbsdm.pkl bash[10479]: 2025-11-06 09:26:59,451 | DEBUG | System: BBS Messages Saved bash[10479]: 2025-11-06 09:26:59,453 | DEBUG | System: Mesh Leaderboard saved to leaderboard.pkl bash[10479]: 2025-11-06 09:26:59,453 | DEBUG | System: Exiting systemd[1]: mesh_bot.service: Deactivated successfully. systemd[1]: Stopped mesh_bot.service - MESH-BOT. systemd[1]: mesh_bot.service: Consumed 2.784s CPU time. ``` <!-- github-sync-id:3596070882 -->
adam closed this issue 2026-10-05 01:23:17 +02:00
Author
Owner

Thank you!

Thank you! <!-- github-comment-id:3497655102 -->
Author
Owner

Thanks for the quick fix! The bot has been great so far.

Thanks for the quick fix! The bot has been great so far. <!-- github-comment-id:3497676876 -->
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: adam/meshing-around#112