Commit Graph

79 Commits

Author SHA1 Message Date
KenADev
625df7982d feat: Mesh-Ingestor: Ability to provide already-existing interface instance (#395)
* feat: Mesh-Ingestor: Ability to provide already-existing interface instance

* Prevent Signal-Registration if not main thread (causes exception)

* fix redundant ternary operator

---------

Co-authored-by: Ken Ahr <ken.a.iphone@googlemail.com>
2025-10-26 20:47:23 +01:00
KenADev
8eeb13166b fix: Ingestor: Fix error for non-existing datetime.UTC reference (#396)
Co-authored-by: Ken Ahr <ken.a.iphone@googlemail.com>
2025-10-26 20:46:31 +01:00
l5y
8143fbd8f7 Add full support to telemetry schema and API (#385)
* feat: auto-upgrade telemetry schema

* Ensure numeric metrics fallback to valid values

* Format data processing numeric metric lookup
2025-10-18 15:19:33 +02:00
l5y
01665b6e3a Fixes POST request 403 errors on instances behind Cloudflare proxy (#368)
* Add full headers to ingestor POST requests to avoid CF bans

* run black

* Guard Authorization header when token absent

---------

Co-authored-by: varna9000 <milen@aeroisk.com>
2025-10-16 22:29:04 +02:00
l5y
4bfc0e25cb Prefer reported primary channel names (#363) 2025-10-16 20:35:24 +02:00
l5y
76b57c08c6 Fix ingestor reconnection detection (#361) 2025-10-16 13:06:32 +02:00
l5y
a46da284e5 Fix ingestor package layout in Docker image (#336) 2025-10-14 18:47:54 +02:00
l5y
03b5a10fe4 Add LoRa metadata fields to nodes and messages (#331)
* Add LoRa metadata fields to nodes and messages

* Filter numeric SQLite keys from message rows
2025-10-14 14:51:28 +02:00
l5y
e97498d09f Add channel metadata capture for message tagging (#329) 2025-10-13 23:10:01 +02:00
l5y
7db76ec2fc Capture radio metadata for ingestor payloads (#327)
* Capture radio metadata and tag ingestor payloads

* Log captured LoRa metadata when initializing radio config
2025-10-13 22:35:06 +02:00
l5y
5133e9d498 refactor: streamline ingestor environment variables (#314)
* refactor: streamline ingestor environment variables

* fix: set connection env var in docker test
2025-10-13 11:02:33 +02:00
l5y
772c5888c3 Fix ingestor debug timestamps for structured logging (#296) 2025-10-12 09:40:57 +02:00
l5y
9e939194ba Update workflows for ingestor, sinatra, and frontend (#295) 2025-10-12 09:36:02 +02:00
l5y
1316d4f2d1 Clarify network target parsing (#285) 2025-10-11 20:38:40 +02:00
l5y
9be390ee09 Ensure queue deactivates when empty (#284) 2025-10-11 20:38:27 +02:00
l5y
4ef1e29034 Fix federation signature handling and IP guard (#276)
* Fix federation signature handling and IP guard

* Avoid defaulting isPrivate before signature verification

* Normalize instance domain host handling for restricted IP check

* ignore web app credentials

---------

Co-authored-by: l5yth <d220195275+l5yth@users.noreply.github.com>
2025-10-11 17:39:52 +02:00
l5y
ab9ae796f3 Add hourly reconnect handling for inactive mesh interface (#267)
* Add hourly reconnect handling for inactive mesh interface

* Reset inactivity timestamp after reconnect
2025-10-07 16:41:16 +02:00
l5y
f2ed5f5c03 Add energy saving mode to ingestor (#256) 2025-10-07 15:28:41 +02:00
l5y
db04b85134 chore: restore apache headers (#260) 2025-10-07 10:28:24 +02:00
l5y
a5a2ae5edc Document mesh ingestor modules with PDoc-style docstrings (#255) 2025-10-07 08:59:38 +02:00
l5y
363b4c5525 Handle missing node IDs in Meshtastic nodeinfo packets (#251)
* Handle Meshtastic nodeinfo packets without IDs

* Guard BLE reader against missing payloads
2025-10-07 08:56:36 +02:00
l5y
6969ae6c4a Fix mesh ingestor telemetry and neighbor handling (#249)
* Refine mesh ingestor modularization

* Handle script execution in mesh wrapper

* Ensure mesh shim finds package when run as script

* Ensure queue state resets after send errors
2025-10-07 08:40:28 +02:00
l5y
5dfcc1a5fe docs: add ingestor inline docstrings (#244) 2025-10-07 00:06:42 +02:00
l5y
09a2d849ec Improve live node positions and expose precision metadata (#231)
* Fetch latest node positions and precision metadata

* Stop showing position source and precision in UI

* Guard node positions against stale merges
2025-10-05 23:08:57 +02:00
l5y
78168ce3db Handle Bluetooth shutdown hangs gracefully (#221)
* Handle Bluetooth shutdown hangs gracefully

* Make interface close guard compatible with patched Event
2025-10-05 21:07:19 +02:00
l5y
332abbc183 Adjust mesh priorities and receive topics (#220) 2025-10-05 20:50:34 +02:00
l5y
c136c5cf26 Add BLE and fallback mesh interface handling (#219)
* Add BLE and fallback mesh interface support

* Handle SIGINT by propagating KeyboardInterrupt

* Guard optional BLE dependency

* run black
2025-10-05 20:48:23 +02:00
l5y
2a65e89eee Add neighbor info ingestion and API endpoints (#218)
* Add neighbor info ingestion and API support

* Fix neighbor spec and add fixture

* run black

* run rufo
2025-10-05 12:35:13 +02:00
l5y
bb4cbfa62c Improve debug logging for node and telemetry data (#213)
* Improve debug logging for node and telemetry data

* run black
2025-10-04 21:03:03 +02:00
l5y
f0d600e5d7 Improve stored message debug logging (#211) 2025-10-04 20:53:54 +02:00
l5y
e0f0a6390d Stop repeating ingestor node info snapshot and timestamp debug logs (#210)
* Adjust ingestor node snapshot cadence and debug logging

* Ensure node snapshot waits for data

* run black
2025-10-04 20:41:53 +02:00
l5y
d4a27dccf7 Add telemetry API and ingestion support (#205)
* Add telemetry ingestion and API support

* Flatten telemetry storage and API responses

* Fix telemetry insert placeholder count

* Adjust telemetry node updates

* run black

* run rufo
2025-10-04 18:28:18 +02:00
l5y
4548f750d3 Add connection recovery for TCP interface (#186)
* Add connection recovery for TCP interface

* run black
2025-09-27 18:52:56 +02:00
l5y
66389dd27c Handle plain IP addresses in mesh TCP detection (#154)
* Fix TCP target detection for plain IPs

* run black
2025-09-26 13:25:42 +02:00
l5y
ee6501243f Handle encrypted messages (#173)
* Handle encrypted messages

* Remove redundant message node columns

* Preserve original numeric message senders

* Normalize message sender IDs in API responses

* Exclude encrypted messages from API responses

* run rufo
2025-09-24 07:34:28 +02:00
l5y
a46bed1c33 Handle proto values in nodeinfo payloads (#167) 2025-09-23 18:31:22 +02:00
l5y
d711300442 Remove raw payload storage from database schema (#166) 2025-09-23 17:29:08 +02:00
l5y
98a8203591 Add POSITION_APP ingestion and API support (#160)
* Add POSITION_APP ingestion and API support

* Adjust mesh receive subscriptions and priorities

* run linters
2025-09-23 16:42:51 +02:00
l5y
084c5ae158 Add support for NODEINFO_APP packets (#159)
* Add support for NODEINFO_APP packets

* run black
2025-09-23 14:40:35 +02:00
l5y
5a610cf08a Support mock serial interface in CI (#143) 2025-09-21 10:00:30 +02:00
l5y
a4256cee83 fix: retain runtime libs for alpine production (#138) 2025-09-21 09:18:55 +02:00
l5y
89f0b1bcfe fix: support windows ingestor build (#136)
* fix: support windows ingestor build

* fix: restore alpine build deps for ingestor (#137)
2025-09-20 22:00:45 +02:00
Taylor Rose
812d3c851f feat: Add comprehensive Docker support (#122)
* feat: Add comprehensive Docker support

- Add multi-container Docker setup with web app and data ingestor
- Create production-ready Dockerfiles with multi-stage builds
- Add Docker Compose configurations for dev, prod, and custom environments
- Implement CI/CD pipeline with GitHub Actions for automated builds
- Add comprehensive Docker documentation and setup guides
- Include security scanning and multi-platform builds
- Support for Meshtastic device integration via serial access
- Persistent data storage with named volumes
- Health checks and monitoring capabilities

Addresses GitHub issue #120: Dockerize the project for easier community adoption

Files added:
- web/Dockerfile: Ruby web application container
- data/Dockerfile: Python data ingestor container
- data/requirements.txt: Python dependencies
- docker-compose.yml: Base Docker Compose configuration
- docker-compose.dev.yml: Development environment overrides
- docker-compose.prod.yml: Production environment overrides
- .env.example: Environment configuration template
- .dockerignore: Docker build context optimization
- .github/workflows/docker.yml: CI/CD pipeline
- DOCKER.md: Comprehensive Docker documentation

This implementation transforms PotatoMesh from a complex manual setup
to a single-command deployment: docker-compose up -d

* feat: Add Docker support with multi-architecture builds

- Add web/Dockerfile with Ruby 3.4 Alpine base
- Add data/Dockerfile with Python 3.13 Alpine base
- Use Alpine's SQLite3 packages for cross-platform compatibility
- Support AMD64, ARM64, ARMv7, and Windows architectures
- Multi-stage builds for optimized production images
- Non-root user security and proper file permissions

* feat: Add Docker Compose configurations for different environments

- docker-compose.yml: Production setup with GHCR images
- docker-compose.dev.yml: Development setup with local builds
- docker-compose.raspberry-pi.yml: Pi-optimized with resource limits
- Support for all architectures (AMD64, ARM64, ARMv7)
- Proper volume mounts and network configuration
- Environment variable configuration for different deployments

* feat: Add GitHub Actions workflows for Docker CI/CD

- docker.yml: Multi-architecture build and push to GHCR
- test-raspberry-pi-hardware.yml: ARM64 testing with QEMU
- Support for manual workflow dispatch with version input
- Build and test all Docker variants (AMD64, ARM64, ARMv7, Windows)
- Automated publishing to GitHub Container Registry
- Comprehensive testing for Raspberry Pi deployments

* feat: Add Docker documentation and configuration tools

- docs/DOCKER.md: Comprehensive Docker setup and usage guide
- configure.sh: Interactive configuration script for deployment
- Platform-specific setup instructions (macOS, Linux, Windows)
- Raspberry Pi optimization guidelines
- Environment variable configuration
- Troubleshooting and best practices

* docs: Update README with comprehensive Docker support

- Add Docker Quick Start section with published images
- Add comprehensive table of all available GHCR images
- Include architecture-specific pull commands
- Update manual installation instructions
- Add platform-specific deployment examples
- Document all supported architectures and use cases

* chore: Update dependencies and project configuration

- Update data/requirements.txt for Python 3.13 compatibility
- Add v0.3.0 changelog entry documenting Docker support
- Update .gitignore for Docker-related files
- Prepare project for Docker deployment

* feat: Update web interface for Denver Mesh Network

- Update default configuration to center on Denver, Colorado
- Set SITE_NAME to 'Denver Mesh Network'
- Configure 915MHz frequency for US region
- Update map center coordinates (39.7392, -104.9903)
- Set appropriate node distance and Matrix room settings

* Update Docker configuration and documentation

- Remove Raspberry Pi specific Docker files and workflows
- Update Docker workflow configuration
- Consolidate Docker documentation
- Add AGENTS.md for opencode integration
- Update README with current project status

* cleanup: workflow/readme

* Update README.md

Co-authored-by: l5y <220195275+l5yth@users.noreply.github.com>

* Add .env.example and simplify documentation

- Add comprehensive .env.example with all environment variables
- Update web Dockerfile to use Berlin coordinates instead of Denver
- Simplify README Docker quick start with helpful comments
- Greatly simplify DOCKER.md with only essential information

* cleanup: readme

* Remove Stadia API key references

- Remove STADIA_API_KEY from docker-compose.yml environment variables
- Remove Stadia Maps configuration section from configure.sh
- Remove Stadia API key references from .env.example
- Simplify configuration to use basic OpenStreetMap tiles only

* quickfix

* cleanup: remove example usage from docker gh action output

---------

Co-authored-by: l5y <220195275+l5yth@users.noreply.github.com>
2025-09-20 21:04:19 +02:00
l5y
671a910936 Adjust python ingestor interval to 60 seconds (#112) 2025-09-16 21:07:53 +02:00
l5y
cffdb7dca6 Prioritize node posts in queued API updates (#107)
* Prioritize node posts in queued API updates

* run black
2025-09-16 19:30:38 +02:00
l5y
3edf60c625 chore: clean up repository (#96)
* chore: clean up repository

* Fix message spec node lookup for numeric IDs (#98)

* Fix message spec node lookup for numeric IDs

* run rufo

* Fix message node fallback lookup (#99)
2025-09-16 15:25:12 +02:00
l5y
051d09dcaf Document Python and Ruby source modules (#94) 2025-09-16 13:13:12 +02:00
l5y
1c13b99f3b Skip null fields when choosing packet identifiers (#88) 2025-09-16 11:56:02 +02:00
l5y
54a1eb5b42 create python yml ga (#90)
* Create python.yml

* ci: add black

* run an actual formatter

* also add rufo

* fix pytest

* run black
2025-09-16 11:50:33 +02:00
l5y
542f4dd0e2 Handle concurrent node snapshot updates (#85) 2025-09-16 11:10:11 +02:00