forked from iarv/meshcore-hub
Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a1fb71ce65 | |||
| 6a5549081f | |||
| 68e24ee886 | |||
| 61d6b6287e | |||
| 7007c84577 | |||
| fd928d9fea | |||
| 68b6aa85cd | |||
| abbc07edb3 | |||
| b42add310e | |||
| 98a5526e80 |
@@ -16,7 +16,7 @@ jobs:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
@@ -39,7 +39,7 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11"]
|
||||
python-version: ["3.13"]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -60,7 +60,7 @@ jobs:
|
||||
|
||||
- name: Upload coverage to Codecov
|
||||
uses: codecov/codecov-action@v4
|
||||
if: matrix.python-version == '3.11'
|
||||
if: matrix.python-version == '3.13'
|
||||
with:
|
||||
files: ./coverage.xml
|
||||
fail_ci_if_error: false
|
||||
@@ -76,7 +76,7 @@ jobs:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install build tools
|
||||
run: |
|
||||
|
||||
@@ -14,7 +14,7 @@ repos:
|
||||
rev: 24.3.0
|
||||
hooks:
|
||||
- id: black
|
||||
language_version: python3.11
|
||||
language_version: python3.13
|
||||
args: ["--line-length=88"]
|
||||
|
||||
- repo: https://github.com/pycqa/flake8
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
3.11
|
||||
3.13
|
||||
|
||||
@@ -18,7 +18,7 @@ This document provides context and guidelines for AI coding assistants working o
|
||||
|
||||
## Project Overview
|
||||
|
||||
MeshCore Hub is a Python 3.11+ monorepo for managing and orchestrating MeshCore mesh networks. It consists of five main components:
|
||||
MeshCore Hub is a Python 3.13+ monorepo for managing and orchestrating MeshCore mesh networks. It consists of five main components:
|
||||
|
||||
- **meshcore_interface**: Serial/USB interface to MeshCore companion nodes, publishes/subscribes to MQTT
|
||||
- **meshcore_collector**: Collects MeshCore events from MQTT and stores them in a database
|
||||
@@ -37,7 +37,7 @@ MeshCore Hub is a Python 3.11+ monorepo for managing and orchestrating MeshCore
|
||||
|
||||
| Category | Technology |
|
||||
|----------|------------|
|
||||
| Language | Python 3.11+ |
|
||||
| Language | Python 3.13+ |
|
||||
| Package Management | pip with pyproject.toml |
|
||||
| CLI Framework | Click |
|
||||
| Configuration | Pydantic Settings |
|
||||
@@ -545,6 +545,22 @@ When enabled, the collector automatically removes nodes where:
|
||||
|
||||
**Note:** Both event data and node cleanup run on the same schedule (DATA_RETENTION_INTERVAL_HOURS).
|
||||
|
||||
**Contact Cleanup (Interface RECEIVER):**
|
||||
|
||||
The interface RECEIVER mode can automatically remove stale contacts from the MeshCore companion node's contact database. This prevents the companion node from resyncing old/dead contacts back to the collector, freeing up memory on the device (typically limited to ~100 contacts).
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `CONTACT_CLEANUP_ENABLED` | Enable automatic removal of stale contacts (default: true) |
|
||||
| `CONTACT_CLEANUP_DAYS` | Remove contacts not advertised for this many days (default: 7) |
|
||||
|
||||
When enabled, during each contact sync the receiver checks each contact's `last_advert` timestamp:
|
||||
- Contacts with `last_advert` older than `CONTACT_CLEANUP_DAYS` are removed from the device
|
||||
- Stale contacts are not published to MQTT (preventing collector database pollution)
|
||||
- Contacts without a `last_advert` timestamp are preserved (no removal without data)
|
||||
|
||||
This cleanup runs automatically whenever the receiver syncs contacts (on startup and after each advertisement event).
|
||||
|
||||
Manual cleanup can be triggered at any time with:
|
||||
```bash
|
||||
# Dry run to see what would be deleted
|
||||
@@ -571,6 +587,10 @@ Webhook payload structure:
|
||||
2. **Database Migration Errors**: Ensure `DATA_HOME` is writable, run `meshcore-hub db upgrade`
|
||||
3. **Import Errors**: Ensure package is installed with `pip install -e .`
|
||||
4. **Type Errors**: Run `pre-commit run --all-files` to check type annotations and other issues
|
||||
5. **NixOS greenlet errors**: On NixOS, the pre-built greenlet wheel may fail with `libstdc++.so.6` errors. Rebuild from source:
|
||||
```bash
|
||||
pip install --no-binary greenlet greenlet
|
||||
```
|
||||
|
||||
### Debugging
|
||||
|
||||
|
||||
+2
-2
@@ -4,7 +4,7 @@
|
||||
# =============================================================================
|
||||
# Stage 1: Builder - Install dependencies and build package
|
||||
# =============================================================================
|
||||
FROM python:3.11-slim AS builder
|
||||
FROM python:3.13-slim AS builder
|
||||
|
||||
# Set environment variables
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
@@ -39,7 +39,7 @@ RUN sed -i "s|__version__ = \"dev\"|__version__ = \"${BUILD_VERSION}\"|" src/mes
|
||||
# =============================================================================
|
||||
# Stage 2: Runtime - Final production image
|
||||
# =============================================================================
|
||||
FROM python:3.11-slim AS runtime
|
||||
FROM python:3.13-slim AS runtime
|
||||
|
||||
# Labels
|
||||
LABEL org.opencontainers.image.title="MeshCore Hub" \
|
||||
|
||||
@@ -17,41 +17,45 @@ MeshCore Hub provides a complete solution for monitoring, collecting, and intera
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ MeshCore │ │ MeshCore │ │ MeshCore │
|
||||
│ Device 1 │ │ Device 2 │ │ Device 3 │
|
||||
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
|
||||
│ │ │
|
||||
│ Serial/USB │ Serial/USB │ Serial/USB
|
||||
│ │ │
|
||||
┌────────▼────────┐ ┌────────▼────────┐ ┌────────▼────────┐
|
||||
│ Interface │ │ Interface │ │ Interface │
|
||||
│ (RECEIVER) │ │ (RECEIVER) │ │ (SENDER) │
|
||||
└────────┬────────┘ └────────┬────────┘ └────────▲────────┘
|
||||
│ │ │
|
||||
│ Publish │ Publish │ Subscribe
|
||||
│ │ │
|
||||
└───────────┬───────────┴───────────────────────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ MQTT │
|
||||
│ Broker │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ Collector │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ Database │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
│ │
|
||||
┌──────▼──────┐ ┌───────▼───────┐
|
||||
│ API │◄──────│ Web Dashboard │
|
||||
└─────────────┘ └───────────────┘
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Devices["MeshCore Devices"]
|
||||
D1["Device 1"]
|
||||
D2["Device 2"]
|
||||
D3["Device 3"]
|
||||
end
|
||||
|
||||
subgraph Interfaces["Interface Layer"]
|
||||
I1["RECEIVER"]
|
||||
I2["RECEIVER"]
|
||||
I3["SENDER"]
|
||||
end
|
||||
|
||||
D1 -->|Serial| I1
|
||||
D2 -->|Serial| I2
|
||||
D3 -->|Serial| I3
|
||||
|
||||
I1 -->|Publish| MQTT
|
||||
I2 -->|Publish| MQTT
|
||||
MQTT -->|Subscribe| I3
|
||||
|
||||
MQTT["MQTT Broker"]
|
||||
|
||||
subgraph Backend["Backend Services"]
|
||||
Collector --> Database --> API
|
||||
end
|
||||
|
||||
MQTT --> Collector
|
||||
API --> Web["Web Dashboard"]
|
||||
|
||||
style Devices fill:none,stroke:#0288d1,stroke-width:2px
|
||||
style Interfaces fill:none,stroke:#f57c00,stroke-width:2px
|
||||
style Backend fill:none,stroke:#388e3c,stroke-width:2px
|
||||
style MQTT fill:none,stroke:#7b1fa2,stroke-width:3px
|
||||
style Collector fill:none,stroke:#388e3c,stroke-width:2px
|
||||
style Database fill:none,stroke:#c2185b,stroke-width:2px
|
||||
style API fill:none,stroke:#1976d2,stroke-width:2px
|
||||
style Web fill:none,stroke:#ffa000,stroke-width:2px
|
||||
```
|
||||
|
||||
## Features
|
||||
@@ -97,33 +101,34 @@ This starts all services: MQTT broker, collector, API, web dashboard, and the in
|
||||
|
||||
For larger deployments, you can separate receiver nodes from the central infrastructure. This allows multiple community members to contribute receiver coverage while hosting the backend centrally.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Community Members │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Raspberry Pi │ │ Raspberry Pi │ │ Any Linux │ │
|
||||
│ │ + MeshCore │ │ + MeshCore │ │ + MeshCore │ │
|
||||
│ │ Device │ │ Device │ │ Device │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
||||
│ │ │ │ │
|
||||
│ │ receiver profile only │ │
|
||||
│ └──────────────────┼──────────────────┘ │
|
||||
│ │ │
|
||||
│ MQTT (port 1883) │
|
||||
│ │ │
|
||||
└────────────────────────────┼─────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Community VPS / Server │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌──────────────┐ │
|
||||
│ │ MQTT │──▶│ Collector │──▶│ API │◀──│ Web Dashboard│ │
|
||||
│ │ Broker │ │ │ │ │ │ (public) │ │
|
||||
│ └──────────┘ └───────────┘ └─────────┘ └──────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Community["Community Members"]
|
||||
R1["Raspberry Pi + MeshCore"]
|
||||
R2["Raspberry Pi + MeshCore"]
|
||||
R3["Any Linux + MeshCore"]
|
||||
end
|
||||
|
||||
subgraph Server["Community VPS / Server"]
|
||||
MQTT["MQTT Broker"]
|
||||
Collector
|
||||
API
|
||||
Web["Web Dashboard (public)"]
|
||||
|
||||
MQTT --> Collector --> API
|
||||
API <--- Web
|
||||
end
|
||||
|
||||
R1 -->|MQTT port 1883| MQTT
|
||||
R2 -->|MQTT port 1883| MQTT
|
||||
R3 -->|MQTT port 1883| MQTT
|
||||
|
||||
style Community fill:none,stroke:#0288d1,stroke-width:2px
|
||||
style Server fill:none,stroke:#388e3c,stroke-width:2px
|
||||
style MQTT fill:none,stroke:#7b1fa2,stroke-width:3px
|
||||
style Collector fill:none,stroke:#388e3c,stroke-width:2px
|
||||
style API fill:none,stroke:#1976d2,stroke-width:2px
|
||||
style Web fill:none,stroke:#ffa000,stroke-width:2px
|
||||
```
|
||||
|
||||
**On each receiver node (Raspberry Pi, etc.):**
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 133 KiB After Width: | Height: | Size: 124 KiB |
+4
-5
@@ -8,7 +8,7 @@ version = "0.0.0"
|
||||
description = "Python monorepo for managing and orchestrating MeshCore mesh networks"
|
||||
readme = "README.md"
|
||||
license = {text = "GPL-3.0-or-later"}
|
||||
requires-python = ">=3.11"
|
||||
requires-python = ">=3.13"
|
||||
authors = [
|
||||
{name = "MeshCore Hub Contributors"}
|
||||
]
|
||||
@@ -18,8 +18,7 @@ classifiers = [
|
||||
"License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Topic :: Communications",
|
||||
"Topic :: System :: Networking",
|
||||
]
|
||||
@@ -78,7 +77,7 @@ meshcore_hub = ["py.typed"]
|
||||
|
||||
[tool.black]
|
||||
line-length = 88
|
||||
target-version = ["py311"]
|
||||
target-version = ["py312"]
|
||||
include = '\.pyi?$'
|
||||
extend-exclude = '''
|
||||
/(
|
||||
@@ -97,7 +96,7 @@ extend-exclude = '''
|
||||
'''
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.11"
|
||||
python_version = "3.13"
|
||||
warn_return_any = true
|
||||
warn_unused_ignores = true
|
||||
disallow_untyped_defs = true
|
||||
|
||||
@@ -78,6 +78,17 @@ class InterfaceSettings(CommonSettings):
|
||||
default=None, description="Device/node name (optional)"
|
||||
)
|
||||
|
||||
# Contact cleanup settings
|
||||
contact_cleanup_enabled: bool = Field(
|
||||
default=True,
|
||||
description="Enable automatic removal of stale contacts from companion node",
|
||||
)
|
||||
contact_cleanup_days: int = Field(
|
||||
default=7,
|
||||
description="Remove contacts not advertised for this many days",
|
||||
ge=1,
|
||||
)
|
||||
|
||||
|
||||
class CollectorSettings(CommonSettings):
|
||||
"""Settings for the Collector component."""
|
||||
|
||||
@@ -100,6 +100,19 @@ def interface() -> None:
|
||||
envvar="MQTT_TLS",
|
||||
help="Enable TLS/SSL for MQTT connection",
|
||||
)
|
||||
@click.option(
|
||||
"--contact-cleanup/--no-contact-cleanup",
|
||||
default=True,
|
||||
envvar="CONTACT_CLEANUP_ENABLED",
|
||||
help="Enable/disable automatic removal of stale contacts (RECEIVER mode only)",
|
||||
)
|
||||
@click.option(
|
||||
"--contact-cleanup-days",
|
||||
type=int,
|
||||
default=7,
|
||||
envvar="CONTACT_CLEANUP_DAYS",
|
||||
help="Remove contacts not advertised for this many days (RECEIVER mode only)",
|
||||
)
|
||||
@click.option(
|
||||
"--log-level",
|
||||
type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]),
|
||||
@@ -120,6 +133,8 @@ def run(
|
||||
mqtt_password: str | None,
|
||||
prefix: str,
|
||||
mqtt_tls: bool,
|
||||
contact_cleanup: bool,
|
||||
contact_cleanup_days: int,
|
||||
log_level: str,
|
||||
) -> None:
|
||||
"""Run the interface component.
|
||||
@@ -162,6 +177,8 @@ def run(
|
||||
mqtt_password=mqtt_password,
|
||||
mqtt_prefix=prefix,
|
||||
mqtt_tls=mqtt_tls,
|
||||
contact_cleanup_enabled=contact_cleanup,
|
||||
contact_cleanup_days=contact_cleanup_days,
|
||||
)
|
||||
elif mode_upper == "SENDER":
|
||||
from meshcore_hub.interface.sender import run_sender
|
||||
@@ -262,6 +279,19 @@ def run(
|
||||
envvar="MQTT_TLS",
|
||||
help="Enable TLS/SSL for MQTT connection",
|
||||
)
|
||||
@click.option(
|
||||
"--contact-cleanup/--no-contact-cleanup",
|
||||
default=True,
|
||||
envvar="CONTACT_CLEANUP_ENABLED",
|
||||
help="Enable/disable automatic removal of stale contacts",
|
||||
)
|
||||
@click.option(
|
||||
"--contact-cleanup-days",
|
||||
type=int,
|
||||
default=7,
|
||||
envvar="CONTACT_CLEANUP_DAYS",
|
||||
help="Remove contacts not advertised for this many days",
|
||||
)
|
||||
def receiver(
|
||||
port: str,
|
||||
baud: int,
|
||||
@@ -274,6 +304,8 @@ def receiver(
|
||||
mqtt_password: str | None,
|
||||
prefix: str,
|
||||
mqtt_tls: bool,
|
||||
contact_cleanup: bool,
|
||||
contact_cleanup_days: int,
|
||||
) -> None:
|
||||
"""Run interface in RECEIVER mode.
|
||||
|
||||
@@ -293,12 +325,15 @@ def receiver(
|
||||
baud=baud,
|
||||
mock=mock,
|
||||
node_address=node_address,
|
||||
device_name=device_name,
|
||||
mqtt_host=mqtt_host,
|
||||
mqtt_port=mqtt_port,
|
||||
mqtt_username=mqtt_username,
|
||||
mqtt_password=mqtt_password,
|
||||
mqtt_prefix=prefix,
|
||||
mqtt_tls=mqtt_tls,
|
||||
contact_cleanup_enabled=contact_cleanup,
|
||||
contact_cleanup_days=contact_cleanup_days,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -211,6 +211,32 @@ class BaseMeshCoreDevice(ABC):
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def remove_contact(self, public_key: str) -> bool:
|
||||
"""Remove a contact from the device's contact database.
|
||||
|
||||
Args:
|
||||
public_key: The 64-character hex public key of the contact to remove
|
||||
|
||||
Returns:
|
||||
True if contact was removed successfully
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def schedule_remove_contact(self, public_key: str) -> bool:
|
||||
"""Schedule a remove_contact request on the event loop.
|
||||
|
||||
This is safe to call from event handlers while the event loop is running.
|
||||
|
||||
Args:
|
||||
public_key: The 64-character hex public key of the contact to remove
|
||||
|
||||
Returns:
|
||||
True if request was scheduled successfully
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def run(self) -> None:
|
||||
"""Run the device event loop (blocking)."""
|
||||
@@ -627,6 +653,54 @@ class MeshCoreDevice(BaseMeshCoreDevice):
|
||||
logger.error(f"Failed to schedule get contacts: {e}")
|
||||
return False
|
||||
|
||||
def remove_contact(self, public_key: str) -> bool:
|
||||
"""Remove a contact from the device's contact database.
|
||||
|
||||
Note: This method should only be called before the event loop is running
|
||||
(e.g., during initialization). For calling during event processing,
|
||||
use schedule_remove_contact() instead.
|
||||
"""
|
||||
if not self._connected or not self._mc:
|
||||
logger.error("Cannot remove contact: not connected")
|
||||
return False
|
||||
|
||||
try:
|
||||
|
||||
async def _remove_contact() -> None:
|
||||
await self._mc.commands.remove_contact(public_key)
|
||||
|
||||
self._loop.run_until_complete(_remove_contact())
|
||||
logger.info(f"Removed contact {public_key[:12]}...")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to remove contact: {e}")
|
||||
return False
|
||||
|
||||
def schedule_remove_contact(self, public_key: str) -> bool:
|
||||
"""Schedule a remove_contact request on the event loop.
|
||||
|
||||
This is safe to call from event handlers while the event loop is running.
|
||||
The request is scheduled as a task on the event loop.
|
||||
|
||||
Returns:
|
||||
True if request was scheduled, False if device not connected
|
||||
"""
|
||||
if not self._connected or not self._mc:
|
||||
logger.error("Cannot remove contact: not connected")
|
||||
return False
|
||||
|
||||
try:
|
||||
|
||||
async def _remove_contact() -> None:
|
||||
await self._mc.commands.remove_contact(public_key)
|
||||
|
||||
asyncio.run_coroutine_threadsafe(_remove_contact(), self._loop)
|
||||
logger.debug(f"Scheduled removal of contact {public_key[:12]}...")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to schedule remove contact: {e}")
|
||||
return False
|
||||
|
||||
def run(self) -> None:
|
||||
"""Run the device event loop."""
|
||||
self._running = True
|
||||
|
||||
@@ -329,6 +329,30 @@ class MockMeshCoreDevice(BaseMeshCoreDevice):
|
||||
"""
|
||||
return self.get_contacts()
|
||||
|
||||
def remove_contact(self, public_key: str) -> bool:
|
||||
"""Remove a contact from the mock device's contact database."""
|
||||
if not self._connected:
|
||||
logger.error("Cannot remove contact: not connected")
|
||||
return False
|
||||
|
||||
# Find and remove the contact from mock_config.nodes
|
||||
for i, node in enumerate(self.mock_config.nodes):
|
||||
if node.public_key == public_key:
|
||||
del self.mock_config.nodes[i]
|
||||
logger.info(f"Mock: Removed contact {public_key[:12]}...")
|
||||
return True
|
||||
|
||||
logger.warning(f"Mock: Contact {public_key[:12]}... not found")
|
||||
return True # Return True even if not found (idempotent)
|
||||
|
||||
def schedule_remove_contact(self, public_key: str) -> bool:
|
||||
"""Schedule a remove_contact request.
|
||||
|
||||
For the mock device, this is the same as remove_contact() since we
|
||||
don't have a real async event loop.
|
||||
"""
|
||||
return self.remove_contact(public_key)
|
||||
|
||||
def run(self) -> None:
|
||||
"""Run the mock device event loop."""
|
||||
self._running = True
|
||||
|
||||
@@ -20,6 +20,9 @@ from meshcore_hub.interface.device import (
|
||||
create_device,
|
||||
)
|
||||
|
||||
# Default contact cleanup settings
|
||||
DEFAULT_CONTACT_CLEANUP_DAYS = 7
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -34,6 +37,8 @@ class Receiver:
|
||||
device: BaseMeshCoreDevice,
|
||||
mqtt_client: MQTTClient,
|
||||
device_name: Optional[str] = None,
|
||||
contact_cleanup_enabled: bool = True,
|
||||
contact_cleanup_days: int = DEFAULT_CONTACT_CLEANUP_DAYS,
|
||||
):
|
||||
"""Initialize receiver.
|
||||
|
||||
@@ -41,10 +46,14 @@ class Receiver:
|
||||
device: MeshCore device instance
|
||||
mqtt_client: MQTT client instance
|
||||
device_name: Optional device/node name to set on startup
|
||||
contact_cleanup_enabled: Whether to remove stale contacts from device
|
||||
contact_cleanup_days: Remove contacts not advertised for this many days
|
||||
"""
|
||||
self.device = device
|
||||
self.mqtt = mqtt_client
|
||||
self.device_name = device_name
|
||||
self.contact_cleanup_enabled = contact_cleanup_enabled
|
||||
self.contact_cleanup_days = contact_cleanup_days
|
||||
self._running = False
|
||||
self._shutdown_event = threading.Event()
|
||||
self._device_connected = False
|
||||
@@ -167,6 +176,8 @@ class Receiver:
|
||||
|
||||
The device returns contacts as a dict keyed by public_key.
|
||||
We split this into individual 'contact' events for cleaner processing.
|
||||
Stale contacts (not advertised for > contact_cleanup_days) are removed
|
||||
from the device and not published.
|
||||
|
||||
Args:
|
||||
payload: Dict of contacts keyed by public_key
|
||||
@@ -188,22 +199,54 @@ class Receiver:
|
||||
return
|
||||
|
||||
device_key = self.device.public_key # Capture for type narrowing
|
||||
count = 0
|
||||
current_time = int(time.time())
|
||||
stale_threshold = current_time - (self.contact_cleanup_days * 24 * 60 * 60)
|
||||
|
||||
published_count = 0
|
||||
removed_count = 0
|
||||
|
||||
for contact in contacts:
|
||||
if not isinstance(contact, dict):
|
||||
continue
|
||||
|
||||
public_key = contact.get("public_key")
|
||||
if not public_key:
|
||||
continue
|
||||
|
||||
# Check if contact is stale based on last_advert timestamp
|
||||
# Only check if cleanup is enabled and last_advert exists
|
||||
if self.contact_cleanup_enabled:
|
||||
last_advert = contact.get("last_advert")
|
||||
if last_advert is not None and last_advert > 0:
|
||||
if last_advert < stale_threshold:
|
||||
# Contact is stale - remove from device
|
||||
adv_name = contact.get("adv_name", contact.get("name", ""))
|
||||
logger.info(
|
||||
f"Removing stale contact {public_key[:12]}... "
|
||||
f"({adv_name}) - last advertised "
|
||||
f"{(current_time - last_advert) // 86400} days ago"
|
||||
)
|
||||
self.device.schedule_remove_contact(public_key)
|
||||
removed_count += 1
|
||||
continue # Don't publish stale contacts
|
||||
|
||||
try:
|
||||
self.mqtt.publish_event(
|
||||
device_key,
|
||||
"contact", # Use singular 'contact' for individual events
|
||||
contact,
|
||||
)
|
||||
count += 1
|
||||
published_count += 1
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to publish contact event: {e}")
|
||||
|
||||
logger.info(f"Published {count} contact events to MQTT")
|
||||
if removed_count > 0:
|
||||
logger.info(
|
||||
f"Contact sync: published {published_count}, "
|
||||
f"removed {removed_count} stale contacts"
|
||||
)
|
||||
else:
|
||||
logger.info(f"Published {published_count} contact events to MQTT")
|
||||
|
||||
def start(self) -> None:
|
||||
"""Start the receiver."""
|
||||
@@ -306,6 +349,8 @@ def create_receiver(
|
||||
mqtt_password: Optional[str] = None,
|
||||
mqtt_prefix: str = "meshcore",
|
||||
mqtt_tls: bool = False,
|
||||
contact_cleanup_enabled: bool = True,
|
||||
contact_cleanup_days: int = DEFAULT_CONTACT_CLEANUP_DAYS,
|
||||
) -> Receiver:
|
||||
"""Create a configured receiver instance.
|
||||
|
||||
@@ -321,6 +366,8 @@ def create_receiver(
|
||||
mqtt_password: MQTT password
|
||||
mqtt_prefix: MQTT topic prefix
|
||||
mqtt_tls: Enable TLS/SSL for MQTT connection
|
||||
contact_cleanup_enabled: Whether to remove stale contacts from device
|
||||
contact_cleanup_days: Remove contacts not advertised for this many days
|
||||
|
||||
Returns:
|
||||
Configured Receiver instance
|
||||
@@ -345,7 +392,13 @@ def create_receiver(
|
||||
)
|
||||
mqtt_client = MQTTClient(mqtt_config)
|
||||
|
||||
return Receiver(device, mqtt_client, device_name=device_name)
|
||||
return Receiver(
|
||||
device,
|
||||
mqtt_client,
|
||||
device_name=device_name,
|
||||
contact_cleanup_enabled=contact_cleanup_enabled,
|
||||
contact_cleanup_days=contact_cleanup_days,
|
||||
)
|
||||
|
||||
|
||||
def run_receiver(
|
||||
@@ -360,6 +413,8 @@ def run_receiver(
|
||||
mqtt_password: Optional[str] = None,
|
||||
mqtt_prefix: str = "meshcore",
|
||||
mqtt_tls: bool = False,
|
||||
contact_cleanup_enabled: bool = True,
|
||||
contact_cleanup_days: int = DEFAULT_CONTACT_CLEANUP_DAYS,
|
||||
) -> None:
|
||||
"""Run the receiver (blocking).
|
||||
|
||||
@@ -377,6 +432,8 @@ def run_receiver(
|
||||
mqtt_password: MQTT password
|
||||
mqtt_prefix: MQTT topic prefix
|
||||
mqtt_tls: Enable TLS/SSL for MQTT connection
|
||||
contact_cleanup_enabled: Whether to remove stale contacts from device
|
||||
contact_cleanup_days: Remove contacts not advertised for this many days
|
||||
"""
|
||||
receiver = create_receiver(
|
||||
port=port,
|
||||
@@ -390,6 +447,8 @@ def run_receiver(
|
||||
mqtt_password=mqtt_password,
|
||||
mqtt_prefix=mqtt_prefix,
|
||||
mqtt_tls=mqtt_tls,
|
||||
contact_cleanup_enabled=contact_cleanup_enabled,
|
||||
contact_cleanup_days=contact_cleanup_days,
|
||||
)
|
||||
|
||||
# Set up signal handlers
|
||||
|
||||
@@ -48,8 +48,58 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Messages Table -->
|
||||
<div class="overflow-x-auto overflow-y-visible bg-base-100 rounded-box shadow">
|
||||
<!-- Messages List - Mobile Card View -->
|
||||
<div class="lg:hidden space-y-3">
|
||||
{% for msg in messages %}
|
||||
<div class="card bg-base-100 shadow-sm">
|
||||
<div class="card-body p-3">
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div class="flex items-center gap-2 min-w-0">
|
||||
<span class="text-lg flex-shrink-0" title="{{ msg.message_type|capitalize }}">
|
||||
{% if msg.message_type == 'channel' %}📻{% else %}👤{% endif %}
|
||||
</span>
|
||||
<div class="min-w-0">
|
||||
<div class="font-medium text-sm truncate">
|
||||
{% if msg.message_type == 'channel' %}
|
||||
<span class="font-mono">CH{{ msg.channel_idx }}</span>
|
||||
{% else %}
|
||||
{% if msg.sender_tag_name or msg.sender_name %}
|
||||
{{ msg.sender_tag_name or msg.sender_name }}
|
||||
{% else %}
|
||||
<span class="font-mono text-xs">{{ (msg.pubkey_prefix or '-')[:12] }}</span>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</div>
|
||||
<div class="text-xs opacity-60">
|
||||
{{ msg.received_at[:16].replace('T', ' ') if msg.received_at else '-' }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center gap-2 flex-shrink-0">
|
||||
{% if msg.snr is not none %}
|
||||
<span class="badge badge-ghost badge-xs">{{ "%.1f"|format(msg.snr) }}dB</span>
|
||||
{% endif %}
|
||||
{% if msg.receivers and msg.receivers|length >= 1 %}
|
||||
<div class="flex gap-0.5">
|
||||
{% for recv in msg.receivers %}
|
||||
<a href="/nodes/{{ recv.public_key }}" class="text-sm hover:opacity-70" title="{{ recv.tag_name or recv.name or recv.public_key[:12] }}">📡</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% elif msg.received_by %}
|
||||
<a href="/nodes/{{ msg.received_by }}" class="text-sm hover:opacity-70" title="{{ msg.receiver_tag_name or msg.receiver_name or msg.received_by[:12] }}">📡</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
<p class="text-sm mt-2 break-words whitespace-pre-wrap">{{ msg.text or '-' }}</p>
|
||||
</div>
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="text-center py-8 opacity-70">No messages found.</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
|
||||
<!-- Messages Table - Desktop View -->
|
||||
<div class="hidden lg:block overflow-x-auto overflow-y-visible bg-base-100 rounded-box shadow">
|
||||
<table class="table table-zebra">
|
||||
<thead>
|
||||
<tr>
|
||||
|
||||
@@ -6,26 +6,12 @@ from meshcore_hub.common.config import (
|
||||
CollectorSettings,
|
||||
APISettings,
|
||||
WebSettings,
|
||||
LogLevel,
|
||||
InterfaceMode,
|
||||
)
|
||||
|
||||
|
||||
class TestCommonSettings:
|
||||
"""Tests for CommonSettings."""
|
||||
|
||||
def test_default_values(self) -> None:
|
||||
"""Test default setting values without .env file influence."""
|
||||
settings = CommonSettings(_env_file=None)
|
||||
|
||||
assert settings.data_home == "./data"
|
||||
assert settings.log_level == LogLevel.INFO
|
||||
assert settings.mqtt_host == "localhost"
|
||||
assert settings.mqtt_port == 1883
|
||||
assert settings.mqtt_username is None
|
||||
assert settings.mqtt_password is None
|
||||
assert settings.mqtt_prefix == "meshcore"
|
||||
|
||||
def test_custom_data_home(self) -> None:
|
||||
"""Test custom DATA_HOME setting."""
|
||||
settings = CommonSettings(_env_file=None, data_home="/custom/data")
|
||||
@@ -36,37 +22,19 @@ class TestCommonSettings:
|
||||
class TestInterfaceSettings:
|
||||
"""Tests for InterfaceSettings."""
|
||||
|
||||
def test_default_values(self) -> None:
|
||||
"""Test default setting values without .env file influence."""
|
||||
settings = InterfaceSettings(_env_file=None)
|
||||
def test_custom_values(self) -> None:
|
||||
"""Test custom setting values."""
|
||||
settings = InterfaceSettings(
|
||||
_env_file=None, serial_port="/dev/ttyACM0", serial_baud=9600
|
||||
)
|
||||
|
||||
assert settings.interface_mode == InterfaceMode.RECEIVER
|
||||
assert settings.serial_port == "/dev/ttyUSB0"
|
||||
assert settings.serial_baud == 115200
|
||||
assert settings.mock_device is False
|
||||
assert settings.serial_port == "/dev/ttyACM0"
|
||||
assert settings.serial_baud == 9600
|
||||
|
||||
|
||||
class TestCollectorSettings:
|
||||
"""Tests for CollectorSettings."""
|
||||
|
||||
def test_default_values(self) -> None:
|
||||
"""Test default setting values without .env file influence."""
|
||||
settings = CollectorSettings(_env_file=None)
|
||||
|
||||
# database_url is None by default, effective_database_url computes it
|
||||
assert settings.database_url is None
|
||||
# Path normalizes ./data to data
|
||||
assert settings.effective_database_url == "sqlite:///data/collector/meshcore.db"
|
||||
assert settings.data_home == "./data"
|
||||
assert settings.collector_data_dir == "data/collector"
|
||||
|
||||
# seed_home defaults to ./seed (normalized to "seed")
|
||||
assert settings.seed_home == "./seed"
|
||||
assert settings.effective_seed_home == "seed"
|
||||
# node_tags_file and members_file are derived from effective_seed_home
|
||||
assert settings.node_tags_file == "seed/node_tags.yaml"
|
||||
assert settings.members_file == "seed/members.yaml"
|
||||
|
||||
def test_custom_data_home(self) -> None:
|
||||
"""Test that custom data_home affects effective paths."""
|
||||
settings = CollectorSettings(_env_file=None, data_home="/custom/data")
|
||||
@@ -76,10 +44,6 @@ class TestCollectorSettings:
|
||||
== "sqlite:////custom/data/collector/meshcore.db"
|
||||
)
|
||||
assert settings.collector_data_dir == "/custom/data/collector"
|
||||
# seed_home is independent of data_home
|
||||
assert settings.effective_seed_home == "seed"
|
||||
assert settings.node_tags_file == "seed/node_tags.yaml"
|
||||
assert settings.members_file == "seed/members.yaml"
|
||||
|
||||
def test_explicit_database_url_overrides(self) -> None:
|
||||
"""Test that explicit database_url overrides the default."""
|
||||
@@ -103,19 +67,6 @@ class TestCollectorSettings:
|
||||
class TestAPISettings:
|
||||
"""Tests for APISettings."""
|
||||
|
||||
def test_default_values(self) -> None:
|
||||
"""Test default setting values without .env file influence."""
|
||||
settings = APISettings(_env_file=None)
|
||||
|
||||
assert settings.api_host == "0.0.0.0"
|
||||
assert settings.api_port == 8000
|
||||
# database_url is None by default, effective_database_url computes it
|
||||
assert settings.database_url is None
|
||||
# Path normalizes ./data to data
|
||||
assert settings.effective_database_url == "sqlite:///data/collector/meshcore.db"
|
||||
assert settings.api_read_key is None
|
||||
assert settings.api_admin_key is None
|
||||
|
||||
def test_custom_data_home(self) -> None:
|
||||
"""Test that custom data_home affects effective database path."""
|
||||
settings = APISettings(_env_file=None, data_home="/custom/data")
|
||||
@@ -136,17 +87,6 @@ class TestAPISettings:
|
||||
class TestWebSettings:
|
||||
"""Tests for WebSettings."""
|
||||
|
||||
def test_default_values(self) -> None:
|
||||
"""Test default setting values without .env file influence."""
|
||||
settings = WebSettings(_env_file=None)
|
||||
|
||||
assert settings.web_host == "0.0.0.0"
|
||||
assert settings.web_port == 8080
|
||||
assert settings.api_base_url == "http://localhost:8000"
|
||||
assert settings.network_name == "MeshCore Network"
|
||||
# Path normalizes ./data to data
|
||||
assert settings.web_data_dir == "data/web"
|
||||
|
||||
def test_custom_data_home(self) -> None:
|
||||
"""Test that custom data_home affects effective paths."""
|
||||
settings = WebSettings(_env_file=None, data_home="/custom/data")
|
||||
|
||||
Reference in New Issue
Block a user