164 Commits

Author SHA1 Message Date
l5y 1804a8a75e web: add visual feedback to pubsub events (#824) 2026-06-24 20:34:35 +02:00
l5y 8083206210 web: implement pubsub (#821) 2026-06-24 17:35:33 +02:00
l5y f5f9298746 web: add pagination to all APIs (#820) 2026-06-24 13:02:50 +02:00
l5y 7eccbfb89a fix: unify Meshtastic custom LoRa label with MeshCore format (#818) 2026-06-23 21:58:18 +02:00
l5y 3790686ab2 web: version JS/CSS assets to bust stale browser caches (#815) 2026-06-23 21:23:35 +02:00
George c8668a730c fix: report custom LoRa config as "Custom SF{sf}/BW{bw}/CR{cr}" when use_preset=False (#811) (#812) 2026-06-23 19:46:30 +02:00
l5y af64b2c60f web: fix apis to use consistently use camel case (#802) 2026-06-21 20:17:40 +02:00
l5y 5e0363a0ec web: breaking change on stats api (#801)
* web: breaking change on stats api

* address review comments
2026-06-21 13:41:41 +02:00
l5y 512b4f157b Fix regression where Meshcore chat senders show as Meshtastic (#794)
* Fix regression where Meshcore chat senders show as Meshtastic

* Address review feedback for protocol misclassification fix

- ingest.rb: exclude wrapper ``protocol`` key from /api/nodes batch-limit
  count so the documented 1000-node maximum still applies after the
  Python ingestor started stamping protocol at the wrapper level.
- Drop plan-file references from production and test comments per the
  repo guidelines; the why is already explained inline.

* Address protocol-fallback review feedback

- Neighbor placeholder now inherits the source node's protocol from the
  surrounding /api/neighbors entry, so the badge tracks the radio the
  peer lives on instead of collapsing to the neutral "Unknown" label
  (review item #1).
- resolve_record_protocol logs one warn_log line when an explicit
  protocol stamp is rejected as malformed, making misbehaving custom
  protocol adapters visible in the operator log instead of silently
  falling back (review item #3).

* Extract buildNodePlaceholder helper for testability

The neighbor placeholder logic in main.js lives inside an untested
closure, so codecov reported the protocol-propagation lines as
uncovered.  Extract the small placeholder builder into long-link-router
so it can be unit tested directly; the closure-internal call site stays
trivial (one factory call + one fallback call).
2026-05-24 09:49:45 +02:00
l5y 8a89185fbe fix sentinel position data (#792)
* fix sentinel position data

* address review comments
2026-05-18 21:27:47 +02:00
l5y b187ba2066 data: move connected event to after ensure_contacts (#789)
* data: move connected event to after ensure_contacts

* data: address review comments
2026-05-18 08:57:08 +02:00
l5y c32da2c541 chore: bump to 0.6.4 (#785) 2026-05-03 17:01:28 +02:00
l5y 73e161f432 web: fix liveliness of api data hydration bug (#783)
* web: fix liveliness of api data hydration bug

* web: address review comments
2026-05-03 13:05:37 +02:00
l5y 1041e06644 data: refactor 4/7 interfaces (#775)
* data: refactor 4/7 interfaces

* data: address PR #775 review feedback

Fix the two CI test regressions caused by the package split:
- ``factory._load_ble_interface`` no longer keeps a stale module-level
  ``BLEInterface`` cache that survived ``monkeypatch`` teardown across
  tests. The package-level attribute is now the single cache; the
  ``factory.py`` global was removed.  This unblocks
  ``test_load_ble_interface_sets_global``.
- ``interfaces/__init__.py`` re-resolves ``SerialInterface`` and
  ``TCPInterface`` from ``meshtastic.*`` at package-load time so that a
  test that pops ``data.mesh_ingestor.interfaces`` from ``sys.modules``
  and re-imports picks up the freshly registered classes rather than
  whatever a cached ``factory.py`` first resolved.  This unblocks
  ``test_interfaces_patch_handles_preimported_serial``.

Restore 100% patch coverage on the interfaces subpackage by:
- Adding tests for previously uncovered, testable paths:
  ``_extract_host_node_id(None)``, ``_ensure_channel_metadata``,
  ``_normalise_nodeinfo_packet`` (None input + dict-conversion fallback),
  ``_resolve_lora_message`` (radio_section paths), ``_modem_preset``
  (preset attr fallback + unparseable value), ``_camelcase_enum_name``
  separator-only input, ``_region_frequency`` no-digit enum name,
  ``_ensure_radio_metadata`` unresolvable-message path, plus the
  unknown-section recursive branch of ``_candidate_node_id``.
- Marking genuinely unreachable defensive branches with
  ``pragma: no cover`` (BLE receive loop body, upstream API regression
  guards, patch re-entry guard, unreachable ``NoAvailableMeshInterface``
  fallback).
2026-05-02 22:59:52 +02:00
l5y f6aff3bdb8 data: refactor 3/7 protocols (#774)
* data: refactor 3/7 protocols

* data: address PR #774 review feedback

- Rewrite the parents[4] path comment in protocols/meshcore/debug_log.py
  to clearly explain why the index changed from parents[3] (the original
  pre-split index) without contradicting the code.
- Add tests covering the six lines flagged uncovered by codecov:
  * _process_self_info host-position branch (handlers.py:78)
  * on_contact_msg early-return for missing text/sender_ts (handlers.py:278)
  * close() RuntimeError swallow when loop closes mid-call (interface.py:155-156)
  * _run_meshcore wrapper around _ensure_channel_names failure (runner.py:131-132)

Restores 100% patch coverage on the meshcore package.
2026-05-02 22:05:43 +02:00
l5y 09d9e7be13 chore: bump version to 0.6.3 (#779)
* chore: bump version to 0.6.3

* chore: bump version to 0.6.3
2026-04-29 22:06:29 +02:00
l5y 161f22bf12 chore: bump version to 0.6.2 (#763)
Backfill v0.6.1 CHANGELOG entry (previously undocumented),
add v0.6.2 entry covering 9 commits since v0.6.1, and bump
the version string across data, web, matrix, and app along
with the README docker-pull examples.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 19:02:26 +02:00
l5y 491678f75b data: catch packet handler errors (#759)
* data: catch packet handler errors

* data: address review comments

* data: address review comments
2026-04-21 11:41:44 +02:00
l5y db236d58e2 web: fix meshcore message duplication with 120s dupe protection (#758)
* web: fix meshcore message duplication with 120s dupe protection

* web: fix meshcore message duplication with 120s dupe protection

* web: address review comments

* web: address review comments
2026-04-21 10:13:39 +02:00
l5y 7f5b52db67 ingestor: deduplicate meshcore messages (#752)
* ingestor: deduplicate meshcore messages

* ingestor: address review comments

* ingestor: address review comments
2026-04-16 22:16:22 +02:00
l5y 8ac12ee2c8 fix reaction handling and classification (#750)
* fix reaction handling and classification

* fix reaction handling and classification

* web: add tests

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments
2026-04-16 20:07:41 +02:00
l5y c175445251 ingestor: fix serial connection failures (#736)
* ingestor: fix serial connection failures

* ingestor: address review comments

* ingestor: address review comments

* ingestor: further hardening

* ingestor: add tests

* ingestor: address review comments

* ingestor: address review comments
2026-04-13 23:42:07 +02:00
l5y b951dbffeb web: per protocol active node counts (#735)
* web: per protocol active node counts

* web: address review comments
2026-04-13 18:26:16 +02:00
l5y 10e6c99196 data: better lora frequency handling for meshtastic (#733)
* data: better lora frequency handling for meshtastic

* ingestor: address review comments
2026-04-12 16:02:15 +02:00
l5y aeb97477f0 chore: bump version to 0.6.1 (#726) 2026-04-09 13:14:20 +02:00
l5y 5b9e6e3d48 data: trace analysus multi ingestor support (#721)
* data: trace analysus multi ingestor support

* address review comments
2026-04-08 11:58:32 +02:00
l5y 4a6ba38e94 chore: prepare codebase for breaking release (#718)
* chore: prepare codebase for breaking release

* docker: fix debug flug in prod matrix bridge
2026-04-08 10:51:38 +02:00
l5y 4d38ddd341 web: facelift (#716)
* web: facelift

* web: facelift

* web: facelift

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments

* web: more css magic

* web: link parsing for chat contact

* web: remove one-letter fallback for shortnames

* Potential fix for pull request finding 'CodeQL / Incomplete multi-character sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* web: fix fallback for shortnames

* web: address review comments

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-07 21:38:43 +02:00
l5y 267d2ec9e1 data: fix position time updates (#715)
* data: fix position time updates

* data: fix position time updates
2026-04-06 19:29:38 +02:00
l5y 526a0c7246 data: fix meshcore ingestore self reporting (#713)
* data: fix meshcore ingestore self reporting

* data: fix meshcore ingestore self reporting

* address review comments
2026-04-06 15:19:01 +02:00
l5y d8b80c2a97 web: reference meshcore nodes in chat (#709)
* web: reference meshcore nodes in chat

* data: add adv_name to messages

* web: address review comments

* derive actual companion from name string

* derive actual companion from name string

* derive actual companion from name string

* web: address review comments

* web: address review comments
2026-04-06 13:39:00 +02:00
l5y de1ccc5a2e release: v0.6.0 — remove deprecated env var aliases (#704)
* chore: bump version to 0.6.0 and remove deprecated env var aliases

BREAKING CHANGES:
- POTATOMESH_INSTANCE removed — use INSTANCE_DOMAIN
- PROVIDER removed — use PROTOCOL
- MESH_SERIAL removed — use CONNECTION
- PORT config alias removed — use CONNECTION

The _ConfigModule proxy class (which kept PROTOCOL/PROVIDER and
CONNECTION/PORT in sync) is deleted. docker-compose.yml now defaults
INSTANCE_DOMAIN to http://web:41447 so deployments without an explicit
value continue to work.

* tests: run black

* address review comments
2026-04-05 16:49:10 +02:00
l5y 8c59396ec8 fix: derive channel probe bound from device max_channels (#701)
Replace the hardcoded max_idx=8 parameter on _ensure_channel_names with
a DEVICE_INFO query (send_device_query → max_channels) so the full range
of configured channels is always probed regardless of firmware variant.
Falls back to _CHANNEL_PROBE_FALLBACK_MAX (32) when the query fails or
the device returns an older firmware that omits max_channels.

Also removes always=True from the warning-severity channel failure log
(redundant — only debug-severity is gated behind the DEBUG flag) and adds
a deferred-import comment in _ensure_channel_names.
2026-04-05 13:46:04 +02:00
l5y adc122fce0 data: register meshcore channel mappings (#695)
* data: register meshcore channel mappings

* fix: use mc.commands.get_channel for MeshCore channel name probing

MeshCore exposes device commands via the commands sub-object
(CommandHandler), not directly on MeshCore instances.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: probe all channel indices regardless of ERROR responses

Removed the consecutive-error early-stop heuristic from
_ensure_channel_names so sparse channel configurations (e.g. slots 0
and 5 configured with slots 1–4 empty) are fully probed. Only a hard
exception aborts the loop early.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 13:36:03 +02:00
l5y d33ebd8f4c data: provide frequency and modem preset for meshcore (#694)
* data: provide frequency and modem preset for meshcore

* data: provide frequency and modem preset for meshcore

* ingestor: address review comments

* fix: remove duplicate _mark_packet_seen entry from handlers __all__

* ci: install meshcore in Python workflow

protocols/meshcore.py now imports meshcore at module level (required to
fix a self-referential import failure after the providers/ → protocols/
rename).  test_provider_unit.py imports that module unconditionally, so
meshcore must be present in the test environment.

* data: run black
2026-04-05 09:13:48 +02:00
l5y 3cfa0db7e6 web: distinguish meshcore from meshtastic in frontend (#688)
* web: distinguish meshcore from meshtastic in frontend

* fix mark_packet_seen bug

* web: distinguish meshcore from meshtastic in frontend

* address review comments

* address review comments

* address review comments
2026-04-04 17:14:16 +02:00
l5y d9420ff13b fix: address review comments from PRs #676 and #681 (#689)
* fix: address review comments from PRs #676 and #681

- Introduce ClosedBeforeConnectedError(ConnectionError) subclass so
  callers can distinguish a user-initiated shutdown from a hardware
  failure without string-matching the exception message (#676)
- Add test covering the close-before-connected path: asserts
  isConnected stays False and error_holder contains the typed error
- Add protocolIconPrefixHtml unit tests covering null, meshtastic,
  meshcore, and unknown protocol strings (#681)
- Add buildDisplayContext tests for protocol extraction from trace,
  node, and absent candidate sources (#681)
- Expose buildDisplayContext via _testUtils to make it directly testable
- Add meshcore icon presence assertions to createAnnouncementEntry and
  createMessageChatEntry tests (previously only checked absence of
  meshtastic icon)

* fix: address #689 review comments

- Move createMessageChatEntry meshcore icon test into its own section,
  after the createMessageChatEntry divider where it belongs
- Export ClosedBeforeConnectedError from providers/__init__.py via the
  existing lazy-load __getattr__ so callers outside the providers/
  subpackage can catch it without importing the full meshcore module

* refactor: eliminate test boilerplate to fix SonarCloud duplication gate

Introduce withApp() and innerHtml() helpers in main-protocol.test.js to
replace the 18-repeated setupApp/try/finally/cleanup pattern and the
inconsistent innerHTML extraction expression. No test logic changed.

* refactor: extract stalled-run helpers to fix SonarCloud duplication gate

The two stall-based _run_meshcore tests shared ~20 lines of identical
setup and spin-loop boilerplate. Extract _setup_stalled_run() and
_start_stalled_run() so each test contains only its distinct assertions.
2026-04-04 13:28:26 +02:00
Ben Allfree 257e26c996 [meshcore] fix: race condition (#676)
* fix: ensure stop_event is set before connection completion in _run_meshcore

* Fix CancelledError lint in meshcore cancel test
2026-04-04 12:41:56 +02:00
l5y dcb374fbf9 enh: surface meshcore role types (#680) (#685)
* enh: surface meshcore role types (#680)

Map MeshCore ADV_TYPE_* integers to user.role strings so COMPANION,
REPEATER, ROOM_SERVER, and SENSOR roles are surfaced to the dashboard.
Role is omitted when ADV_TYPE_NONE (0) or unknown.

Co-authored-by: Ben Allfree <ben@benallfree.com>

* data: run black

---------

Co-authored-by: Ben Allfree <ben@benallfree.com>
2026-04-04 10:41:06 +02:00
l5y 9c3dae3e7d chore: refactor codebase before meshcore release (#682)
* chore: refactor codebase before meshcore release

* data: run black

* fix: resolve SonarCloud S1244/S5796 reliability issues in test files

Replace floating-point equality comparisons with pytest.approx() to
satisfy S1244, and replace the `is` identity operator with id()-based
comparison to satisfy S5796.

* fix: remove duplicate encrypted_flag assignment in store_packet_dict

The encrypted_flag was computed identically on lines 307 and 345 with no
mutation of `encrypted` between them. Remove the dead second assignment.
2026-04-04 10:22:31 +02:00
Ben Allfree 7806efb2cf meshcore/fix: short name should be 1st 4 hex digits of public key (#679) 2026-04-04 09:40:49 +02:00
Ben Allfree 295d4cf2bb chore: update mesh.sh to use requirements file (#675) 2026-04-03 23:20:48 +02:00
l5y 09ea277a40 data/meshcore: fix ble and enable tcp (#669)
* data/meshcore: fix ble and enable tcp

* ingestor: address review comments

* ingestor: address review comments
2026-04-02 22:31:33 +02:00
l5y 4fa0745d1b data: handle store_forward and router_heartbeat portnum (#667)
* data: handle store_forward and router_heartbeat portnum

* ingestor: address review comments
2026-03-31 23:42:26 +02:00
l5y a62a068c08 feat: implement meshcore provider (#663)
* feat: add meshcore support

* fix: address PR #663 review comments

* fix: address PR #663 review comments

* address review comments
2026-03-31 13:44:05 +02:00
l5y e48c575b9d web: prepare release (#665)
* web: prepare release

* fix: address pre-release review concerns

- Emit invalid telemetry_type warning at severity=warning/always=True so
  it surfaces in production logs, not just under DEBUG=1
- Hoist VALID_TELEMETRY_TYPES to a module-level constant in DataProcessing
  to avoid per-call allocation inside insert_telemetry
- Add Python test covering the invalid-type drop path in store_telemetry_packet
- Add Ruby spec asserting that an invalid telemetry_type in a POST payload
  is discarded and metric-based inference takes over
2026-03-30 23:15:55 +02:00
l5y d6a2e263cc data: prepare ingestor for meshcore (#658)
* data: prepare ingestor for meshcore

* ingestor: address review comments

* ingestor: address review comments

* ingestor: address review comments

* ingestor: address review comments
2026-03-30 09:17:10 +02:00
l5y f638c79e13 web: fix css issues (#659)
* web: fix css issues

* chore: bump version to 0.5.12
2026-03-30 08:55:35 +02:00
l5y a5d0008555 feat: split device and power-sensor telemetry charts (#643) (#656)
* feat: split device and power-sensor telemetry charts (#643)

Add telemetry_type TEXT discriminator column across the full stack so
device_metrics rows no longer mix with power_metrics in the same chart.
Python and Ruby ingestors detect the protobuf subtype at write time;
classifySnapshot() provides field-presence fallback for legacy rows.
'Power metrics' chart split into 'Device health' and 'Power sensor'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: skip typeFilter for aggregated telemetry; add air_quality coverage

- renderTelemetryChart now skips spec.typeFilter when chartOptions.isAggregated
  is true, preventing mixed-bucket aggregated snapshots from losing series data
- renderTelemetryCharts detects the aggregated vs per-packet path and sets
  isAggregated accordingly; typeFilter still applies for per-packet history
- JS tests: extract makeAggregatedNode/makeHistoryNode helpers to eliminate
  fixture duplication; add aggregated-mixed-bucket regression test; move
  type-separation tests onto the history path where filtering actually applies
- Ruby + Python: add air_quality_metrics telemetry_type tests for coverage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: reduce test duplication flagged by Sonar

Hoist CHART_NOW_MS/CHART_NOW_SECONDS constants to eliminate 14 repeated
setup lines across renderTelemetryCharts tests.  Extract
expect_stored_telemetry_type helper in app_spec to replace the four
identical with_db/SELECT/expect blocks in telemetry_type inference tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* web: address review comments

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 00:07:24 +02:00
l5y 4d0d6f8565 web: implement a 'protocol' field across systems (#655)
* web: implement a 'protocol' field across systems

* web: address review feedback on multi-protocol support

- Rebase on main (pick up coordinate-clearing bugfix from #654)
- P1: prevent cross-protocol message merges on shared packet IDs
- P2: exclude "ingestor" key when enforcing /api/nodes batch limit
- Extract append_protocol_filter helper + PROTOCOL_CLAUSE constant to
  reduce cognitive complexity and deduplicate SQL fragment in queries.rb
- Extract coerce_bool helper to reduce upsert_node cognitive complexity
- Merge nested if in insert_message protocol update path (Sonar)
- Add explicit UPDATE backfill in ensure_schema_upgrades so any pre-existing
  NULL/empty protocol rows are set to meshtastic on upgrade
- Rename migration file to 20260328_ (correct year)
- Expand protocol_spec.rb: filter tests for all 7 endpoints,
  cross-protocol non-merge test, batch limit test, Sonar constant fixes,
  ENV.fetch, P1 regression test


* web: address review comments
2026-03-29 11:48:32 +02:00