29 Commits

Author SHA1 Message Date
l5y d323cab32a web: fix meshcore node synthisation, merging, and deduplication (#808) 2026-06-22 15:54:12 +02:00
l5y 03caf391e7 web: refactor 1/7 data processing (#772)
* web: refactor 1/7 data processing

* web: close coverage gaps in data_processing submodules

Bring every file under lib/potato_mesh/application/data_processing/ to
100% line coverage so codecov/patch passes on the 1/7 refactor PR. The
gap was a relocation of pre-existing untested branches; closing them
here keeps the subsequent refactor PRs in the series unblocked.

* Add unit tests covering canonical sender/recipient overrides,
  reply_id/emoji updates on existing rows, and the rare INSERT
  ConstraintException recovery path inside +insert_message+.
* Cover the non-canonical reporter and per-neighbour resolution
  branches in +insert_neighbors+.
* Cover the SQLException rescue in +upsert_ingestor+, the
  fallback_num branch in +touch_node_last_seen+, the limit fallback
  in +read_json_body+, the unrecognised-type branch in
  +store_decrypted_payload+, the +power+ telemetry_type fallback,
  the default-coercion path in +resolve_numeric_metric+, and the
  numeric/bare-hex paths in +canonical_node_parts+ and
  +coerce_trace_node_id+.

Drop dead code surfaced while pinning behaviour:

* +clear_encrypted+ in +insert_message+ has been initialised to
  +false+ and never reassigned since #633 dropped the
  decrypted-text override; remove it and the four dependent
  branches.
* The +rescue ArgumentError; nil+ tails in
  +identity.resolve_node_num+ and +traces.coerce_trace_node_id+ are
  unreachable because every +Integer(...)+ call inside is guarded by
  a regex pre-check.

Add a comment to the +data_processing.rb+ shim explaining that the
+require_relative+ list is ordered by dependency rather than
alphabetically, addressing review nit #5.
2026-05-02 22:08:21 +02:00
l5y db236d58e2 web: fix meshcore message duplication with 120s dupe protection (#758)
* web: fix meshcore message duplication with 120s dupe protection

* web: fix meshcore message duplication with 120s dupe protection

* web: address review comments

* web: address review comments
2026-04-21 10:13:39 +02:00
l5y a6cac6ced5 web: fix node duplication through message synthetization (#757)
* web: fix node duplication through message synthetization

* web: fix edge case

* web: address review comments
2026-04-21 08:53:09 +02:00
l5y 13b2ce9067 web: fix meshcore node misclassification (#748)
* web: fix meshcore node misclassification

* web: address review comments

* web: address review comments
2026-04-15 12:38:50 +02:00
l5y d8b80c2a97 web: reference meshcore nodes in chat (#709)
* web: reference meshcore nodes in chat

* data: add adv_name to messages

* web: address review comments

* derive actual companion from name string

* derive actual companion from name string

* derive actual companion from name string

* web: address review comments

* web: address review comments
2026-04-06 13:39:00 +02:00
l5y 406fa80dd0 web: fix node disappearance role reset (#707)
* web: fix node disappearance role reset

* web: address review comments

* web: address review comments

* web: address review comments
2026-04-05 23:43:36 +02:00
l5y 0a479e4517 web: protect real node names from fallback (#702)
* web: protect real node names from fallback

* web: address review comments

* web: address review comments
2026-04-05 13:57:18 +02:00
l5y 9c3dae3e7d chore: refactor codebase before meshcore release (#682)
* chore: refactor codebase before meshcore release

* data: run black

* fix: resolve SonarCloud S1244/S5796 reliability issues in test files

Replace floating-point equality comparisons with pytest.approx() to
satisfy S1244, and replace the `is` identity operator with id()-based
comparison to satisfy S5796.

* fix: remove duplicate encrypted_flag assignment in store_packet_dict

The encrypted_flag was computed identically on lines 307 and 345 with no
mutation of `encrypted` between them. Remove the dead second assignment.
2026-04-04 10:22:31 +02:00
l5y e48c575b9d web: prepare release (#665)
* web: prepare release

* fix: address pre-release review concerns

- Emit invalid telemetry_type warning at severity=warning/always=True so
  it surfaces in production logs, not just under DEBUG=1
- Hoist VALID_TELEMETRY_TYPES to a module-level constant in DataProcessing
  to avoid per-call allocation inside insert_telemetry
- Add Python test covering the invalid-type drop path in store_telemetry_packet
- Add Ruby spec asserting that an invalid telemetry_type in a POST payload
  is discarded and metric-based inference takes over
2026-03-30 23:15:55 +02:00
l5y a5d0008555 feat: split device and power-sensor telemetry charts (#643) (#656)
* feat: split device and power-sensor telemetry charts (#643)

Add telemetry_type TEXT discriminator column across the full stack so
device_metrics rows no longer mix with power_metrics in the same chart.
Python and Ruby ingestors detect the protobuf subtype at write time;
classifySnapshot() provides field-presence fallback for legacy rows.
'Power metrics' chart split into 'Device health' and 'Power sensor'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: skip typeFilter for aggregated telemetry; add air_quality coverage

- renderTelemetryChart now skips spec.typeFilter when chartOptions.isAggregated
  is true, preventing mixed-bucket aggregated snapshots from losing series data
- renderTelemetryCharts detects the aggregated vs per-packet path and sets
  isAggregated accordingly; typeFilter still applies for per-packet history
- JS tests: extract makeAggregatedNode/makeHistoryNode helpers to eliminate
  fixture duplication; add aggregated-mixed-bucket regression test; move
  type-separation tests onto the history path where filtering actually applies
- Ruby + Python: add air_quality_metrics telemetry_type tests for coverage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: reduce test duplication flagged by Sonar

Hoist CHART_NOW_MS/CHART_NOW_SECONDS constants to eliminate 14 repeated
setup lines across renderTelemetryCharts tests.  Extract
expect_stored_telemetry_type helper in app_spec to replace the four
identical with_db/SELECT/expect blocks in telemetry_type inference tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* web: address review comments

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 00:07:24 +02:00
l5y 4d0d6f8565 web: implement a 'protocol' field across systems (#655)
* web: implement a 'protocol' field across systems

* web: address review feedback on multi-protocol support

- Rebase on main (pick up coordinate-clearing bugfix from #654)
- P1: prevent cross-protocol message merges on shared packet IDs
- P2: exclude "ingestor" key when enforcing /api/nodes batch limit
- Extract append_protocol_filter helper + PROTOCOL_CLAUSE constant to
  reduce cognitive complexity and deduplicate SQL fragment in queries.rb
- Extract coerce_bool helper to reduce upsert_node cognitive complexity
- Merge nested if in insert_message protocol update path (Sonar)
- Add explicit UPDATE backfill in ensure_schema_upgrades so any pre-existing
  NULL/empty protocol rows are set to meshtastic on upgrade
- Rename migration file to 20260328_ (correct year)
- Expand protocol_spec.rb: filter tests for all 7 endpoints,
  cross-protocol non-merge test, batch limit test, Sonar constant fixes,
  ENV.fetch, P1 regression test


* web: address review comments
2026-03-29 11:48:32 +02:00
l5y 7b1d25e286 fix upsert clearing node coordinates bug (#654) 2026-03-28 21:21:13 +01:00
l5y b45629f13c web: do not touch neighbor last seen on neighbor info (#636)
* web: do not touch neighbor last seen on neighbor info

* web: address review comments
2026-02-14 14:43:46 +01:00
l5y 96421c346d ingestor: report self id per packet (#635)
* ingestor: report self id per packet

* ingestor: address review comments

* ingestor: address review comments

* ingestor: address review comments

* ingestor: address review comments
2026-02-14 14:29:05 +01:00
l5y e8c83a2774 web: supress encrypted text messages in frontend (#633)
* web: supress encrypted text messages in frontend

* web: address review comments

* web: address review comments

* web: address review comments

* web: address review comments
2026-02-14 13:11:02 +01:00
l5y c3181e9bd5 web: decrypted takes precedence (#614)
* web: decrypted takes precedence

* web: run rufo

* web: fix tests

* web: fix tests

* web: cover missing unit test vectors

* web: fix tests
2026-01-10 13:13:55 +01:00
l5y e0237108c6 web: decrypt PSK-1 unencrypted messages on arrival (#611)
* web: decrypt PSK-1 unencrypted messages on arrival

* web: address review comments

* web: use proper psk to decrypt instead of alias

* cover missing unit test vectors

* tests: run black formatter

* web: fix tests

* web: refine decryption data processing logic

* web: address review comments

* web: cover missing unit test vectors

* web: cover missing unit test vectors

* web: cover missing unit test vectors

* web: cover missing unit test vectors
2026-01-10 12:33:59 +01:00
l5y 5572c6cd12 web: ensure unknown nodes for messages and traces (#572) 2025-12-17 10:21:03 +01:00
l5y 06fb90513f data: track ingestors heartbeat (#549)
* data: track ingestors heartbeat

* data: address review comments

* cover missing unit test vectors

* cover missing unit test vectors
2025-12-14 18:42:17 +01:00
l5y 135de0863c web: fix traces submission with optional fields on udp (#545) 2025-12-14 13:27:07 +01:00
l5y 700fcef33f feat: implement traceroute app packet handling across the stack (#463)
* feat: implement traceroute app packet handling across the stack

* run linter

* tests: fix

* Spec: add more unit tests
2025-11-18 11:23:46 +01:00
l5y 2bb8e3fd66 Chore: update license headers (#424) 2025-11-08 10:41:57 +01:00
l5y 8bb98f65d6 implement support for replies and reactions app (#411)
* implement support for replies and reactions app

* Allow numeric reaction port packets

* allow reaction packets through mai channel filter
2025-11-06 20:58:35 +01:00
l5y 8143fbd8f7 Add full support to telemetry schema and API (#385)
* feat: auto-upgrade telemetry schema

* Ensure numeric metrics fallback to valid values

* Format data processing numeric metric lookup
2025-10-18 15:19:33 +02:00
l5y 03b5a10fe4 Add LoRa metadata fields to nodes and messages (#331)
* Add LoRa metadata fields to nodes and messages

* Filter numeric SQLite keys from message rows
2025-10-14 14:51:28 +02:00
l5y 772c5888c3 Fix ingestor debug timestamps for structured logging (#296) 2025-10-12 09:40:57 +02:00
l5y f04e917cd9 Add Apache license headers to missing sources (#297) 2025-10-12 09:38:04 +02:00
l5y 522213c040 Restore modular app functionality (#291)
* Restore modular app functionality

* Fix federation thread settings and add coverage

* Use Sinatra set for federation threads

* Restore 41447 as default web port
2025-10-12 08:54:11 +02:00