10 Commits

Author SHA1 Message Date
Jack Kingsman
5213c8c84c Updating changelog + build for 3.5.0 2026-03-19 21:53:45 -07:00
Jack Kingsman
33c2b0c948 Be better about identity resolution for stats view 2026-03-19 21:42:39 -07:00
Jack Kingsman
b021a4a8ac Fix e2e tests for contact stuff 2026-03-19 21:39:18 -07:00
Jack Kingsman
c74fdec10b Add database information to debug endpoint 2026-03-19 21:17:23 -07:00
Jack Kingsman
cf314e02ff Be cleaner about message cache dedupe after trimming inactive convos 2026-03-19 21:03:20 -07:00
Jack Kingsman
8ae600d010 Docs updates 2026-03-19 20:58:49 -07:00
Jack Kingsman
fdd82e1f77 Clean up orphaned contact child rows and add foreign key enforcement 2026-03-19 20:56:36 -07:00
Jack Kingsman
9d129260fd Fix up the header collapse to be less terrible 2026-03-19 20:53:24 -07:00
Jack Kingsman
2b80760696 Add DB entry for outgoing inside the radio lock (didn't we just do the opposite?) 2026-03-19 20:43:35 -07:00
Jack Kingsman
c2655c1809 Add basic room server support. Closes #78.
Allow basic room server usage
2026-03-19 20:28:44 -07:00
28 changed files with 980 additions and 211 deletions

View File

@@ -165,7 +165,8 @@ MeshCore firmware can encode path hops as 1-byte, 2-byte, or 3-byte identifiers.
Direct-message send behavior intentionally mirrors the firmware/library `send_msg_with_retry(...)` flow:
- We push the contact's effective route to the radio via `add_contact(...)` before sending.
- Non-final attempts use the effective route (`override > direct > flood`).
- If the initial `MSG_SENT` result includes an expected ACK code, background retries are armed.
- Non-final retry attempts use the effective route (`override > direct > flood`).
- Retry timing follows the radio's `suggested_timeout`.
- The final retry is sent as flood by resetting the path on the radio first, even if an override or direct route exists.
- Path math is always hop-count based; hop bytes are interpreted using the stored `path_hash_mode`.
@@ -174,7 +175,7 @@ Direct-message send behavior intentionally mirrors the firmware/library `send_ms
**Direct messages**: Expected ACK code is tracked. When ACK event arrives, message marked as acked.
Outgoing DMs send once immediately, then may retry up to 2 more times in the background if still unacked. Retry timing follows the radio's `suggested_timeout` from `PACKET_MSG_SENT`, and the final retry is sent as flood even when a routing override is configured. DM ACK state is terminal on first ACK: sibling retry ACK codes are cleared so one DM should not accumulate multiple delivery confirmations from different retry attempts.
Outgoing DMs send once immediately, then may retry up to 2 more times in the background only when the initial `MSG_SENT` result includes an expected ACK code and the message remains unacked. Retry timing follows the radio's `suggested_timeout` from `PACKET_MSG_SENT`, and the final retry is sent as flood even when a routing override is configured. DM ACK state is terminal on first ACK: sibling retry ACK codes are cleared so one DM should not accumulate multiple delivery confirmations from different retry attempts.
ACKs are not a contact-route source. They drive message delivery state and may appear in analytics/detail surfaces, but they do not update `direct_path*` or otherwise influence route selection for future sends.
@@ -324,6 +325,7 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| POST | `/api/contacts/{public_key}/command` | Send CLI command to repeater |
| POST | `/api/contacts/{public_key}/routing-override` | Set or clear a forced routing override |
| POST | `/api/contacts/{public_key}/trace` | Trace route to contact |
| POST | `/api/contacts/{public_key}/path-discovery` | Discover forward/return paths and persist the learned direct route |
| POST | `/api/contacts/{public_key}/repeater/login` | Log in to a repeater |
| POST | `/api/contacts/{public_key}/repeater/status` | Fetch repeater status telemetry |
| POST | `/api/contacts/{public_key}/repeater/lpp-telemetry` | Fetch CayenneLPP sensor data |
@@ -348,7 +350,7 @@ All endpoints are prefixed with `/api` (e.g., `/api/health`).
| GET | `/api/packets/undecrypted/count` | Count of undecrypted packets |
| POST | `/api/packets/decrypt/historical` | Decrypt stored packets |
| POST | `/api/packets/maintenance` | Delete old packets and vacuum |
| GET | `/api/read-state/unreads` | Server-computed unread counts, mentions, last message times |
| GET | `/api/read-state/unreads` | Server-computed unread counts, mentions, last message times, and `last_read_ats` boundaries |
| POST | `/api/read-state/mark-all-read` | Mark all conversations as read |
| GET | `/api/settings` | Get app settings |
| PATCH | `/api/settings` | Update app settings |
@@ -398,7 +400,7 @@ Read state (`last_read_at`) is tracked **server-side** for consistency across de
- Stored as Unix timestamp in `contacts.last_read_at` and `channels.last_read_at`
- Updated via `POST /api/contacts/{public_key}/mark-read` and `POST /api/channels/{key}/mark-read`
- Bulk update via `POST /api/read-state/mark-all-read`
- Aggregated counts via `GET /api/read-state/unreads` (server-side computation)
- Aggregated counts via `GET /api/read-state/unreads` (server-side computation of counts, mention flags, `last_message_times`, and `last_read_ats`)
**State Tracking Keys (Frontend)**: Generated by `getStateKey()` for message times (sidebar sorting):
- Channels: `channel-{channel_key}`

View File

@@ -1,3 +1,28 @@
## [3.5.0] - 2026-03-19
Feature: Add room server alpha support
Feature: Add option to force-reset node clock when it's too far ahead
Feature: DMs auto-retry before resorting to flood
Feature: Add impulse zero-hop advert
Feature: Utilize PATH packets to correctly source a contact's route
Feature: Metrics view on raw packet pane
Feature: Metric, Imperial, and Smoots are now selectable for distance display
Feature: Allow favorites to be sorted
Feature: Add multi-ack support
Feature: Password-remember checkbox on repeaters + room servers
Bugfix: Serialize radio disconnect in a lock
Bugfix: Fix contact bar layout issues
Bugfix: Fix sidebar ordering for contacts by advert recency
Bugfix: Fix version reporting in community MQTT
Bugfix: Fix Apprise duplicate names
Bugfix: Be better about identity resolution in the stats pane
Misc: Docs, test, and performance enhancements
Misc: Don't prompt "Are you sure" when leaving an unedited interation
Misc: Log node time on startup
Misc: Improve community MQTT error bubble-up
Misc: Unread DMs always have a red unread counter
Misc: Improve information in the debug view to show DB status
## [3.4.1] - 2026-03-16
Bugfix: Improve handling of version information on prebuilt bundles

View File

@@ -330,7 +330,7 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
</details>
### meshcore (2.2.29) — MIT
### meshcore (2.3.1) — MIT
<details>
<summary>Full license text</summary>

View File

@@ -113,16 +113,16 @@ app/
### Read/unread state
- Server is source of truth (`contacts.last_read_at`, `channels.last_read_at`).
- `GET /api/read-state/unreads` returns counts, mention flags, and `last_message_times`.
- `GET /api/read-state/unreads` returns counts, mention flags, `last_message_times`, and `last_read_ats`.
### DM ingest + ACKs
- `services/dm_ingest.py` is the one place that should decide fallback-context resolution, DM dedup/reconciliation, and packet-linked vs. content-based storage behavior.
- `CONTACT_MSG_RECV` is a fallback path, not a parallel source of truth. If you change DM storage behavior, trace both `event_handlers.py` and `packet_processor.py`.
- DM ACK tracking is an in-memory pending/buffered map in `services/dm_ack_tracker.py`, with periodic expiry from `radio_sync.py`.
- Outgoing DMs send once inline, store/broadcast immediately after the first successful `MSG_SENT`, then may retry up to 2 more times in the background if still unacked.
- Outgoing DMs send once inline, store/broadcast immediately after the first successful `MSG_SENT`, then may retry up to 2 more times in the background only when the initial `MSG_SENT` result includes an expected ACK code and the message remains unacked.
- DM retry timing follows the firmware-provided `suggested_timeout` from `PACKET_MSG_SENT`; do not replace it with a fixed app timeout unless you intentionally want more aggressive duplicate-prone retries.
- Direct-message send behavior is intended to emulate `meshcore_py.commands.send_msg_with_retry(...)`: stage the effective contact route on the radio, send, wait for ACK, and on the final retry force flood via `reset_path(...)`.
- Direct-message send behavior is intended to emulate `meshcore_py.commands.send_msg_with_retry(...)` when the radio provides an expected ACK code: stage the effective contact route on the radio, send, wait for ACK, and on the final retry force flood via `reset_path(...)`.
- Non-final DM attempts use the contact's effective route (`override > direct > flood`). The final retry is intentionally sent as flood even when a routing override exists.
- DM ACK state is terminal on first ACK. Retry attempts may register multiple expected ACK codes for the same message, but sibling pending codes are cleared once one ACK wins so a DM should not accrue multiple delivery confirmations from retries.
- ACKs are delivery state, not routing state. Bundled ACKs inside PATH packets still satisfy pending DM sends, but ACK history does not feed contact route learning.
@@ -188,6 +188,7 @@ app/
- `POST /contacts/{public_key}/command`
- `POST /contacts/{public_key}/routing-override`
- `POST /contacts/{public_key}/trace`
- `POST /contacts/{public_key}/path-discovery` — discover forward/return paths, persist the learned direct route, and sync it back to the radio best-effort
- `POST /contacts/{public_key}/repeater/login`
- `POST /contacts/{public_key}/repeater/status`
- `POST /contacts/{public_key}/repeater/lpp-telemetry`
@@ -219,7 +220,7 @@ app/
- `POST /packets/maintenance`
### Read state
- `GET /read-state/unreads`
- `GET /read-state/unreads` — counts, mention flags, `last_message_times`, and `last_read_ats`
- `POST /read-state/mark-all-read`
### Settings

View File

@@ -353,6 +353,13 @@ async def run_migrations(conn: aiosqlite.Connection) -> int:
await set_version(conn, 45)
applied += 1
# Migration 46: Clean orphaned contact child rows left by old prefix promotion
if version < 46:
logger.info("Applying migration 46: clean orphaned contact child rows")
await _migrate_046_cleanup_orphaned_contact_child_rows(conn)
await set_version(conn, 46)
applied += 1
if applied > 0:
logger.info(
"Applied %d migration(s), schema now at version %d", applied, await get_version(conn)
@@ -2773,3 +2780,91 @@ async def _migrate_045_rebuild_contacts_direct_route_columns(conn: aiosqlite.Con
await conn.execute("DROP TABLE contacts")
await conn.execute("ALTER TABLE contacts_new RENAME TO contacts")
await conn.commit()
async def _migrate_046_cleanup_orphaned_contact_child_rows(conn: aiosqlite.Connection) -> None:
"""Move uniquely resolvable orphan contact child rows onto full contacts, drop the rest."""
existing_tables_cursor = await conn.execute("SELECT name FROM sqlite_master WHERE type='table'")
existing_tables = {row[0] for row in await existing_tables_cursor.fetchall()}
if "contacts" not in existing_tables:
await conn.commit()
return
child_tables = [
table
for table in ("contact_name_history", "contact_advert_paths")
if table in existing_tables
]
if not child_tables:
await conn.commit()
return
orphan_keys: set[str] = set()
for table in child_tables:
cursor = await conn.execute(
f"""
SELECT DISTINCT child.public_key
FROM {table} child
LEFT JOIN contacts c ON c.public_key = child.public_key
WHERE c.public_key IS NULL
"""
)
orphan_keys.update(row[0] for row in await cursor.fetchall())
for orphan_key in sorted(orphan_keys, key=len, reverse=True):
match_cursor = await conn.execute(
"""
SELECT public_key
FROM contacts
WHERE length(public_key) = 64
AND public_key LIKE ? || '%'
ORDER BY public_key
""",
(orphan_key.lower(),),
)
matches = [row[0] for row in await match_cursor.fetchall()]
resolved_key = matches[0] if len(matches) == 1 else None
if resolved_key is not None:
if "contact_name_history" in child_tables:
await conn.execute(
"""
INSERT INTO contact_name_history (public_key, name, first_seen, last_seen)
SELECT ?, name, first_seen, last_seen
FROM contact_name_history
WHERE public_key = ?
ON CONFLICT(public_key, name) DO UPDATE SET
first_seen = MIN(contact_name_history.first_seen, excluded.first_seen),
last_seen = MAX(contact_name_history.last_seen, excluded.last_seen)
""",
(resolved_key, orphan_key),
)
if "contact_advert_paths" in child_tables:
await conn.execute(
"""
INSERT INTO contact_advert_paths
(public_key, path_hex, path_len, first_seen, last_seen, heard_count)
SELECT ?, path_hex, path_len, first_seen, last_seen, heard_count
FROM contact_advert_paths
WHERE public_key = ?
ON CONFLICT(public_key, path_hex, path_len) DO UPDATE SET
first_seen = MIN(contact_advert_paths.first_seen, excluded.first_seen),
last_seen = MAX(contact_advert_paths.last_seen, excluded.last_seen),
heard_count = contact_advert_paths.heard_count + excluded.heard_count
""",
(resolved_key, orphan_key),
)
if "contact_name_history" in child_tables:
await conn.execute(
"DELETE FROM contact_name_history WHERE public_key = ?",
(orphan_key,),
)
if "contact_advert_paths" in child_tables:
await conn.execute(
"DELETE FROM contact_advert_paths WHERE public_key = ?",
(orphan_key,),
)
await conn.commit()

View File

@@ -431,6 +431,43 @@ class ContactRepository:
Returns the placeholder public keys that were merged into the full key.
"""
async def migrate_child_rows(old_key: str, new_key: str) -> None:
await db.conn.execute(
"""
INSERT INTO contact_name_history (public_key, name, first_seen, last_seen)
SELECT ?, name, first_seen, last_seen
FROM contact_name_history
WHERE public_key = ?
ON CONFLICT(public_key, name) DO UPDATE SET
first_seen = MIN(contact_name_history.first_seen, excluded.first_seen),
last_seen = MAX(contact_name_history.last_seen, excluded.last_seen)
""",
(new_key, old_key),
)
await db.conn.execute(
"""
INSERT INTO contact_advert_paths
(public_key, path_hex, path_len, first_seen, last_seen, heard_count)
SELECT ?, path_hex, path_len, first_seen, last_seen, heard_count
FROM contact_advert_paths
WHERE public_key = ?
ON CONFLICT(public_key, path_hex, path_len) DO UPDATE SET
first_seen = MIN(contact_advert_paths.first_seen, excluded.first_seen),
last_seen = MAX(contact_advert_paths.last_seen, excluded.last_seen),
heard_count = contact_advert_paths.heard_count + excluded.heard_count
""",
(new_key, old_key),
)
await db.conn.execute(
"DELETE FROM contact_name_history WHERE public_key = ?",
(old_key,),
)
await db.conn.execute(
"DELETE FROM contact_advert_paths WHERE public_key = ?",
(old_key,),
)
normalized_full_key = full_key.lower()
cursor = await db.conn.execute(
"""
@@ -467,6 +504,8 @@ class ContactRepository:
if (match_row["match_count"] if match_row is not None else 0) != 1:
continue
await migrate_child_rows(old_key, normalized_full_key)
if full_exists:
await db.conn.execute(
"""

View File

@@ -554,6 +554,12 @@ class MessageRepository:
return MessageRepository._row_to_message(row)
@staticmethod
async def delete_by_id(message_id: int) -> None:
"""Delete a message row by ID."""
await db.conn.execute("DELETE FROM messages WHERE id = ?", (message_id,))
await db.conn.commit()
@staticmethod
async def get_by_content(
msg_type: str,

View File

@@ -10,7 +10,7 @@ from pydantic import BaseModel, Field
from app.config import get_recent_log_lines, settings
from app.radio_sync import get_contacts_selected_for_radio_sync, get_radio_channel_limit
from app.repository import MessageRepository
from app.repository import MessageRepository, StatisticsRepository
from app.routers.health import HealthResponse, build_health_data
from app.services.radio_runtime import radio_runtime
from app.version_info import get_app_build_info, git_output
@@ -87,11 +87,18 @@ class DebugRadioProbe(BaseModel):
channels: DebugChannelAudit | None = None
class DebugDatabaseInfo(BaseModel):
total_dms: int
total_channel_messages: int
total_outgoing: int
class DebugSnapshotResponse(BaseModel):
captured_at: str
application: DebugApplicationInfo
health: HealthResponse
runtime: DebugRuntimeInfo
database: DebugDatabaseInfo
radio_probe: DebugRadioProbe
logs: list[str]
@@ -258,6 +265,7 @@ async def _probe_radio() -> DebugRadioProbe:
async def debug_support_snapshot() -> DebugSnapshotResponse:
"""Return a support/debug snapshot with recent logs and live radio state."""
health_data = await build_health_data(radio_runtime.is_connected, radio_runtime.connection_info)
statistics = await StatisticsRepository.get_all()
radio_probe = await _probe_radio()
channels_with_incoming_messages = (
await MessageRepository.count_channels_with_incoming_messages()
@@ -282,6 +290,11 @@ async def debug_support_snapshot() -> DebugSnapshotResponse:
"force_channel_slot_reconfigure": settings.force_channel_slot_reconfigure,
},
),
database=DebugDatabaseInfo(
total_dms=statistics["total_dms"],
total_channel_messages=statistics["total_channel_messages"],
total_outgoing=statistics["total_outgoing"],
),
radio_probe=radio_probe,
logs=[*LOG_COPY_BOUNDARY_PREFIX, *get_recent_log_lines(limit=1000)],
)

View File

@@ -13,7 +13,8 @@ from app.region_scope import normalize_region_scope
from app.repository import AppSettingsRepository, ContactRepository, MessageRepository
from app.services import dm_ack_tracker
from app.services.messages import (
build_message_model,
broadcast_message,
build_stored_outgoing_channel_message,
create_outgoing_channel_message,
create_outgoing_direct_message,
increment_ack_and_broadcast,
@@ -586,6 +587,23 @@ async def send_channel_message_to_channel(
requested_timestamp=sent_at,
)
timestamp_bytes = sender_timestamp.to_bytes(4, "little")
outgoing_message = await create_outgoing_channel_message(
conversation_key=channel_key_upper,
text=text_with_sender,
sender_timestamp=sender_timestamp,
received_at=sent_at,
sender_name=radio_name or None,
sender_key=our_public_key,
channel_name=channel.name,
broadcast_fn=broadcast_fn,
broadcast=False,
message_repository=message_repository,
)
if outgoing_message is None:
raise HTTPException(
status_code=500,
detail="Failed to store outgoing message - unexpected duplicate",
)
result = await send_channel_message_with_effective_scope(
mc=mc,
@@ -611,23 +629,11 @@ async def send_channel_message_to_channel(
raise HTTPException(
status_code=500, detail=f"Failed to send message: {result.payload}"
)
outgoing_message = await create_outgoing_channel_message(
conversation_key=channel_key_upper,
text=text_with_sender,
sender_timestamp=sender_timestamp,
received_at=sent_at,
sender_name=radio_name or None,
sender_key=our_public_key,
channel_name=channel.name,
broadcast_fn=broadcast_fn,
message_repository=message_repository,
)
if outgoing_message is None:
raise HTTPException(
status_code=500,
detail="Failed to store outgoing message - unexpected duplicate",
)
except Exception:
if outgoing_message is not None:
await message_repository.delete_by_id(outgoing_message.id)
outgoing_message = None
raise
finally:
if sender_timestamp is not None:
await release_outgoing_sender_timestamp(
@@ -640,22 +646,19 @@ async def send_channel_message_to_channel(
if sent_at is None or sender_timestamp is None or outgoing_message is None:
raise HTTPException(status_code=500, detail="Failed to store outgoing message")
message_id = outgoing_message.id
acked_count, paths = await message_repository.get_ack_and_paths(message_id)
return build_message_model(
message_id=message_id,
msg_type="CHAN",
outgoing_message = await build_stored_outgoing_channel_message(
message_id=outgoing_message.id,
conversation_key=channel_key_upper,
text=text_with_sender,
sender_timestamp=sender_timestamp,
received_at=sent_at,
paths=paths,
outgoing=True,
acked=acked_count,
sender_name=radio_name or None,
sender_key=our_public_key,
channel_name=channel.name,
message_repository=message_repository,
)
broadcast_message(message=outgoing_message, broadcast_fn=broadcast_fn)
return outgoing_message
async def resend_channel_message_record(
@@ -705,6 +708,23 @@ async def resend_channel_message_record(
requested_timestamp=sent_at,
)
timestamp_bytes = sender_timestamp.to_bytes(4, "little")
new_message = await create_outgoing_channel_message(
conversation_key=message.conversation_key,
text=message.text,
sender_timestamp=sender_timestamp,
received_at=sent_at,
sender_name=radio_name or None,
sender_key=resend_public_key,
channel_name=channel.name,
broadcast_fn=broadcast_fn,
broadcast=False,
message_repository=message_repository,
)
if new_message is None:
raise HTTPException(
status_code=500,
detail="Failed to store resent message - unexpected duplicate",
)
result = await send_channel_message_with_effective_scope(
mc=mc,
@@ -729,26 +749,11 @@ async def resend_channel_message_record(
status_code=500,
detail=f"Failed to resend message: {result.payload}",
)
if new_timestamp:
if sent_at is None:
raise HTTPException(status_code=500, detail="Failed to assign resend timestamp")
new_message = await create_outgoing_channel_message(
conversation_key=message.conversation_key,
text=message.text,
sender_timestamp=sender_timestamp,
received_at=sent_at,
sender_name=radio_name or None,
sender_key=resend_public_key,
channel_name=channel.name,
broadcast_fn=broadcast_fn,
message_repository=message_repository,
)
if new_message is None:
raise HTTPException(
status_code=500,
detail="Failed to store resent message - unexpected duplicate",
)
except Exception:
if new_message is not None:
await message_repository.delete_by_id(new_message.id)
new_message = None
raise
finally:
if new_timestamp and sent_at is not None:
await release_outgoing_sender_timestamp(
@@ -762,6 +767,19 @@ async def resend_channel_message_record(
if sent_at is None or new_message is None:
raise HTTPException(status_code=500, detail="Failed to assign resend timestamp")
new_message = await build_stored_outgoing_channel_message(
message_id=new_message.id,
conversation_key=message.conversation_key,
text=message.text,
sender_timestamp=sender_timestamp,
received_at=sent_at,
sender_name=radio_name or None,
sender_key=resend_public_key,
channel_name=channel.name,
message_repository=message_repository,
)
broadcast_message(message=new_message, broadcast_fn=broadcast_fn)
logger.info(
"Resent channel message %d as new message %d to %s",
message.id,

View File

@@ -96,6 +96,36 @@ def broadcast_message(
broadcast_fn("message", payload, realtime=realtime)
async def build_stored_outgoing_channel_message(
*,
message_id: int,
conversation_key: str,
text: str,
sender_timestamp: int,
received_at: int,
sender_name: str | None,
sender_key: str | None,
channel_name: str | None,
message_repository=MessageRepository,
) -> Message:
"""Build the current payload for a stored outgoing channel message."""
acked_count, paths = await message_repository.get_ack_and_paths(message_id)
return build_message_model(
message_id=message_id,
msg_type="CHAN",
conversation_key=conversation_key,
text=text,
sender_timestamp=sender_timestamp,
received_at=received_at,
paths=paths,
outgoing=True,
acked=acked_count,
sender_name=sender_name,
sender_key=sender_key,
channel_name=channel_name,
)
def broadcast_message_acked(
*,
message_id: int,
@@ -428,6 +458,7 @@ async def create_outgoing_channel_message(
sender_key: str | None,
channel_name: str | None,
broadcast_fn: BroadcastFn,
broadcast: bool = True,
message_repository=MessageRepository,
) -> Message | None:
"""Store and broadcast an outgoing channel message."""
@@ -444,18 +475,17 @@ async def create_outgoing_channel_message(
if msg_id is None:
return None
message = build_message_model(
message = await build_stored_outgoing_channel_message(
message_id=msg_id,
msg_type="CHAN",
conversation_key=conversation_key,
text=text,
sender_timestamp=sender_timestamp,
received_at=received_at,
outgoing=True,
acked=0,
sender_name=sender_name,
sender_key=sender_key,
channel_name=channel_name,
message_repository=message_repository,
)
broadcast_message(message=message, broadcast_fn=broadcast_fn)
if broadcast:
broadcast_message(message=message, broadcast_fn=broadcast_fn)
return message

View File

@@ -198,9 +198,9 @@ High-level state is delegated to hooks:
- `useContactsAndChannels`: contact/channel lists, creation, deletion
- `useConversationRouter`: URL hash → active conversation routing
- `useConversationNavigation`: search target, conversation selection reset, and info-pane state
- `useConversationActions`: send/resend/trace/block handlers and channel override updates
- `useConversationActions`: send/resend/trace/path-discovery/block handlers and channel override updates
- `useConversationMessages`: conversation switch loading, embedded conversation-scoped cache, jump-target loading, pagination, dedup/update helpers, reconnect reconciliation, and pending ACK buffering
- `useUnreadCounts`: unread counters, mention tracking, recent-sort timestamps
- `useUnreadCounts`: unread counters, mention tracking, recent-sort timestamps, and server `last_read_ats` boundaries
- `useRealtimeAppState`: typed WS event application, reconnect recovery, cache/unread coordination
- `useRepeaterDashboard`: repeater dashboard state (login, pane data/retries, console, actions)
@@ -331,6 +331,8 @@ Note: MQTT, bot, and community MQTT settings were migrated to the `fanout_config
`RawPacket.decrypted_info` includes `channel_key` and `contact_key` for MQTT topic routing.
`UnreadCounts` includes `counts`, `mentions`, `last_message_times`, and `last_read_ats`. The unread-boundary/jump-to-unread behavior uses the server-provided `last_read_ats` map keyed by `getStateKey(...)`.
## Contact Info Pane
Clicking a contact's avatar in `ChatHeader` or `MessageList` opens a `ContactInfoPane` sheet (right drawer) showing comprehensive contact details fetched from `GET /api/contacts/analytics` using either `?public_key=...` or `?name=...`:

View File

@@ -1,7 +1,7 @@
{
"name": "remoteterm-meshcore-frontend",
"private": true,
"version": "3.4.1",
"version": "3.5.0",
"type": "module",
"scripts": {
"dev": "vite",

View File

@@ -9,6 +9,7 @@ import { handleKeyboardActivate } from '../utils/a11y';
import { isPublicChannelKey } from '../utils/publicChannel';
import { stripRegionScopePrefix } from '../utils/regionScope';
import { isPrefixOnlyContact } from '../utils/pubkey';
import { cn } from '../lib/utils';
import { ContactAvatar } from './ContactAvatar';
import { ContactStatusInfo } from './ContactStatusInfo';
import type {
@@ -118,8 +119,15 @@ export function ChatHeader({
};
return (
<header className="conversation-header flex justify-between items-start px-4 py-2.5 border-b border-border gap-2">
<span className="flex min-w-0 flex-1 items-start gap-2">
<header
className={cn(
'conversation-header grid items-start gap-x-2 gap-y-0.5 border-b border-border px-4 py-2.5',
conversation.type === 'contact' && activeContact
? 'grid-cols-[minmax(0,1fr)_auto] min-[1100px]:grid-cols-[minmax(0,1fr)_auto_auto]'
: 'grid-cols-[minmax(0,1fr)_auto]'
)}
>
<span className="flex min-w-0 items-start gap-2">
{conversation.type === 'contact' && onOpenContactInfo && (
<button
type="button"
@@ -137,16 +145,31 @@ export function ChatHeader({
/>
</button>
)}
<span className="grid min-w-0 flex-1 grid-cols-1 gap-y-0.5 min-[1200px]:grid-cols-[minmax(0,1fr)_auto] min-[1200px]:items-baseline min-[1200px]:gap-x-2">
<span className="flex min-w-0 items-baseline gap-2 whitespace-nowrap">
<h2 className="min-w-0 shrink font-semibold text-base">
{titleClickable ? (
<button
type="button"
className="flex min-w-0 shrink items-center gap-1.5 text-left hover:text-primary transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring rounded-sm"
aria-label={`View info for ${conversation.name}`}
onClick={handleOpenConversationInfo}
>
<span className="flex min-w-0 flex-1 flex-col">
<span className="flex min-w-0 flex-wrap items-baseline gap-x-2 gap-y-0.5">
<span className="flex min-w-0 flex-1 items-baseline gap-2 whitespace-nowrap">
<h2 className="min-w-0 flex-shrink font-semibold text-base">
{titleClickable ? (
<button
type="button"
className="flex max-w-full min-w-0 items-center gap-1.5 overflow-hidden rounded-sm text-left transition-colors hover:text-primary focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring"
aria-label={`View info for ${conversation.name}`}
onClick={handleOpenConversationInfo}
>
<span className="truncate">
{conversation.type === 'channel' &&
!conversation.name.startsWith('#') &&
activeChannel?.is_hashtag
? '#'
: ''}
{conversation.name}
</span>
<Info
className="h-3.5 w-3.5 flex-shrink-0 text-muted-foreground/80"
aria-hidden="true"
/>
</button>
) : (
<span className="truncate">
{conversation.type === 'channel' &&
!conversation.name.startsWith('#') &&
@@ -155,83 +178,72 @@ export function ChatHeader({
: ''}
{conversation.name}
</span>
<Info
className="h-3.5 w-3.5 flex-shrink-0 text-muted-foreground/80"
aria-hidden="true"
/>
)}
</h2>
{isPrivateChannel && !showKey ? (
<button
className="min-w-0 flex-shrink text-[11px] font-mono text-muted-foreground transition-colors hover:text-primary"
onClick={(e) => {
e.stopPropagation();
setShowKey(true);
}}
title="Reveal channel key"
>
Show Key
</button>
) : (
<span className="truncate">
{conversation.type === 'channel' &&
!conversation.name.startsWith('#') &&
activeChannel?.is_hashtag
? '#'
: ''}
{conversation.name}
<span
className="min-w-0 flex-1 truncate font-mono text-[11px] text-muted-foreground transition-colors hover:text-primary"
role="button"
tabIndex={0}
onKeyDown={handleKeyboardActivate}
onClick={(e) => {
e.stopPropagation();
navigator.clipboard.writeText(conversation.id);
toast.success(
conversation.type === 'channel' ? 'Room key copied!' : 'Contact key copied!'
);
}}
title="Click to copy"
aria-label={
conversation.type === 'channel' ? 'Copy channel key' : 'Copy contact key'
}
>
{conversation.type === 'channel'
? conversation.id.toLowerCase()
: conversation.id}
</span>
)}
</h2>
{isPrivateChannel && !showKey ? (
</span>
{conversation.type === 'channel' && activeFloodScopeDisplay && (
<button
className="min-w-0 flex-shrink text-[11px] font-mono text-muted-foreground transition-colors hover:text-primary"
onClick={(e) => {
e.stopPropagation();
setShowKey(true);
}}
title="Reveal channel key"
className="mt-0.5 flex basis-full items-center gap-1 text-left sm:hidden"
onClick={handleEditFloodScopeOverride}
title="Set regional override"
aria-label="Set regional override"
>
Show Key
<Globe2
className="h-3.5 w-3.5 flex-shrink-0 text-[hsl(var(--region-override))]"
aria-hidden="true"
/>
<span className="min-w-0 truncate text-[11px] font-medium text-[hsl(var(--region-override))]">
{activeFloodScopeDisplay}
</span>
</button>
) : (
<span
className="min-w-0 flex-1 truncate font-mono text-[11px] text-muted-foreground transition-colors hover:text-primary"
role="button"
tabIndex={0}
onKeyDown={handleKeyboardActivate}
onClick={(e) => {
e.stopPropagation();
navigator.clipboard.writeText(conversation.id);
toast.success(
conversation.type === 'channel' ? 'Room key copied!' : 'Contact key copied!'
);
}}
title="Click to copy"
aria-label={
conversation.type === 'channel' ? 'Copy channel key' : 'Copy contact key'
}
>
{conversation.type === 'channel' ? conversation.id.toLowerCase() : conversation.id}
</span>
)}
</span>
{conversation.type === 'contact' && activeContact && (
<span className="min-w-0 text-[11px] text-muted-foreground min-[1200px]:justify-self-end">
<ContactStatusInfo
contact={activeContact}
ourLat={config?.lat ?? null}
ourLon={config?.lon ?? null}
/>
</span>
)}
{conversation.type === 'channel' && activeFloodScopeDisplay && (
<button
className="mt-0.5 flex items-center gap-1 text-left sm:hidden"
onClick={handleEditFloodScopeOverride}
title="Set regional override"
aria-label="Set regional override"
>
<Globe2
className="h-3.5 w-3.5 flex-shrink-0 text-[hsl(var(--region-override))]"
aria-hidden="true"
/>
<span className="min-w-0 truncate text-[11px] font-medium text-[hsl(var(--region-override))]">
{activeFloodScopeDisplay}
</span>
</button>
)}
</span>
</span>
<div className="flex items-center justify-end gap-0.5 flex-shrink-0">
{conversation.type === 'contact' && activeContact && (
<div className="col-span-2 row-start-2 min-w-0 text-[11px] text-muted-foreground min-[1100px]:col-span-1 min-[1100px]:col-start-2 min-[1100px]:row-start-1">
<ContactStatusInfo
contact={activeContact}
ourLat={config?.lat ?? null}
ourLon={config?.lon ?? null}
/>
</div>
)}
<div className="flex items-center justify-end gap-0.5">
{conversation.type === 'contact' && !activeContactIsRoomServer && (
<button
className="p-1 rounded hover:bg-accent text-lg leading-none transition-colors disabled:cursor-not-allowed disabled:opacity-50 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring"

View File

@@ -77,12 +77,16 @@ function formatRssi(value: number | null): string {
return value === null ? '-' : `${Math.round(value)} dBm`;
}
function resolveContactLabel(sourceKey: string | null, contacts: Contact[]): string | null {
function normalizeResolvableSourceKey(sourceKey: string): string {
return sourceKey.startsWith('hash1:') ? sourceKey.slice(6) : sourceKey;
}
function resolveContact(sourceKey: string | null, contacts: Contact[]): Contact | null {
if (!sourceKey || sourceKey.startsWith('name:')) {
return null;
}
const normalizedSourceKey = sourceKey.toLowerCase();
const normalizedSourceKey = normalizeResolvableSourceKey(sourceKey).toLowerCase();
const matches = contacts.filter((contact) =>
contact.public_key.toLowerCase().startsWith(normalizedSourceKey)
);
@@ -90,7 +94,14 @@ function resolveContactLabel(sourceKey: string | null, contacts: Contact[]): str
return null;
}
const contact = matches[0];
return matches[0];
}
function resolveContactLabel(sourceKey: string | null, contacts: Contact[]): string | null {
const contact = resolveContact(sourceKey, contacts);
if (!contact) {
return null;
}
return getContactDisplayName(contact.name, contact.public_key, contact.last_advert);
}
@@ -101,11 +112,46 @@ function resolveNeighbor(item: NeighborStat, contacts: Contact[]): NeighborStat
};
}
function mergeResolvedNeighbors(items: NeighborStat[], contacts: Contact[]): NeighborStat[] {
const merged = new Map<string, NeighborStat>();
for (const item of items) {
const contact = resolveContact(item.key, contacts);
const canonicalKey = contact?.public_key ?? item.key;
const resolvedLabel =
contact != null
? getContactDisplayName(contact.name, contact.public_key, contact.last_advert)
: item.label;
const existing = merged.get(canonicalKey);
if (!existing) {
merged.set(canonicalKey, {
...item,
key: canonicalKey,
label: resolvedLabel,
});
continue;
}
existing.count += item.count;
existing.lastSeen = Math.max(existing.lastSeen, item.lastSeen);
existing.bestRssi =
existing.bestRssi === null
? item.bestRssi
: item.bestRssi === null
? existing.bestRssi
: Math.max(existing.bestRssi, item.bestRssi);
existing.label = resolvedLabel;
}
return Array.from(merged.values());
}
function isNeighborIdentityResolvable(item: NeighborStat, contacts: Contact[]): boolean {
if (item.key.startsWith('name:')) {
return true;
}
return resolveContactLabel(item.key, contacts) !== null;
return resolveContact(item.key, contacts) !== null;
}
function formatStrongestPacketDetail(
@@ -219,14 +265,29 @@ function NeighborList({
mode: 'heard' | 'signal' | 'recent';
contacts: Contact[];
}) {
const mergedItems = mergeResolvedNeighbors(items, contacts);
const sortedItems = [...mergedItems].sort((a, b) => {
if (mode === 'heard') {
return b.count - a.count || b.lastSeen - a.lastSeen || a.label.localeCompare(b.label);
}
if (mode === 'signal') {
return (
(b.bestRssi ?? Number.NEGATIVE_INFINITY) - (a.bestRssi ?? Number.NEGATIVE_INFINITY) ||
b.count - a.count ||
a.label.localeCompare(b.label)
);
}
return b.lastSeen - a.lastSeen || b.count - a.count || a.label.localeCompare(b.label);
});
return (
<section className="mb-4 break-inside-avoid rounded-lg border border-border/70 bg-card/70 p-3">
<h3 className="text-sm font-semibold text-foreground">{title}</h3>
{items.length === 0 ? (
{sortedItems.length === 0 ? (
<p className="mt-3 text-sm text-muted-foreground">{emptyLabel}</p>
) : (
<div className="mt-3 space-y-2">
{items.map((item) => (
{sortedItems.map((item) => (
<div
key={item.key}
className="flex items-center justify-between gap-3 rounded-md bg-background/70 px-2 py-1.5"

View File

@@ -12,6 +12,7 @@ import { handleKeyboardActivate } from '../utils/a11y';
import { isValidLocation } from '../utils/pathUtils';
import { ContactStatusInfo } from './ContactStatusInfo';
import type { Contact, Conversation, Favorite, PathDiscoveryResponse } from '../types';
import { cn } from '../lib/utils';
import { TelemetryPane } from './repeater/RepeaterTelemetryPane';
import { NeighborsPane } from './repeater/RepeaterNeighborsPane';
import { AclPane } from './repeater/RepeaterAclPane';
@@ -101,8 +102,15 @@ export function RepeaterDashboard({
return (
<div className="flex-1 flex flex-col min-h-0">
{/* Header */}
<header className="flex justify-between items-start px-4 py-2.5 border-b border-border gap-2">
<span className="flex min-w-0 flex-1 flex-col">
<header
className={cn(
'grid items-start gap-x-2 gap-y-0.5 border-b border-border px-4 py-2.5',
contact
? 'grid-cols-[minmax(0,1fr)_auto] min-[1100px]:grid-cols-[minmax(0,1fr)_auto_auto]'
: 'grid-cols-[minmax(0,1fr)_auto]'
)}
>
<span className="flex min-w-0 flex-col">
<span className="flex min-w-0 flex-wrap items-baseline gap-x-2 gap-y-0.5">
<span className="flex min-w-0 flex-1 items-baseline gap-2">
<span className="min-w-0 flex-shrink truncate font-semibold text-base">
@@ -122,14 +130,14 @@ export function RepeaterDashboard({
{conversation.id}
</span>
</span>
{contact && (
<span className="min-w-0 flex-none text-[11px] text-muted-foreground max-sm:basis-full">
<ContactStatusInfo contact={contact} ourLat={radioLat} ourLon={radioLon} />
</span>
)}
</span>
</span>
<div className="flex items-center gap-0.5 flex-shrink-0">
{contact && (
<div className="col-span-2 row-start-2 min-w-0 text-[11px] text-muted-foreground min-[1100px]:col-span-1 min-[1100px]:col-start-2 min-[1100px]:row-start-1">
<ContactStatusInfo contact={contact} ourLat={radioLat} ourLon={radioLon} />
</div>
)}
<div className="flex items-center gap-0.5">
{loggedIn && (
<Button
variant="outline"

View File

@@ -21,6 +21,24 @@ interface InternalCachedConversationEntry extends CachedConversationEntry {
export class ConversationMessageCache {
private readonly cache = new Map<string, InternalCachedConversationEntry>();
private normalizeEntry(entry: CachedConversationEntry): InternalCachedConversationEntry {
let messages = entry.messages;
let hasOlderMessages = entry.hasOlderMessages;
if (messages.length > MAX_MESSAGES_PER_ENTRY) {
messages = [...messages]
.sort((a, b) => b.received_at - a.received_at)
.slice(0, MAX_MESSAGES_PER_ENTRY);
hasOlderMessages = true;
}
return {
messages,
hasOlderMessages,
contentKeys: new Set(messages.map((message) => getMessageContentKey(message))),
};
}
get(id: string): CachedConversationEntry | undefined {
const entry = this.cache.get(id);
if (!entry) return undefined;
@@ -33,17 +51,7 @@ export class ConversationMessageCache {
}
set(id: string, entry: CachedConversationEntry): void {
const contentKeys = new Set(entry.messages.map((message) => getMessageContentKey(message)));
if (entry.messages.length > MAX_MESSAGES_PER_ENTRY) {
const trimmed = [...entry.messages]
.sort((a, b) => b.received_at - a.received_at)
.slice(0, MAX_MESSAGES_PER_ENTRY);
entry = { ...entry, messages: trimmed, hasOlderMessages: true };
}
const internalEntry: InternalCachedConversationEntry = {
...entry,
contentKeys,
};
const internalEntry = this.normalizeEntry(entry);
this.cache.delete(id);
this.cache.set(id, internalEntry);
if (this.cache.size > MAX_CACHED_CONVERSATIONS) {
@@ -69,15 +77,12 @@ export class ConversationMessageCache {
}
if (entry.contentKeys.has(contentKey)) return false;
if (entry.messages.some((message) => message.id === msg.id)) return false;
entry.contentKeys.add(contentKey);
entry.messages = [...entry.messages, msg];
if (entry.messages.length > MAX_MESSAGES_PER_ENTRY) {
entry.messages = [...entry.messages]
.sort((a, b) => b.received_at - a.received_at)
.slice(0, MAX_MESSAGES_PER_ENTRY);
}
const nextEntry = this.normalizeEntry({
messages: [...entry.messages, msg],
hasOlderMessages: entry.hasOlderMessages,
});
this.cache.delete(id);
this.cache.set(id, entry);
this.cache.set(id, nextEntry);
return true;
}
@@ -123,11 +128,13 @@ export class ConversationMessageCache {
}
this.cache.delete(oldId);
this.cache.set(newId, {
messages: mergedMessages,
hasOlderMessages: newEntry.hasOlderMessages || oldEntry.hasOlderMessages,
contentKeys: new Set([...newEntry.contentKeys, ...oldEntry.contentKeys]),
});
this.cache.set(
newId,
this.normalizeEntry({
messages: mergedMessages,
hasOlderMessages: newEntry.hasOlderMessages || oldEntry.hasOlderMessages,
})
);
}
clear(): void {

View File

@@ -167,17 +167,15 @@ describe('Integration: Duplicate Message Handling', () => {
});
});
describe('Integration: No phantom unreads from mesh echoes (hitlist #8 regression)', () => {
it('does not increment unread when a mesh echo arrives after many unique messages', () => {
describe('Integration: Trimmed cache entries can reappear (hitlist #7 regression)', () => {
it('increments unread when an evicted inactive-conversation message arrives again', () => {
const state = createMockState();
const convKey = 'channel_busy';
// Deliver 1001 unique messages — exceeding the old global
// seenMessageContentRef prune threshold (1000→500). Under the old
// dual-set design the global set would drop msg-0's key during pruning,
// so a later mesh echo of msg-0 would pass the global check and
// phantom-increment unread. With the fix, messageCache's per-conversation
// Cached messages remain the source of truth for inactive-conversation dedup.
// Deliver enough unique messages to evict msg-0 from the inactive
// conversation cache. Once it falls out of that window, a later arrival
// with the same content should be allowed back in instead of being
// suppressed forever by a stale content key.
const MESSAGE_COUNT = 1001;
for (let i = 0; i < MESSAGE_COUNT; i++) {
const msg: Message = {
@@ -219,9 +217,8 @@ describe('Integration: No phantom unreads from mesh echoes (hitlist #8 regressio
};
const result = handleMessageEvent(state, echo, 'other_active_conv');
// Must NOT increment unread — the echo is a duplicate
expect(result.unreadIncremented).toBe(false);
expect(state.unreadCounts[stateKey]).toBe(MESSAGE_COUNT);
expect(result.unreadIncremented).toBe(true);
expect(state.unreadCounts[stateKey]).toBe(MESSAGE_COUNT + 1);
});
});

View File

@@ -214,6 +214,76 @@ describe('messageCache', () => {
expect(entry!.messages.some((m) => m.id === 0)).toBe(false);
});
it('allows a trimmed-out message to be re-added after set() trimming', () => {
const messages = Array.from({ length: MAX_MESSAGES_PER_ENTRY + 1 }, (_, i) =>
createMessage({
id: i,
text: `message-${i}`,
received_at: 1700000000 + i,
sender_timestamp: 1700000000 + i,
})
);
messageCache.set('conv1', createEntry(messages));
const trimmedOut = createMessage({
id: 10_000,
text: 'message-0',
received_at: 1800000000,
sender_timestamp: 1700000000,
});
expect(messageCache.addMessage('conv1', trimmedOut)).toBe(true);
const entry = messageCache.get('conv1');
expect(entry!.messages.some((m) => m.id === 10_000)).toBe(true);
});
it('allows a trimmed-out message to be re-added after addMessage() trimming', () => {
const messages = Array.from({ length: MAX_MESSAGES_PER_ENTRY - 1 }, (_, i) =>
createMessage({
id: i,
text: `message-${i}`,
received_at: 1700000000 + i,
sender_timestamp: 1700000000 + i,
})
);
messageCache.set('conv1', createEntry(messages));
expect(
messageCache.addMessage(
'conv1',
createMessage({
id: MAX_MESSAGES_PER_ENTRY,
text: 'newest-a',
received_at: 1800000000,
sender_timestamp: 1800000000,
})
)
).toBe(true);
expect(
messageCache.addMessage(
'conv1',
createMessage({
id: MAX_MESSAGES_PER_ENTRY + 1,
text: 'newest-b',
received_at: 1800000001,
sender_timestamp: 1800000001,
})
)
).toBe(true);
const readdedTrimmedMessage = createMessage({
id: 10_001,
text: 'message-0',
received_at: 1900000000,
sender_timestamp: 1700000000,
});
expect(messageCache.addMessage('conv1', readdedTrimmedMessage)).toBe(true);
const entry = messageCache.get('conv1');
expect(entry!.messages.some((m) => m.id === 10_001)).toBe(true);
});
it('auto-creates a minimal entry for never-visited conversations and returns true', () => {
const msg = createMessage({ id: 10, text: 'First contact' });
const result = messageCache.addMessage('new_conv', msg);

View File

@@ -297,6 +297,54 @@ describe('RawPacketFeedView', () => {
expect(screen.getAllByText('Identity not resolvable').length).toBeGreaterThan(0);
});
it('collapses uniquely resolved hash buckets into the same visible contact row', () => {
const alphaContact = createContact({
public_key: 'aa11bb22cc33' + '0'.repeat(52),
name: 'Alpha',
});
renderView({
rawPacketStatsSession: createSession({
totalObservedPackets: 2,
observations: [
{
observationKey: 'obs-1',
timestamp: 1_700_000_000,
payloadType: 'TextMessage',
routeType: 'Direct',
decrypted: true,
rssi: -70,
snr: 6,
sourceKey: 'hash1:AA',
sourceLabel: 'AA',
pathTokenCount: 0,
pathSignature: null,
},
{
observationKey: 'obs-2',
timestamp: 1_700_000_030,
payloadType: 'TextMessage',
routeType: 'Direct',
decrypted: true,
rssi: -67,
snr: 7,
sourceKey: alphaContact.public_key.toUpperCase(),
sourceLabel: alphaContact.public_key.slice(0, 12).toUpperCase(),
pathTokenCount: 0,
pathSignature: null,
},
],
}),
contacts: [alphaContact],
});
fireEvent.click(screen.getByRole('button', { name: /show stats/i }));
fireEvent.change(screen.getByLabelText('Stats window'), { target: { value: 'session' } });
expect(screen.getAllByText('Alpha').length).toBeGreaterThan(0);
expect(screen.queryByText('Identity not resolvable')).not.toBeInTheDocument();
});
it('opens a packet detail modal from the raw feed and decrypts room messages when a key is loaded', () => {
renderView({
packets: [

View File

@@ -2,8 +2,12 @@ import { describe, expect, it } from 'vitest';
import {
buildRawPacketStatsSnapshot,
summarizeRawPacketForStats,
type RawPacketStatsSessionState,
} from '../utils/rawPacketStats';
import type { RawPacket } from '../types';
const TEXT_MESSAGE_PACKET = '09046F17C47ED00A13E16AB5B94B1CC2D1A5059C6E5A6253C60D';
function createSession(
overrides: Partial<RawPacketStatsSessionState> = {}
@@ -75,6 +79,49 @@ function createSession(
}
describe('buildRawPacketStatsSnapshot', () => {
it('prefers decrypted contact identity over one-byte sourceHash for stats bucketing', () => {
const packet: RawPacket = {
id: 1,
observation_id: 10,
timestamp: 1_700_000_000,
data: TEXT_MESSAGE_PACKET,
payload_type: 'TextMessage',
snr: 4,
rssi: -72,
decrypted: true,
decrypted_info: {
channel_name: null,
sender: 'Alpha',
channel_key: null,
contact_key: '0a'.repeat(32),
},
};
const summary = summarizeRawPacketForStats(packet);
expect(summary.sourceKey).toBe('0A'.repeat(32));
expect(summary.sourceLabel).toBe('Alpha');
});
it('tags unresolved one-byte source hashes so they do not collide with full contact keys', () => {
const packet: RawPacket = {
id: 2,
observation_id: 11,
timestamp: 1_700_000_000,
data: TEXT_MESSAGE_PACKET,
payload_type: 'TextMessage',
snr: 4,
rssi: -72,
decrypted: false,
decrypted_info: null,
};
const summary = summarizeRawPacketForStats(packet);
expect(summary.sourceKey).toBe('hash1:0A');
expect(summary.sourceLabel).toBe('0A');
});
it('computes counts, rankings, and rolling-window coverage from session observations', () => {
const stats = buildRawPacketStatsSnapshot(createSession(), '5m', 1_000);

View File

@@ -163,10 +163,17 @@ function getSourceInfo(
case PayloadType.TextMessage:
case PayloadType.Request:
case PayloadType.Response: {
const contactKey = packet.decrypted_info?.contact_key?.toUpperCase() ?? null;
if (contactKey) {
return {
sourceKey: contactKey,
sourceLabel: packet.decrypted_info?.sender || toSourceLabel(contactKey),
};
}
const sourceHash = (decoded.payload.decoded as { sourceHash?: string }).sourceHash;
if (!sourceHash) return { sourceKey: null, sourceLabel: null };
return {
sourceKey: sourceHash.toUpperCase(),
sourceKey: `hash1:${sourceHash.toUpperCase()}`,
sourceLabel: sourceHash.toUpperCase(),
};
}

View File

@@ -1,6 +1,6 @@
[project]
name = "remoteterm-meshcore"
version = "3.4.1"
version = "3.5.0"
description = "RemoteTerm - Web interface for MeshCore radio mesh networks"
readme = "README.md"
requires-python = ">=3.10"

View File

@@ -6,9 +6,9 @@ function escapeRegex(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
/** Find a named non-repeater contact (type 2 = repeater). */
/** Find a named normal chat contact (exclude repeaters and room servers). */
function findChatContact(contacts: Contact[]): Contact | undefined {
return contacts.find((c) => c.name && c.name.trim().length > 0 && c.type !== 2);
return contacts.find((c) => c.name && c.name.trim().length > 0 && c.type !== 2 && c.type !== 3);
}
test.describe('Contacts sidebar & info pane', () => {

View File

@@ -160,6 +160,35 @@ class TestDebugEndpoint:
assert payload["radio_probe"]["performed"] is False
assert payload["radio_probe"]["errors"] == ["Radio not connected"]
assert payload["runtime"]["channels_with_incoming_messages"] == 0
assert payload["database"]["total_dms"] == 0
assert payload["database"]["total_channel_messages"] == 0
assert payload["database"]["total_outgoing"] == 0
@pytest.mark.asyncio
async def test_support_snapshot_includes_database_message_totals(self, test_db, client):
"""Debug snapshot includes stored DM/channel/outgoing totals."""
await _insert_contact("ab" * 32, "Alice")
await test_db.conn.execute(
"INSERT INTO channels (key, name, is_hashtag, on_radio) VALUES (?, ?, ?, ?)",
("CD" * 16, "#ops", 1, 0),
)
await test_db.conn.execute(
"INSERT INTO messages (type, conversation_key, text, received_at, outgoing) VALUES (?, ?, ?, ?, ?)",
("PRIV", "ab" * 32, "hello", 1000, 0),
)
await test_db.conn.execute(
"INSERT INTO messages (type, conversation_key, text, received_at, outgoing) VALUES (?, ?, ?, ?, ?)",
("CHAN", "CD" * 16, "room msg", 1001, 1),
)
await test_db.conn.commit()
response = await client.get("/api/debug")
assert response.status_code == 200
payload = response.json()
assert payload["database"]["total_dms"] == 1
assert payload["database"]["total_channel_messages"] == 1
assert payload["database"]["total_outgoing"] == 1
class TestRadioDisconnectedHandler:

View File

@@ -18,6 +18,8 @@ from app.event_handlers import (
track_pending_ack,
)
from app.repository import (
ContactAdvertPathRepository,
ContactNameHistoryRepository,
ContactRepository,
MessageRepository,
)
@@ -618,6 +620,8 @@ class TestContactMessageCLIFiltering:
sender_timestamp=1700000000,
received_at=1700000000,
)
await ContactNameHistoryRepository.record_name(prefix, "Prefix Sender", 1699999990)
await ContactAdvertPathRepository.record_observation(prefix, "1122", 1699999995)
with patch("app.event_handlers.broadcast_event") as mock_broadcast:
@@ -646,6 +650,19 @@ class TestContactMessageCLIFiltering:
assert len(messages) == 1
assert messages[0].conversation_key == full_key
assert await ContactNameHistoryRepository.get_history(prefix) == []
assert await ContactAdvertPathRepository.get_recent_for_contact(prefix) == []
resolved_history = await ContactNameHistoryRepository.get_history(full_key)
assert {entry.name for entry in resolved_history} == {
"Prefix Sender",
"Resolved Sender",
}
resolved_paths = await ContactAdvertPathRepository.get_recent_for_contact(full_key)
assert len(resolved_paths) == 1
assert resolved_paths[0].path == "1122"
event_types = [call.args[0] for call in mock_broadcast.call_args_list]
assert "contact" in event_types
assert "contact_resolved" in event_types

View File

@@ -1247,8 +1247,8 @@ class TestMigration039:
applied = await run_migrations(conn)
assert applied == 7
assert await get_version(conn) == 45
assert applied == 8
assert await get_version(conn) == 46
cursor = await conn.execute(
"""
@@ -1319,8 +1319,8 @@ class TestMigration039:
applied = await run_migrations(conn)
assert applied == 7
assert await get_version(conn) == 45
assert applied == 8
assert await get_version(conn) == 46
cursor = await conn.execute(
"""
@@ -1386,8 +1386,8 @@ class TestMigration039:
applied = await run_migrations(conn)
assert applied == 1
assert await get_version(conn) == 45
assert applied == 2
assert await get_version(conn) == 46
cursor = await conn.execute(
"""
@@ -1439,8 +1439,8 @@ class TestMigration040:
applied = await run_migrations(conn)
assert applied == 6
assert await get_version(conn) == 45
assert applied == 7
assert await get_version(conn) == 46
await conn.execute(
"""
@@ -1501,8 +1501,8 @@ class TestMigration041:
applied = await run_migrations(conn)
assert applied == 5
assert await get_version(conn) == 45
assert applied == 6
assert await get_version(conn) == 46
await conn.execute(
"""
@@ -1554,8 +1554,8 @@ class TestMigration042:
applied = await run_migrations(conn)
assert applied == 4
assert await get_version(conn) == 45
assert applied == 5
assert await get_version(conn) == 46
await conn.execute(
"""
@@ -1577,6 +1577,179 @@ class TestMigration042:
await conn.close()
class TestMigration046:
"""Test migration 046: clean orphaned contact child rows."""
@pytest.mark.asyncio
async def test_merges_uniquely_resolvable_orphans_and_drops_unresolved_ones(self):
conn = await aiosqlite.connect(":memory:")
conn.row_factory = aiosqlite.Row
try:
await set_version(conn, 45)
await conn.execute("""
CREATE TABLE contacts (
public_key TEXT PRIMARY KEY,
name TEXT
)
""")
await conn.execute("""
CREATE TABLE contact_name_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
public_key TEXT NOT NULL,
name TEXT NOT NULL,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
UNIQUE(public_key, name)
)
""")
await conn.execute("""
CREATE TABLE contact_advert_paths (
id INTEGER PRIMARY KEY AUTOINCREMENT,
public_key TEXT NOT NULL,
path_hex TEXT NOT NULL,
path_len INTEGER NOT NULL,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
heard_count INTEGER NOT NULL DEFAULT 1,
UNIQUE(public_key, path_hex, path_len)
)
""")
resolved_prefix = "abc123"
resolved_key = resolved_prefix + ("00" * 29)
ambiguous_prefix = "deadbe"
ambiguous_key_a = ambiguous_prefix + ("11" * 29)
ambiguous_key_b = ambiguous_prefix + ("22" * 29)
dead_prefix = "ffffaa"
await conn.execute(
"INSERT INTO contacts (public_key, name) VALUES (?, ?), (?, ?), (?, ?)",
(
resolved_key,
"Resolved Sender",
ambiguous_key_a,
"Ambiguous A",
ambiguous_key_b,
"Ambiguous B",
),
)
await conn.execute(
"""
INSERT INTO contact_name_history (public_key, name, first_seen, last_seen)
VALUES (?, ?, ?, ?), (?, ?, ?, ?), (?, ?, ?, ?)
""",
(
resolved_key,
"Resolved Sender",
900,
905,
resolved_prefix,
"Prefix Sender",
1000,
1010,
ambiguous_prefix,
"Ambiguous Prefix",
1100,
1110,
),
)
await conn.execute(
"""
INSERT INTO contact_advert_paths
(public_key, path_hex, path_len, first_seen, last_seen, heard_count)
VALUES
(?, ?, ?, ?, ?, ?),
(?, ?, ?, ?, ?, ?),
(?, ?, ?, ?, ?, ?),
(?, ?, ?, ?, ?, ?)
""",
(
resolved_key,
"1122",
1,
950,
960,
2,
resolved_prefix,
"1122",
1,
1001,
1002,
3,
ambiguous_prefix,
"3344",
2,
1200,
1201,
1,
dead_prefix,
"5566",
1,
1300,
1301,
1,
),
)
await conn.commit()
applied = await run_migrations(conn)
assert applied == 1
assert await get_version(conn) == 46
cursor = await conn.execute(
"""
SELECT name, first_seen, last_seen
FROM contact_name_history
WHERE public_key = ?
ORDER BY name
""",
(resolved_key,),
)
rows = await cursor.fetchall()
assert [(row["name"], row["first_seen"], row["last_seen"]) for row in rows] == [
("Prefix Sender", 1000, 1010),
("Resolved Sender", 900, 905),
]
cursor = await conn.execute(
"""
SELECT path_hex, path_len, first_seen, last_seen, heard_count
FROM contact_advert_paths
WHERE public_key = ?
ORDER BY path_hex, path_len
""",
(resolved_key,),
)
rows = await cursor.fetchall()
assert [
(
row["path_hex"],
row["path_len"],
row["first_seen"],
row["last_seen"],
row["heard_count"],
)
for row in rows
] == [
("1122", 1, 950, 1002, 5),
]
for orphan_key in (resolved_prefix, ambiguous_prefix, dead_prefix):
cursor = await conn.execute(
"SELECT COUNT(*) FROM contact_name_history WHERE public_key = ?",
(orphan_key,),
)
assert (await cursor.fetchone())[0] == 0
cursor = await conn.execute(
"SELECT COUNT(*) FROM contact_advert_paths WHERE public_key = ?",
(orphan_key,),
)
assert (await cursor.fetchone())[0] == 0
finally:
await conn.close()
class TestMigrationPacketHelpers:
"""Test migration-local packet helpers against canonical path validation."""

View File

@@ -1521,10 +1521,10 @@ class TestConcurrentChannelSends:
class TestChannelSendLockScope:
"""Channel send should release the radio lock before DB persistence work."""
"""Channel send should persist the outgoing row while the radio lock is held."""
@pytest.mark.asyncio
async def test_channel_message_row_created_after_radio_lock_released(self, test_db):
async def test_channel_message_row_created_inside_radio_lock(self, test_db):
mc = _make_mc(name="TestNode")
chan_key = "de" * 16
await ChannelRepository.upsert(key=chan_key, name="#lockscope")
@@ -1549,4 +1549,66 @@ class TestChannelSendLockScope:
SendChannelMessageRequest(channel_key=chan_key, text="Lock scope test")
)
assert observed_lock_states == [False]
assert observed_lock_states == [True]
@pytest.mark.asyncio
async def test_channel_self_observation_during_send_reconciles_to_reserved_outgoing_row(
self, test_db
):
"""A self-observation that arrives during send should update the reserved outgoing row."""
from app.services.messages import create_fallback_channel_message
mc = _make_mc(name="TestNode")
chan_key = "ef" * 16
await ChannelRepository.upsert(key=chan_key, name="#race")
broadcasts = []
def capture_broadcast(event_type, data, *args, **kwargs):
broadcasts.append({"type": event_type, "data": data})
async def send_with_self_observation(*args, **kwargs):
timestamp_bytes = kwargs["timestamp"]
sender_timestamp = int.from_bytes(timestamp_bytes, "little")
await create_fallback_channel_message(
conversation_key=chan_key.upper(),
message_text="Hello race",
sender_timestamp=sender_timestamp,
received_at=int(time.time()),
path="a1b2",
path_len=2,
txt_type=0,
sender_name="TestNode",
channel_name="#race",
broadcast_fn=capture_broadcast,
)
return _make_radio_result()
mc.commands.send_chan_msg = AsyncMock(side_effect=send_with_self_observation)
with (
patch("app.routers.messages.require_connected", return_value=mc),
patch.object(radio_manager, "_meshcore", mc),
patch("app.routers.messages.broadcast_event", side_effect=capture_broadcast),
):
message = await send_channel_message(
SendChannelMessageRequest(channel_key=chan_key, text="Hello race")
)
assert message.outgoing is True
assert message.acked == 1
assert message.paths is not None
assert len(message.paths) == 1
assert message.paths[0].path == "a1b2"
stored = await MessageRepository.get_all(
msg_type="CHAN", conversation_key=chan_key.upper(), limit=10
)
assert len(stored) == 1
assert stored[0].outgoing is True
assert stored[0].acked == 1
message_events = [entry for entry in broadcasts if entry["type"] == "message"]
ack_events = [entry for entry in broadcasts if entry["type"] == "message_acked"]
assert len(message_events) == 1
assert len(ack_events) == 1

2
uv.lock generated
View File

@@ -1098,7 +1098,7 @@ wheels = [
[[package]]
name = "remoteterm-meshcore"
version = "3.4.1"
version = "3.5.0"
source = { virtual = "." }
dependencies = [
{ name = "aiomqtt" },