23 Commits

Author SHA1 Message Date
Pablo Revilla 707b2ebbbd Merge branch 'master' into 3.0.6 2026-04-30 11:32:37 -07:00
pablorevilla-meshtastic d259230e05 fixed issue reported on issue #143 2026-04-17 15:26:53 -07:00
pablorevilla-meshtastic 7367155fec add note bottom of page 2026-04-03 13:47:29 -07:00
pablorevilla-meshtastic efca531855 remove duplicates from nodes in net.html 2026-04-03 13:44:41 -07:00
pablorevilla-meshtastic d16976e5d1 filter at chat.html 2026-04-03 11:55:50 -07:00
pablorevilla-meshtastic fca376486a Add filter to firehose 2026-04-03 11:49:42 -07:00
pablorevilla-meshtastic 96efe27abc add channel to firehouse 2026-04-03 11:40:56 -07:00
pablorevilla-meshtastic bd9d8ae265 remove file 2026-04-03 10:36:51 -07:00
pablorevilla-meshtastic 698f4ad77f Update protobuf 2026-04-03 10:32:05 -07:00
pablorevilla-meshtastic 1bb84f4c4d update latest list of instances 2026-04-01 12:40:42 -07:00
pablorevilla-meshtastic 17b91bbc7c more changes to report 2026-04-01 11:37:01 -07:00
pablorevilla-meshtastic 2692333fe2 changes to reports 2026-04-01 11:34:04 -07:00
pablorevilla-meshtastic ed18e2003c Fix MQTT store bug. 2026-04-01 10:46:57 -07:00
pablorevilla-meshtastic 65386be327 1 2026-03-05 11:39:36 -08:00
pablorevilla-meshtastic 973dcbfe51 fixed the db and added more reporting 2026-03-05 11:36:26 -08:00
pablorevilla-meshtastic 2490284435 fix ingestion error 2026-03-05 10:05:14 -08:00
pablorevilla-meshtastic 90767f94b4 update schemma 2026-03-05 10:03:17 -08:00
pablorevilla-meshtastic e0023bc8eb Added DB configuration and snapshot for basic detaisl 2026-03-05 09:57:34 -08:00
pablorevilla-meshtastic 432e9f2b67 Adapt PR 102 relay node display to packet page 2026-02-24 13:39:14 -08:00
Paul Picazo a0fbed0a88 Add migration script to introduce relay_node column in packet_seen table 2026-02-24 13:38:48 -08:00
Paul Picazo 8ca388acb8 Add relay_node field to PacketSeen model and update template for display 2026-02-24 13:38:48 -08:00
pablorevilla-meshtastic f87831c3a5 small map change 2026-02-18 16:17:18 -08:00
pablorevilla-meshtastic 9c40ce2d59 add world meshview tag 2026-02-18 15:32:16 -08:00
20 changed files with 808 additions and 140 deletions
+32 -32
View File
@@ -100,27 +100,38 @@ See [README-Docker.md](README-Docker.md) for container deployment and [docs/](do
Samples of currently running instances: Samples of currently running instances:
- https://meshview.bayme.sh (SF Bay Area - USA) | Area | URL | Version | Nodes | Last Updated |
- https://www.svme.sh (Sacramento Valley - USA) | --- | --- | --- | ---: | --- |
- https://meshview.nyme.sh (New York - USA) | Polski Mesh | https://meshview.format.ovh | 3.0.6 (65386be) | 4,827 | 4/1/2026, 12:12:25 PM |
- https://meshview.socalmesh.org (Los Angenles - USA) | Poland | https://meshview.szastan.pl | 3.0.5 (unknown) | 4,194 | 4/1/2026, 12:12:39 PM |
- https://map.wpamesh.net (Western Pennsylvania - USA) | Spain | https://meshview.meshtastic.es | 3.0.5 (unknown) | 2,466 | 4/1/2026, 12:11:17 PM |
- https://meshview.chicagolandmesh.org (Chicago - USA) | San Francisco Bay Area | https://meshview.bayme.sh | 3.0.5 (bdf70da) | 1,983 | 4/1/2026, 12:11:08 PM |
- https://meshview.freq51.net/ (Salt Lake City - USA) | Southern California | https://meshview.socalmesh.org | 3.0.5 (unknown) | 1,205 | 4/1/2026, 12:11:50 PM |
- https://meshview.mt.gt (Canada) | Oregon (USA) | https://meshview.meshoregon.com | 3.0.5 (bdf70da) | 1,167 | 4/1/2026, 12:11:41 PM |
- https://canadaverse.org (Canada) | Australia/ New Zeland | https://my.meshview.world/anz | plus 1.0.0 | 1,165 | 4/1/2026, 12:11:59 PM |
- https://meshview.meshtastic.es (Spain) | Hessen - Germany | https://meshview.lsinfra.de | 3.0.5 (unknown) | 1,082 | 4/1/2026, 12:11:30 PM |
- https://view.mtnme.sh (North Georgia / East Tennessee - USA) | New York City | https://meshview.nyme.sh | 3.0.5 (bdf70da) | 984 | 4/1/2026, 12:11:26 PM |
- https://meshview.lsinfra.de (Hessen - Germany) | Salt Lake City (USA) | https://meshview.freq51.net | — | 801 | 4/1/2026, 12:11:36 PM |
- https://meshview.pvmesh.org (Pioneer Valley, Massachusetts - USA) | Salzburg / Austria | https://meshview-salzburg.jmt.gr | — | 748 | 4/1/2026, 12:11:21 PM |
- https://meshview.louisianamesh.org (Louisiana - USA) | North Georgia / East Tennessee - USA | https://view.mtnme.sh | 3.0.3 (ff30623) | 730 | 4/1/2026, 12:11:33 PM |
- https://www.swlamesh.com (Southwest Louisiana- USA) | Canada | https://mv.canadaverse.org | 3.0.1 (unknown) | 674 | 4/1/2026, 12:12:06 PM |
- https://meshview.meshcolombia.co (Colombia) | Pioneer Valley, Massachusetts (USA) | https://meshview.pvmesh.org | 3.0.4 (018e16e) | 560 | 4/1/2026, 12:11:44 PM |
- https://meshview-salzburg.jmt.gr (Salzburg / Austria) | Western Pennsylvania (USA) | https://map.wpamesh.net | 3.0.5 (unknown) | 447 | 4/1/2026, 12:11:38 PM |
- https://map.cromesh.eu (Coatia) | Great Britan | https://my.meshview.world/gb | plus 1.0.0 | 421 | 4/1/2026, 12:12:03 PM |
- https://view.meshdresden.eu (Dresden / Germany) | Chicago (USA) | https://meshview.chicagolandmesh.org | 3.0.5 (unknown) | 420 | 4/1/2026, 12:12:41 PM |
- https://meshview.meshoregon.com (Oregon - USA) | Georgia (USA) | https://meshview.gamesh.net | 3.0.1 (unknown) | 384 | 4/1/2026, 12:11:45 PM |
- https://meshview.gamesh.net (Georgia - USA) | Brazil | https://my.meshview.world/brazil | plus 1.0.0 | 364 | 4/1/2026, 12:12:01 PM |
| Mexico | https://my.meshview.world/mexico | plus 1.0.0 | 268 | 4/1/2026, 12:11:59 PM |
| Northeast Ohio | https://meshview.neomesh.org | 3.0.5 (unknown) | 264 | 4/1/2026, 12:11:52 PM |
| South Africa | https://my.meshview.world/za | plus 1.0.0 | 263 | 4/1/2026, 12:12:07 PM |
| Argentina | https://my.meshview.world/argentina | plus 1.0.0 | 258 | 4/1/2026, 12:12:00 PM |
| Northwest Indiana (USA) | https://meshview.nwimesh.net | 3.0.5 (2cc53dc) | 219 | 4/1/2026, 12:12:08 PM |
| Louisiana (USA) | https://meshview.louisianamesh.org | — | 78 | 4/1/2026, 12:11:51 PM |
| Chile | https://my.meshview.world/chile | plus 1.0.0 | 58 | 4/1/2026, 12:12:06 PM |
| Colombia | https://meshview.meshcolombia.co | 3.0.5 (2cc53dc) | 55 | 4/1/2026, 12:11:46 PM |
| Croatia | https://map.cromesh.eu | — | 0 | 4/1/2026, 12:11:45 PM |
| Saint Louis (USA) | https://meshview.meshstl.org | 3.0.5 (2cc53dc) | 0 | 4/1/2026, 12:11:51 PM |
| Southwest Louisiana (USA) | https://www.swlamesh.com | — | 0 | 4/1/2026, 12:11:46 PM |
--- ---
@@ -652,17 +663,6 @@ Check the log file to see it the script run at the specific time.
--- ---
## Testing
MeshView includes a test suite using pytest. For detailed testing documentation, see [README-testing.md](README-testing.md).
Quick start:
```bash
./env/bin/pytest tests/test_api_simple.py -v
```
---
## Technical Documentation ## Technical Documentation
For more detailed technical documentation including database migrations, architecture details, and advanced topics, see the [docs/](docs/) directory. For more detailed technical documentation including database migrations, architecture details, and advanced topics, see the [docs/](docs/) directory.
@@ -0,0 +1,35 @@
"""Add daily_snapshot table
Revision ID: 4f1d2a9c8b71
Revises: 23dad03d2e42
Create Date: 2026-03-05 00:00:00.000000
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "4f1d2a9c8b71"
down_revision: str | None = "23dad03d2e42"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.create_table(
"daily_snapshot",
sa.Column("snapshot_date", sa.Date(), nullable=False),
sa.Column("node_count", sa.BigInteger(), nullable=False),
sa.Column("packet_count", sa.BigInteger(), nullable=False),
sa.Column("gateway_count", sa.BigInteger(), nullable=False),
sa.Column("captured_at_us", sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint("snapshot_date"),
)
def downgrade() -> None:
op.drop_table("daily_snapshot")
@@ -0,0 +1,22 @@
"""Compatibility placeholder for removed revision c6b1d8f2a9e3.
Revision ID: c6b1d8f2a9e3
Revises: 4f1d2a9c8b71
Create Date: 2026-03-05 00:00:00.000000
"""
from collections.abc import Sequence
# revision identifiers, used by Alembic.
revision: str = "c6b1d8f2a9e3"
down_revision: str | None = "4f1d2a9c8b71"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
+1 -1
View File
@@ -1 +1 @@
e1a6b3a868d735da72cd6c94c574d655129d390a cb1f89372a70b0d4b4f8caf05aec28de8d4a13e0
+2 -2
View File
@@ -3,8 +3,8 @@
import subprocess import subprocess
from pathlib import Path from pathlib import Path
__version__ = "3.0.5" __version__ = "3.0.6"
__release_date__ = "2026-2-6" __release_date__ = "2026-3-6"
def get_git_revision(): def get_git_revision():
+7 -1
View File
@@ -51,7 +51,9 @@
"chat": { "chat": {
"chat_title": "Chats:", "chat_title": "Chats:",
"replying_to": "Replying to:", "replying_to": "Replying to:",
"view_packet_details": "View packet details" "view_packet_details": "View packet details",
"channel": "Channel",
"all_channels": "All Channels"
}, },
"nodelist": { "nodelist": {
"search_placeholder": "Search by name or ID...", "search_placeholder": "Search by name or ID...",
@@ -127,6 +129,8 @@
"total_gateways": "Total Gateways", "total_gateways": "Total Gateways",
"total_packets": "Total Packets", "total_packets": "Total Packets",
"total_packets_seen": "Total Packets Seen", "total_packets_seen": "Total Packets Seen",
"daily_snapshot_packets_history": "Daily Packet Snapshot History",
"daily_snapshot_nodes_gateways_history": "Daily Node/Gateway Snapshot History",
"packets_per_day_all": "Packets per Day - All Ports (Last 14 Days)", "packets_per_day_all": "Packets per Day - All Ports (Last 14 Days)",
"packets_per_day_text": "Packets per Day - Text Messages (Port 1, Last 14 Days)", "packets_per_day_text": "Packets per Day - Text Messages (Port 1, Last 14 Days)",
"packets_per_hour_all": "Packets per Hour - All Ports", "packets_per_hour_all": "Packets per Hour - All Ports",
@@ -178,6 +182,8 @@
"packet_id": "Packet ID", "packet_id": "Packet ID",
"from": "From", "from": "From",
"to": "To", "to": "To",
"channel": "Channel",
"all_channels": "All Channels",
"port": "Port", "port": "Port",
"links": "Links", "links": "Links",
"unknown_app": "UNKNOWN APP", "unknown_app": "UNKNOWN APP",
+7 -1
View File
@@ -52,7 +52,9 @@
"chat": { "chat": {
"chat_title": "Conversaciones:", "chat_title": "Conversaciones:",
"replying_to": "Respondiendo a:", "replying_to": "Respondiendo a:",
"view_packet_details": "Ver detalles del paquete" "view_packet_details": "Ver detalles del paquete",
"channel": "Canal",
"all_channels": "Todos los canales"
}, },
"nodelist": { "nodelist": {
@@ -123,6 +125,8 @@
"total_gateways": "Gateways Totales", "total_gateways": "Gateways Totales",
"total_packets": "Paquetes Totales", "total_packets": "Paquetes Totales",
"total_packets_seen": "Paquetes Totales Vistos", "total_packets_seen": "Paquetes Totales Vistos",
"daily_snapshot_packets_history": "Historial Diario de Instantáneas de Paquetes (Todos los Días Disponibles)",
"daily_snapshot_nodes_gateways_history": "Historial Diario de Instantáneas de Nodos/Gateways (Todos los Días Disponibles)",
"packets_per_day_all": "Paquetes por Día - Todos los Puertos (Últimos 14 Días)", "packets_per_day_all": "Paquetes por Día - Todos los Puertos (Últimos 14 Días)",
"packets_per_day_text": "Paquetes por Día - Mensajes de Texto (Puerto 1, Últimos 14 Días)", "packets_per_day_text": "Paquetes por Día - Mensajes de Texto (Puerto 1, Últimos 14 Días)",
"packets_per_hour_all": "Paquetes por Hora - Todos los Puertos", "packets_per_hour_all": "Paquetes por Hora - Todos los Puertos",
@@ -177,6 +181,8 @@
"packet_id": "ID de paquete", "packet_id": "ID de paquete",
"from": "De", "from": "De",
"to": "A", "to": "A",
"channel": "Canal",
"all_channels": "Todos los canales",
"port": "Puerto", "port": "Puerto",
"direct_to_mqtt": "Directo a MQTT", "direct_to_mqtt": "Directo a MQTT",
"all_broadcast": "Todos" "all_broadcast": "Todos"
+13 -1
View File
@@ -1,4 +1,6 @@
from sqlalchemy import BigInteger, ForeignKey, Index, desc from datetime import date
from sqlalchemy import BigInteger, Date, ForeignKey, Index, desc
from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import AsyncAttrs
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship
@@ -119,3 +121,13 @@ class NodePublicKey(Base):
Index("idx_node_public_key_node_id", "node_id"), Index("idx_node_public_key_node_id", "node_id"),
Index("idx_node_public_key_public_key", "public_key"), Index("idx_node_public_key_public_key", "public_key"),
) )
class DailySnapshot(Base):
__tablename__ = "daily_snapshot"
snapshot_date: Mapped[date] = mapped_column(Date, primary_key=True)
node_count: Mapped[int] = mapped_column(BigInteger, nullable=False)
packet_count: Mapped[int] = mapped_column(BigInteger, nullable=False)
gateway_count: Mapped[int] = mapped_column(BigInteger, nullable=False)
captured_at_us: Mapped[int] = mapped_column(BigInteger, nullable=False)
+101 -24
View File
@@ -1,8 +1,9 @@
import logging import logging
import re import re
import time import time
from datetime import UTC, datetime
from sqlalchemy import select, update from sqlalchemy import func, select, update
from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.dialects.sqlite import insert as sqlite_insert from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
@@ -11,13 +12,69 @@ from meshtastic.protobuf.config_pb2 import Config
from meshtastic.protobuf.mesh_pb2 import HardwareModel from meshtastic.protobuf.mesh_pb2 import HardwareModel
from meshtastic.protobuf.portnums_pb2 import PortNum from meshtastic.protobuf.portnums_pb2 import PortNum
from meshview import decode_payload, mqtt_database from meshview import decode_payload, mqtt_database
from meshview.models import Node, NodePublicKey, Packet, PacketSeen, Traceroute from meshview.models import DailySnapshot, Node, NodePublicKey, Packet, PacketSeen, Traceroute
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
MQTT_GATEWAY_CACHE: set[int] = set() MQTT_GATEWAY_CACHE: set[int] = set()
async def capture_daily_snapshot() -> None:
today = datetime.now(UTC).date()
async with mqtt_database.async_session() as session:
node_count = (await session.execute(select(func.count()).select_from(Node))).scalar_one()
packet_count = (
await session.execute(select(func.count()).select_from(Packet))
).scalar_one()
gateway_count = (
await session.execute(
select(func.count()).select_from(Node).where(Node.is_mqtt_gateway.is_(True))
)
).scalar_one()
captured_at_us = int(time.time() * 1_000_000)
values = {
"snapshot_date": today,
"node_count": node_count,
"packet_count": packet_count,
"gateway_count": gateway_count,
"captured_at_us": captured_at_us,
}
dialect = session.get_bind().dialect.name
stmt = None
if dialect == "sqlite":
stmt = (
sqlite_insert(DailySnapshot)
.values(**values)
.on_conflict_do_update(index_elements=["snapshot_date"], set_=values)
)
elif dialect == "postgresql":
stmt = (
pg_insert(DailySnapshot)
.values(**values)
.on_conflict_do_update(index_elements=["snapshot_date"], set_=values)
)
if stmt is not None:
await session.execute(stmt)
else:
snapshot = (
await session.execute(
select(DailySnapshot).where(DailySnapshot.snapshot_date == today)
)
).scalar_one_or_none()
if snapshot is None:
session.add(DailySnapshot(**values))
else:
snapshot.node_count = node_count
snapshot.packet_count = packet_count
snapshot.gateway_count = gateway_count
snapshot.captured_at_us = captured_at_us
await session.commit()
async def process_envelope(topic, env): async def process_envelope(topic, env):
# MAP_REPORT_APP # MAP_REPORT_APP
if env.packet.decoded.portnum == PortNum.MAP_REPORT_APP: if env.packet.decoded.portnum == PortNum.MAP_REPORT_APP:
@@ -124,7 +181,7 @@ async def process_envelope(topic, env):
except IntegrityError: except IntegrityError:
pass pass
# --- PacketSeen (no conflict handling here, normal insert) # --- PacketSeen insert with conflict-safe handling
if not env.gateway_id: if not env.gateway_id:
print("WARNING: Missing gateway_id, skipping PacketSeen entry") print("WARNING: Missing gateway_id, skipping PacketSeen entry")
@@ -139,28 +196,43 @@ async def process_envelope(topic, env):
update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True) update(Node).where(Node.node_id == node_id).values(is_mqtt_gateway=True)
) )
result = await session.execute(
select(PacketSeen).where(
PacketSeen.packet_id == env.packet.id,
PacketSeen.node_id == node_id,
PacketSeen.rx_time == env.packet.rx_time,
)
)
if not result.scalar_one_or_none():
now_us = int(time.time() * 1_000_000) now_us = int(time.time() * 1_000_000)
seen = PacketSeen( seen_values = {
packet_id=env.packet.id, "packet_id": env.packet.id,
node_id=int(env.gateway_id[1:], 16), "node_id": int(env.gateway_id[1:], 16),
channel=env.channel_id, "channel": env.channel_id,
rx_time=env.packet.rx_time, "rx_time": env.packet.rx_time,
rx_snr=env.packet.rx_snr, "rx_snr": env.packet.rx_snr,
rx_rssi=env.packet.rx_rssi, "rx_rssi": env.packet.rx_rssi,
hop_limit=env.packet.hop_limit, "hop_limit": env.packet.hop_limit,
hop_start=env.packet.hop_start, "hop_start": env.packet.hop_start,
topic=topic, "topic": topic,
import_time_us=now_us, "import_time_us": now_us,
}
dialect = session.get_bind().dialect.name
seen_stmt = None
if dialect == "sqlite":
seen_stmt = (
sqlite_insert(PacketSeen)
.values(**seen_values)
.on_conflict_do_nothing(index_elements=["packet_id", "node_id", "rx_time"])
) )
session.add(seen) elif dialect == "postgresql":
seen_stmt = (
pg_insert(PacketSeen)
.values(**seen_values)
.on_conflict_do_nothing(index_elements=["packet_id", "node_id", "rx_time"])
)
if seen_stmt is not None:
await session.execute(seen_stmt)
else:
try:
async with session.begin_nested():
session.add(PacketSeen(**seen_values))
await session.flush()
except IntegrityError:
pass
# --- NODEINFO_APP handling # --- NODEINFO_APP handling
if env.packet.decoded.portnum == PortNum.NODEINFO_APP: if env.packet.decoded.portnum == PortNum.NODEINFO_APP:
@@ -172,7 +244,12 @@ async def process_envelope(topic, env):
if user.id[0] == "!" and re.fullmatch(r"[0-9a-fA-F]+", user.id[1:]): if user.id[0] == "!" and re.fullmatch(r"[0-9a-fA-F]+", user.id[1:]):
node_id = int(user.id[1:], 16) node_id = int(user.id[1:], 16)
else: else:
node_id = None logger.warning(
"Skipping NODEINFO_APP packet %s: unable to determine node_id",
env.packet.id,
)
await session.commit()
return
hw_model = ( hw_model = (
HardwareModel.Name(user.hw_model) HardwareModel.Name(user.hw_model)
+1
View File
@@ -176,6 +176,7 @@ async function initializePage() {
items.push(`<a href="${urls[i]}">${dict[key] || key}</a>`); items.push(`<a href="${urls[i]}">${dict[key] || key}</a>`);
} }
} }
items.push('<a href="https://meshview.world" target="_blank" rel="noopener noreferrer">MeshviewWorld</a>');
menu.innerHTML = items.join("&nbsp;-&nbsp;"); menu.innerHTML = items.join("&nbsp;-&nbsp;");
} }
+59 -1
View File
@@ -46,6 +46,13 @@
padding-left: 10px; padding-left: 10px;
} }
.replying-to .reply-preview { color: #aaa; } .replying-to .reply-preview { color: #aaa; }
.chat-controls {
gap: 12px;
margin-bottom: 10px;
}
.chat-filter {
min-width: 220px;
}
{% endblock %} {% endblock %}
{% block body %} {% block body %}
@@ -53,10 +60,19 @@
<!-- ⭐ CHAT TITLE WITH ICON, aligned to container ⭐ --> <!-- ⭐ CHAT TITLE WITH ICON, aligned to container ⭐ -->
<div class="container px-2"> <div class="container px-2">
<h2 style="color:white; margin:0 0 10px 0;"> <div class="d-flex align-items-center justify-content-between chat-controls">
<h2 style="color:white; margin:0;">
<span class="icon">💬</span> <span class="icon">💬</span>
<span data-translate="chat_title"></span> <span data-translate="chat_title"></span>
</h2> </h2>
<div class="d-flex align-items-center gap-2">
<label for="channel-filter" class="mb-0" data-translate="channel">Channel</label>
<select id="channel-filter" class="form-select form-select-sm chat-filter">
<option value="" data-translate="all_channels">All Channels</option>
</select>
</div>
</div>
</div> </div>
<div class="container" id="chat-log"></div> <div class="container" id="chat-log"></div>
@@ -68,6 +84,7 @@ document.addEventListener("DOMContentLoaded", async () => {
if (!chatContainer) return console.error("#chat-log not found"); if (!chatContainer) return console.error("#chat-log not found");
let lastTime = null; let lastTime = null;
let selectedChannel = "";
const renderedPacketIds = new Set(); const renderedPacketIds = new Set();
const packetMap = new Map(); const packetMap = new Map();
let chatLang = {}; let chatLang = {};
@@ -99,6 +116,26 @@ document.addEventListener("DOMContentLoaded", async () => {
} }
} }
async function loadChannels() {
try {
const res = await fetch("/api/channels");
if (!res.ok) return;
const data = await res.json();
const select = document.getElementById("channel-filter");
const channels = data.channels || [];
for (const channel of channels) {
const option = document.createElement("option");
option.value = channel;
option.textContent = channel;
select.appendChild(option);
}
} catch (err) {
console.error("Failed loading channels:", err);
}
}
/* ========================================================== /* ==========================================================
SAFE HTML SAFE HTML
========================================================== */ ========================================================== */
@@ -191,6 +228,18 @@ document.addEventListener("DOMContentLoaded", async () => {
if (highlight) setTimeout(() => div.classList.remove("flash"), 2500); if (highlight) setTimeout(() => div.classList.remove("flash"), 2500);
} }
function matchesSelectedChannel(packet) {
if (!selectedChannel) return true;
return (packet.channel || "").toLowerCase() === selectedChannel.toLowerCase();
}
function resetChat() {
lastTime = null;
renderedPacketIds.clear();
packetMap.clear();
chatContainer.innerHTML = "";
}
function renderPacketsEnsureDescending(packets, highlight=false) { function renderPacketsEnsureDescending(packets, highlight=false) {
if (!Array.isArray(packets) || packets.length===0) return; if (!Array.isArray(packets) || packets.length===0) return;
const sortedDesc = packets.slice().sort((a,b)=>{ const sortedDesc = packets.slice().sort((a,b)=>{
@@ -199,6 +248,7 @@ document.addEventListener("DOMContentLoaded", async () => {
return bTime - aTime; return bTime - aTime;
}); });
for (let i=sortedDesc.length-1; i>=0; i--) { for (let i=sortedDesc.length-1; i>=0; i--) {
if (!matchesSelectedChannel(sortedDesc[i])) continue;
renderPacket(sortedDesc[i], highlight); renderPacket(sortedDesc[i], highlight);
} }
} }
@@ -235,6 +285,14 @@ document.addEventListener("DOMContentLoaded", async () => {
INIT INIT
========================================================== */ ========================================================== */
await loadChatLang(); // load translations FIRST await loadChatLang(); // load translations FIRST
await loadChannels();
document.getElementById("channel-filter").addEventListener("change", async (e) => {
selectedChannel = e.target.value;
resetChat();
await fetchInitial();
});
await fetchInitial(); // then fetch initial packets await fetchInitial(); // then fetch initial packets
setInterval(fetchUpdates, 5000); setInterval(fetchUpdates, 5000);
+58 -2
View File
@@ -11,6 +11,12 @@
font-size: 0.9rem; font-size: 0.9rem;
border-radius: 6px; border-radius: 6px;
} }
.firehose-controls {
gap: 12px;
}
.firehose-filter {
min-width: 220px;
}
.port-tag { .port-tag {
display: inline-block; display: inline-block;
@@ -89,15 +95,22 @@
{% block body %} {% block body %}
<div class="container"> <div class="container">
<form class="d-flex align-items-center justify-content-between mb-3"> <form class="d-flex align-items-center justify-content-between mb-3 firehose-controls">
<h2 class="mb-0" data-translate-lang="live_feed">📡 Live Feed</h2> <h2 class="mb-0" data-translate-lang="live_feed">📡 Live Feed</h2>
<div class="d-flex align-items-center gap-2">
<label for="channel-filter" class="mb-0" data-translate-lang="channel">Channel</label>
<select id="channel-filter" class="form-select form-select-sm firehose-filter">
<option value="" data-translate-lang="all_channels">All Channels</option>
</select>
<button type="button" <button type="button"
id="pause-button" id="pause-button"
class="btn btn-sm btn-outline-secondary" class="btn btn-sm btn-outline-secondary"
data-translate-lang="pause"> data-translate-lang="pause">
Pause Pause
</button> </button>
</div>
</form> </form>
<table class="packet-table"> <table class="packet-table">
@@ -107,6 +120,7 @@
<th data-translate-lang="packet_id">Packet ID</th> <th data-translate-lang="packet_id">Packet ID</th>
<th data-translate-lang="from">From</th> <th data-translate-lang="from">From</th>
<th data-translate-lang="to">To</th> <th data-translate-lang="to">To</th>
<th data-translate-lang="channel">Channel</th>
<th data-translate-lang="port">Port</th> <th data-translate-lang="port">Port</th>
</tr> </tr>
</thead> </thead>
@@ -240,6 +254,7 @@ function logPacketTimes(packet) {
let lastImportTimeUs = null; let lastImportTimeUs = null;
let updatesPaused = false; let updatesPaused = false;
let updateInterval = 3000; let updateInterval = 3000;
let selectedChannel = "";
async function configureFirehose() { async function configureFirehose() {
try { try {
@@ -249,6 +264,36 @@ async function configureFirehose() {
} catch {} } catch {}
} }
async function loadChannels() {
try {
const res = await fetch("/api/channels");
if (!res.ok) return;
const data = await res.json();
const select = document.getElementById("channel-filter");
const channels = data.channels || [];
for (const channel of channels) {
const option = document.createElement("option");
option.value = channel;
option.textContent = channel;
select.appendChild(option);
}
} catch (err) {
console.error("Failed loading channels:", err);
}
}
function resetFirehose() {
lastImportTimeUs = null;
document.getElementById("packet_list").innerHTML = "";
}
function matchesSelectedChannel(packet) {
if (!selectedChannel) return true;
return (packet.channel || "").toLowerCase() === selectedChannel.toLowerCase();
}
async function fetchUpdates() { async function fetchUpdates() {
if (updatesPaused) return; if (updatesPaused) return;
@@ -269,6 +314,8 @@ async function fetchUpdates() {
const list = document.getElementById("packet_list"); const list = document.getElementById("packet_list");
for (const pkt of packets.reverse()) { for (const pkt of packets.reverse()) {
if (!matchesSelectedChannel(pkt)) continue;
logPacketTimes(pkt); logPacketTimes(pkt);
/* FROM — includes translation */ /* FROM — includes translation */
@@ -343,12 +390,13 @@ async function fetchUpdates() {
<td>${from}</td> <td>${from}</td>
<td>${to}</td> <td>${to}</td>
<td>${pkt.channel || "—"}</td>
<td>${portLabel(pkt.portnum, pkt.payload, inlineLinks)}</td> <td>${portLabel(pkt.portnum, pkt.payload, inlineLinks)}</td>
</tr> </tr>
<tr class="payload-row"> <tr class="payload-row">
<td colspan="5" class="payload-cell">${safePayload}</td> <td colspan="6" class="payload-cell">${safePayload}</td>
</tr> </tr>
`; `;
@@ -371,6 +419,7 @@ async function fetchUpdates() {
document.addEventListener("DOMContentLoaded", async () => { document.addEventListener("DOMContentLoaded", async () => {
const pauseBtn = document.getElementById("pause-button"); const pauseBtn = document.getElementById("pause-button");
const channelFilter = document.getElementById("channel-filter");
pauseBtn.addEventListener("click", () => { pauseBtn.addEventListener("click", () => {
updatesPaused = !updatesPaused; updatesPaused = !updatesPaused;
@@ -381,6 +430,12 @@ document.addEventListener("DOMContentLoaded", async () => {
: (firehoseTranslations.pause || "Pause"); : (firehoseTranslations.pause || "Pause");
}); });
channelFilter.addEventListener("change", async (e) => {
selectedChannel = e.target.value;
resetFirehose();
await fetchUpdates();
});
document.addEventListener("click", e => { document.addEventListener("click", e => {
const btn = e.target.closest(".toggle-btn"); const btn = e.target.closest(".toggle-btn");
if (!btn) return; if (!btn) return;
@@ -395,6 +450,7 @@ document.addEventListener("DOMContentLoaded", async () => {
await loadTranslationsFirehose(); await loadTranslationsFirehose();
await configureFirehose(); await configureFirehose();
await loadNodes(); await loadNodes();
await loadChannels();
fetchUpdates(); fetchUpdates();
setInterval(fetchUpdates, updateInterval); setInterval(fetchUpdates, updateInterval);
+71 -7
View File
@@ -23,7 +23,25 @@
#reset-filters-button:hover { background-color:#da190b; } #reset-filters-button:hover { background-color:#da190b; }
#reset-filters-button:active { background-color:#c41e0d; } #reset-filters-button:active { background-color:#c41e0d; }
.blinking-tooltip { background:white;color:black;border:1px solid black;border-radius:4px;padding:2px 5px; } .blinking-tooltip {
background: white;
color: black;
border: 1px solid #111;
border-radius: 6px;
padding: 6px 10px;
font-size: 14px;
font-weight: 400;
box-shadow: 0 2px 8px rgba(0,0,0,0.25);
}
.blinking-tooltip.text-packet {
animation: textPulse 1.1s ease-in-out 6;
border-color: #ff8c00;
}
@keyframes textPulse {
0% { box-shadow: 0 2px 8px rgba(0,0,0,0.25); }
50% { box-shadow: 0 4px 14px rgba(255,140,0,0.45); }
100% { box-shadow: 0 2px 8px rgba(0,0,0,0.25); }
}
#map-wrapper { #map-wrapper {
position: relative; position: relative;
@@ -312,7 +330,7 @@ function fetchNewPackets(){
const marker = markerById[pkt.from_node_id]; const marker = markerById[pkt.from_node_id];
const nodeData = nodeMap.get(pkt.from_node_id); const nodeData = nodeMap.get(pkt.from_node_id);
if(marker && nodeData) { if(marker && nodeData) {
blinkNode(marker,nodeData.long_name,pkt.portnum); blinkNode(marker, nodeData.long_name, pkt.portnum, pkt.payload);
} else { } else {
addUnmappedPacket(pkt, nodeData); addUnmappedPacket(pkt, nodeData);
} }
@@ -451,7 +469,29 @@ function renderNodesOnMap(){
marker.originalColor = color; marker.originalColor = color;
markerById[node.key] = marker; markerById[node.key] = marker;
marker.bindPopup(buildNodePopup(node)); const popup = `
<a href="/node/${node.node_id}">${node.long_name}</a> (${node.short_name})<br>
<span data-translate-lang="channel_label"></span> ${node.channel}<br>
<span data-translate-lang="model_label"></span> ${node.hw_model}<br>
<span data-translate-lang="role_label"></span> ${node.role}<br>
<span data-translate-lang="mqtt_gateway"></span> ${
node.is_mqtt_gateway ? (mapTranslations.yes || "Yes") : (mapTranslations.no || "No")
}<br>
${
node.last_seen_us
? `<span data-translate-lang="last_seen"></span> ${timeAgoFromUs(node.last_seen_us)}<br>`
: ""
}
${
node.firmware
? `<span data-translate-lang="firmware"></span> ${node.firmware}<br>`
: ""
}
`;
marker.on('click', () => { marker.on('click', () => {
onNodeClick(node); onNodeClick(node);
marker.setPopupContent(buildNodePopup(node)); marker.setPopupContent(buildNodePopup(node));
@@ -585,7 +625,20 @@ map.on('click', e=>{
BLINKING BLINKING
====================================================== */ ====================================================== */
function blinkNode(marker,longName,portnum){ function escapeHtml(value){
return String(value)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}
function isTextPort(portnum){
return portnum === 1 || portnum === 7;
}
function blinkNode(marker,longName,portnum,payload){
if(!map.hasLayer(marker)) return; if(!map.hasLayer(marker)) return;
if(activeBlinks.has(marker)){ if(activeBlinks.has(marker)){
@@ -595,13 +648,24 @@ function blinkNode(marker,longName,portnum){
} }
let blinkCount = 0; let blinkCount = 0;
const blinkStart = Date.now();
const blinkDurationMs = 7000;
const safeName = escapeHtml(longName);
const portLabel = portMap[portnum] || `Port ${portnum ?? "?"}`;
const payloadText = (payload || "").trim();
const showPayload = isTextPort(portnum) && payloadText && payloadText !== "Did not decode";
const shortPayload = showPayload && payloadText.length > 80
? `${payloadText.slice(0, 77)}...`
: payloadText;
const payloadLine = showPayload ? `<br><span>${escapeHtml(shortPayload)}</span>` : "";
const tooltip = L.tooltip({ const tooltip = L.tooltip({
permanent:true, permanent:true,
direction:'top', direction:'top',
offset:[0,-marker.options.radius-5], offset:[0,-marker.options.radius-5],
className:'blinking-tooltip' className: isTextPort(portnum) ? 'blinking-tooltip text-packet' : 'blinking-tooltip'
}) })
.setContent(`${longName} (${portMap[portnum] || "Port "+portnum})`) .setContent(`${safeName} (${escapeHtml(portLabel)})${payloadLine}`)
.setLatLng(marker.getLatLng()) .setLatLng(marker.getLatLng())
.addTo(map); .addTo(map);
@@ -616,7 +680,7 @@ function blinkNode(marker,longName,portnum){
} }
blinkCount++; blinkCount++;
if(blinkCount>7){ if(Date.now() - blinkStart > blinkDurationMs){
clearInterval(interval); clearInterval(interval);
marker.setStyle({ fillColor: marker.originalColor }); marker.setStyle({ fillColor: marker.originalColor });
map.removeLayer(tooltip); map.removeLayer(tooltip);
+20 -2
View File
@@ -25,6 +25,7 @@
#weekly-message { margin: 15px 0; font-weight: bold; color: #ffeb3b; } #weekly-message { margin: 15px 0; font-weight: bold; color: #ffeb3b; }
#total-count { margin-bottom: 10px; font-style: italic; color: #ccc; } #total-count { margin-bottom: 10px; font-style: italic; color: #ccc; }
#net-note { margin: 16px 0 0; font-size: 0.9rem; color: #aaa; }
{% endblock %} {% endblock %}
{% block body %} {% block body %}
@@ -50,6 +51,10 @@
<div id="chat-container"> <div id="chat-container">
<div class="container" id="chat-log"></div> <div class="container" id="chat-log"></div>
</div> </div>
<div id="net-note">
Only the earliest report from each device is listed.
</div>
</div> </div>
<script> <script>
@@ -65,6 +70,7 @@ document.addEventListener("DOMContentLoaded", async () => {
} }
const renderedPacketIds = new Set(); const renderedPacketIds = new Set();
const renderedNodeIds = new Set();
let netTranslations = {}; let netTranslations = {};
let netTag = ""; let netTag = "";
@@ -95,7 +101,7 @@ document.addEventListener("DOMContentLoaded", async () => {
Update count Update count
----------------------------------- */ ----------------------------------- */
function updateTotalCount() { function updateTotalCount() {
totalCountValueEl.textContent = renderedPacketIds.size; totalCountValueEl.textContent = renderedNodeIds.size;
} }
/* ----------------------------------- /* -----------------------------------
@@ -104,6 +110,7 @@ document.addEventListener("DOMContentLoaded", async () => {
function renderPacket(packet) { function renderPacket(packet) {
if (renderedPacketIds.has(packet.id)) return; if (renderedPacketIds.has(packet.id)) return;
renderedPacketIds.add(packet.id); renderedPacketIds.add(packet.id);
renderedNodeIds.add(packet.from_node_id);
const date = new Date(packet.import_time_us / 1000); const date = new Date(packet.import_time_us / 1000);
@@ -161,7 +168,18 @@ document.addEventListener("DOMContentLoaded", async () => {
----------------------------------- */ ----------------------------------- */
function renderPacketsEnsureDescending(packets) { function renderPacketsEnsureDescending(packets) {
if (!packets || !packets.length) return; if (!packets || !packets.length) return;
const sorted = packets.slice().sort((a, b) => b.import_time_us - a.import_time_us); const earliestByNode = new Map();
for (const packet of packets) {
const existing = earliestByNode.get(packet.from_node_id);
if (!existing || packet.import_time_us < existing.import_time_us) {
earliestByNode.set(packet.from_node_id, packet);
}
}
const sorted = Array.from(earliestByNode.values()).sort(
(a, b) => b.import_time_us - a.import_time_us
);
for (let i = sorted.length - 1; i >= 0; i--) { for (let i = sorted.length - 1; i >= 0; i--) {
renderPacket(sorted[i]); renderPacket(sorted[i]);
} }
+30 -22
View File
@@ -18,21 +18,23 @@
/* --- Node Info --- */ /* --- Node Info --- */
.node-info { .node-info {
background-color: #1f2226;
border: 1px solid #3a3f44;
color: #ddd; color: #ddd;
font-size: 0.88rem; font-size: 0.88rem;
padding: 12px 14px;
margin-bottom: 14px; margin-bottom: 14px;
border-radius: 8px;
display: grid; display: grid;
grid-template-columns: repeat(3, minmax(120px, 1fr)); grid-template-columns: repeat(3, minmax(120px, 1fr));
grid-column-gap: 14px; grid-column-gap: 12px;
grid-row-gap: 6px; grid-row-gap: 12px;
} }
.node-info div { padding: 2px 0; } .node-info-col {
background-color: #1f2226;
border: 1px solid #3a3f44;
border-radius: 8px;
padding: 12px 14px;
}
.node-info-col div { padding: 2px 0; }
.node-info strong { .node-info strong {
color: #9fd4ff; color: #9fd4ff;
font-weight: 600; font-weight: 600;
@@ -357,22 +359,24 @@
<!-- Node Info --> <!-- Node Info -->
<div id="node-info" class="node-info"> <div id="node-info" class="node-info">
<div class="node-info-col">
<div><strong data-translate-lang="node_id">Node ID</strong><strong>: </strong><span id="info-node-id"></span></div> <div><strong data-translate-lang="node_id">Node ID</strong><strong>: </strong><span id="info-node-id"></span></div>
<div><strong data-translate-lang="id">Hex ID</strong><strong>: </strong><span id="info-id"></span></div> <div><strong data-translate-lang="id">Hex ID</strong><strong>: </strong><span id="info-id"></span></div>
<div><strong data-translate-lang="long_name">Long Name</strong><strong>: </strong> <span id="info-long-name"></span></div>
<div><strong data-translate-lang="short_name">Short Name</strong><strong>: </strong> <span id="info-short-name"></span></div> <div><strong data-translate-lang="short_name">Short Name</strong><strong>: </strong> <span id="info-short-name"></span></div>
<div><strong data-translate-lang="long_name">Long Name</strong><strong>: </strong> <span id="info-long-name"></span></div>
<div><strong data-translate-lang="hw_model">Hardware Model</strong><strong>: </strong> <span id="info-hw-model"></span></div>
<div><strong data-translate-lang="firmware">Firmware</strong><strong>: </strong> <span id="info-firmware"></span></div>
<div><strong data-translate-lang="role">Role</strong><strong>: </strong> <span id="info-role"></span></div>
<div><strong data-translate-lang="mqtt_gateway">MQTT Gateway</strong><strong>: </strong> <span id="info-mqtt-gateway"></span></div>
<div><strong data-translate-lang="channel">Channel</strong><strong>: </strong> <span id="info-channel"></span></div> <div><strong data-translate-lang="channel">Channel</strong><strong>: </strong> <span id="info-channel"></span></div>
</div>
<div class="node-info-col">
<div><strong data-translate-lang="latitude">Latitude</strong><strong>: </strong> <span id="info-lat"></span></div> <div><strong data-translate-lang="latitude">Latitude</strong><strong>: </strong> <span id="info-lat"></span></div>
<div><strong data-translate-lang="longitude">Longitude</strong><strong>: </strong> <span id="info-lon"></span></div> <div><strong data-translate-lang="longitude">Longitude</strong><strong>: </strong> <span id="info-lon"></span></div>
<div><strong data-translate-lang="role">Role</strong><strong>: </strong> <span id="info-role"></span></div>
<div><strong data-translate-lang="hw_model">Hardware Model</strong><strong>: </strong> <span id="info-hw-model"></span></div>
<div><strong data-translate-lang="mqtt_gateway">MQTT Gateway</strong><strong>: </strong> <span id="info-mqtt-gateway"></span></div>
</div>
<div class="node-info-col">
<div><strong data-translate-lang="first_update">First Update</strong><strong>: </strong> <span id="info-first-update"></span></div> <div><strong data-translate-lang="first_update">First Update</strong><strong>: </strong> <span id="info-first-update"></span></div>
<div><strong data-translate-lang="last_update">Last Update</strong><strong>: </strong> <span id="info-last-update"></span></div> <div><strong data-translate-lang="last_update">Last Update</strong><strong>: </strong> <span id="info-last-update"></span></div>
<div><strong data-translate-lang="firmware">Firmware</strong><strong>: </strong> <span id="info-firmware"></span></div>
<div> <div>
<strong data-translate-lang="statistics">Statistics</strong><strong>: </strong> <strong data-translate-lang="statistics">Statistics</strong><strong>: </strong>
<span id="info-stats" <span id="info-stats"
@@ -380,8 +384,7 @@
data-label-sent="Packets sent" data-label-sent="Packets sent"
data-label-seen="Times seen">—</span> data-label-seen="Times seen">—</span>
</div> </div>
</div>
</div> </div>
<!-- Map. --> <!-- Map. -->
@@ -1447,13 +1450,20 @@ async function loadPacketHistogram() {
const DAYS = 7; const DAYS = 7;
const now = new Date(); const now = new Date();
const dayKeyFromDate = (d) => {
const y = d.getFullYear();
const m = String(d.getMonth() + 1).padStart(2, "0");
const day = String(d.getDate()).padStart(2, "0");
return `${y}-${m}-${day}`;
};
const dayKeys = []; const dayKeys = [];
const dayLabels = []; const dayLabels = [];
for (let i = DAYS - 1; i >= 0; i--) { for (let i = DAYS - 1; i >= 0; i--) {
const d = new Date(now); const d = new Date(now);
d.setDate(d.getDate() - i); d.setDate(d.getDate() - i);
dayKeys.push(d.toISOString().slice(0, 10)); dayKeys.push(dayKeyFromDate(d));
dayLabels.push( dayLabels.push(
d.toLocaleDateString([], { month: "short", day: "numeric" }) d.toLocaleDateString([], { month: "short", day: "numeric" })
); );
@@ -1481,9 +1491,7 @@ async function loadPacketHistogram() {
for (const pkt of packets) { for (const pkt of packets) {
if (!pkt.import_time_us) continue; if (!pkt.import_time_us) continue;
const day = new Date(pkt.import_time_us / 1000) const day = dayKeyFromDate(new Date(pkt.import_time_us / 1000));
.toISOString()
.slice(0, 10);
if (!dayKeys.includes(day)) continue; if (!dayKeys.includes(day)) continue;
+223 -15
View File
@@ -85,6 +85,15 @@
border:none; border:none;
border-radius:4px; border-radius:4px;
} }
.chart-filter {
margin-bottom: 8px;
padding: 4px 6px;
background: #444;
color: #fff;
border: none;
border-radius: 4px;
}
{% endblock %} {% endblock %}
{% block head %} {% block head %}
@@ -118,15 +127,38 @@
</div> </div>
</div> </div>
<!-- Daily Charts -->
<div class="card-section"> <div class="card-section">
<p class="section-header" data-translate-lang="packets_per_day_all"> <p class="section-header" data-translate-lang="daily_snapshot_packets_history">
Packets per Day - All Ports (Last 14 Days) Daily Packet Snapshot History
</p> </p>
<div id="total_daily_all" class="total-count">Total: 0</div> <select id="snapshotPacketsRange" class="chart-filter" aria-label="Snapshot packet timeframe">
<button class="expand-btn" data-chart="chart_daily_all" data-translate-lang="expand_chart">Expand Chart</button> <option value="7">7 days</option>
<button class="export-btn" data-chart="chart_daily_all" data-translate-lang="export_csv">Export CSV</button> <option value="14">14 days</option>
<div id="chart_daily_all" class="chart"></div> <option value="30">1 month</option>
<option value="183">6 months</option>
<option value="365">1 year</option>
<option value="all" selected>All</option>
</select>
<button class="expand-btn" data-chart="chart_daily_snapshot_packets" data-translate-lang="expand_chart">Expand Chart</button>
<button class="export-btn" data-chart="chart_daily_snapshot_packets" data-translate-lang="export_csv">Export CSV</button>
<div id="chart_daily_snapshot_packets" class="chart"></div>
</div>
<div class="card-section">
<p class="section-header" data-translate-lang="daily_snapshot_nodes_gateways_history">
Daily Node/Gateway Snapshot History
</p>
<select id="snapshotNodesGatewaysRange" class="chart-filter" aria-label="Snapshot node and gateway timeframe">
<option value="7">7 days</option>
<option value="14">14 days</option>
<option value="30">1 month</option>
<option value="183">6 months</option>
<option value="365">1 year</option>
<option value="all" selected>All</option>
</select>
<button class="expand-btn" data-chart="chart_daily_snapshot_nodes_gateways" data-translate-lang="expand_chart">Expand Chart</button>
<button class="export-btn" data-chart="chart_daily_snapshot_nodes_gateways" data-translate-lang="export_csv">Export CSV</button>
<div id="chart_daily_snapshot_nodes_gateways" class="chart"></div>
</div> </div>
<!-- Packet Types Pie Chart with Channel Selector --> <!-- Packet Types Pie Chart with Channel Selector -->
@@ -247,6 +279,15 @@ async function fetchStats(period_type,length,portnum=null,channel=null){
}catch{return [];} }catch{return [];}
} }
async function fetchDailySnapshots(){
try{
const res=await fetch("/api/snapshots/daily");
if(!res.ok) return [];
const json=await res.json();
return json.data||[];
}catch{return [];}
}
async function fetchNodes(){ async function fetchNodes(){
try{ try{
const res=await fetch("/api/nodes"); const res=await fetch("/api/nodes");
@@ -267,6 +308,18 @@ async function fetchChannels(){
} }
} }
function buildPacketTypeBreakdownFromSeries(allData, perPortSeries){
const totalAll = (allData || []).reduce((sum,d)=>sum+(d.count??d.packet_count??0),0);
const results = perPortSeries.map(({portnum, data}) => ({
portnum,
count: (data || []).reduce((sum,d)=>sum+(d.count??d.packet_count??0),0),
}));
const trackedTotal = results.reduce((sum,d)=>sum+d.count,0);
const other = Math.max(totalAll - trackedTotal,0);
if(other>0) results.push({portnum:"other", count:other});
return results;
}
function processCountField(nodes,field){ function processCountField(nodes,field){
const counts={}; const counts={};
nodes.forEach(n=>{ nodes.forEach(n=>{
@@ -293,6 +346,22 @@ function prepareTopN(data,n=20){
return top; return top;
} }
function filterSnapshotsByRange(data, rangeValue){
if(rangeValue === "all") return data;
const days = Number.parseInt(rangeValue, 10);
if(Number.isNaN(days) || days <= 0 || !data.length) return data;
const latest = data.reduce((maxDate, item) => {
const itemDate = new Date(`${item.date}T00:00:00Z`);
return itemDate > maxDate ? itemDate : maxDate;
}, new Date(`${data[0].date}T00:00:00Z`));
const cutoff = new Date(latest);
cutoff.setUTCDate(cutoff.getUTCDate() - (days - 1));
return data.filter(item => new Date(`${item.date}T00:00:00Z`) >= cutoff);
}
// --- Chart Rendering --- // --- Chart Rendering ---
function renderChart(domId,data,type,color){ function renderChart(domId,data,type,color){
const el=document.getElementById(domId); const el=document.getElementById(domId);
@@ -365,6 +434,106 @@ function renderPieChart(elId,data,name){
return chart; return chart;
} }
function makeTelemetryLine(name, color, seriesData) {
return {
name,
type: "line",
smooth: true,
connectNulls: true,
showSymbol: true,
symbol: "circle",
symbolSize: 7,
lineStyle: {
width: 2,
color,
shadowColor: color.replace("1)", "0.35)"),
shadowBlur: 8,
shadowOffsetY: 3
},
itemStyle: {
color,
borderColor: "#000",
borderWidth: 1
},
areaStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [
{ offset: 0, color: color.replace("1)", "0.5)") },
{ offset: 0.6, color: color.replace("1)", "0.2)") },
{ offset: 1, color: "rgba(0,0,0,0)" }
])
},
data: seriesData
};
}
function renderDailySnapshotPacketsChart(domId,data){
const el=document.getElementById(domId);
if(!el) return;
const chart=echarts.getInstanceByDom(el) || echarts.init(el);
const labels=data.map(d=>d.date||"");
const packetCounts=data.map(d=>d.packet_count??0);
chart.setOption({
backgroundColor:'#272b2f',
tooltip:{trigger:'axis'},
legend:{
data:['Packets'],
textStyle:{color:'#ccc'}
},
grid:{left:'6%', right:'6%', bottom:'18%'},
xAxis:{
type:'category',
data:labels,
axisLine:{lineStyle:{color:'#aaa'}},
axisLabel:{rotate:45,color:'#ccc'}
},
yAxis:{
type:'value',
axisLine:{lineStyle:{color:'#aaa'}},
axisLabel:{color:'#ccc'}
},
series:[
makeTelemetryLine("Packets", "rgba(255,214,82,1)", packetCounts)
]
});
return chart;
}
function renderDailySnapshotNodesGatewaysChart(domId,data){
const el=document.getElementById(domId);
if(!el) return;
const chart=echarts.getInstanceByDom(el) || echarts.init(el);
const labels=data.map(d=>d.date||"");
const nodeCounts=data.map(d=>d.node_count??0);
const gatewayCounts=data.map(d=>d.gateway_count??0);
chart.setOption({
backgroundColor:'#272b2f',
tooltip:{trigger:'axis'},
legend:{
data:['Nodes','Gateways'],
textStyle:{color:'#ccc'}
},
grid:{left:'6%', right:'6%', bottom:'18%'},
xAxis:{
type:'category',
data:labels,
axisLine:{lineStyle:{color:'#aaa'}},
axisLabel:{rotate:45,color:'#ccc'}
},
yAxis:{
type:'value',
axisLine:{lineStyle:{color:'#aaa'}},
axisLabel:{color:'#ccc'}
},
series:[
makeTelemetryLine("Nodes", "rgba(79,155,255,1)", nodeCounts),
makeTelemetryLine("Gateways", "rgba(138,255,108,1)", gatewayCounts)
]
});
return chart;
}
// --- Packet Type Pie Chart --- // --- Packet Type Pie Chart ---
async function fetchPacketTypeBreakdown(channel=null) { async function fetchPacketTypeBreakdown(channel=null) {
@@ -388,9 +557,25 @@ async function fetchPacketTypeBreakdown(channel=null) {
// --- Init --- // --- Init ---
let chartHourlyAll, chartPortnum1, chartPortnum3, chartPortnum4, chartPortnum67, chartPortnum70, chartPortnum71; let chartHourlyAll, chartPortnum1, chartPortnum3, chartPortnum4, chartPortnum67, chartPortnum70, chartPortnum71;
let chartDailyAll, chartDailyPortnum1; let chartDailyAll, chartDailyPortnum1;
let chartDailySnapshotPackets, chartDailySnapshotNodesGateways;
let chartHwModel, chartRole, chartChannel; let chartHwModel, chartRole, chartChannel;
let chartGatewayChannel, chartGatewayRole, chartGatewayFirmware; let chartGatewayChannel, chartGatewayRole, chartGatewayFirmware;
let chartPacketTypes; let chartPacketTypes;
let defaultPacketTypesData = [];
let dailySnapshots = [];
function renderSnapshotCharts(){
const packetsRange = document.getElementById("snapshotPacketsRange")?.value || "all";
const nodesGatewaysRange = document.getElementById("snapshotNodesGatewaysRange")?.value || "all";
chartDailySnapshotPackets = renderDailySnapshotPacketsChart(
"chart_daily_snapshot_packets",
filterSnapshotsByRange(dailySnapshots, packetsRange)
);
chartDailySnapshotNodesGateways = renderDailySnapshotNodesGatewaysChart(
"chart_daily_snapshot_nodes_gateways",
filterSnapshotsByRange(dailySnapshots, nodesGatewaysRange)
);
}
async function init(){ async function init(){
// Channel selector // Channel selector
@@ -403,10 +588,9 @@ async function init(){
select.appendChild(opt); select.appendChild(opt);
}); });
// Daily all ports // Daily snapshot histogram
const dailyAllData=await fetchStats('day',14); dailySnapshots = await fetchDailySnapshots();
updateTotalCount('total_daily_all',dailyAllData); renderSnapshotCharts();
chartDailyAll=renderChart('chart_daily_all',dailyAllData,'line','#66bb6a');
// Daily port 1 // Daily port 1
const dailyPort1Data=await fetchStats('day',14,1); const dailyPort1Data=await fetchStats('day',14,1);
@@ -463,7 +647,9 @@ async function init(){
} }
// Packet types pie // Packet types pie
const packetTypesData = await fetchPacketTypeBreakdown(); const packetSeries = portnums.map((pn, i) => ({ portnum: pn, data: allData[i] }));
const packetTypesData = buildPacketTypeBreakdownFromSeries(hourlyAllData, packetSeries);
defaultPacketTypesData = packetTypesData;
const formatted = packetTypesData const formatted = packetTypesData
.filter(d=>d.count>0) .filter(d=>d.count>0)
.map(d=>({ .map(d=>({
@@ -502,6 +688,8 @@ window.addEventListener('resize',()=>{
chartPortnum67, chartPortnum67,
chartPortnum70, chartPortnum70,
chartPortnum71, chartPortnum71,
chartDailySnapshotPackets,
chartDailySnapshotNodesGateways,
chartDailyAll, chartDailyAll,
chartDailyPortnum1, chartDailyPortnum1,
chartHwModel, chartHwModel,
@@ -554,11 +742,24 @@ document.querySelectorAll(".export-btn").forEach(btn=>{
const option=chart.getOption(); const option=chart.getOption();
let rows=[]; let rows=[];
if(option.series[0].type==="bar"||option.series[0].type==="line"){ if(option.series[0].type==="bar"||option.series[0].type==="line"){
rows.push(["Period","Count"]);
const xData=option.xAxis[0].data; const xData=option.xAxis[0].data;
const yData=option.series[0].data; const series = option.series || [];
if(series.length > 1){
const headers = ["Period", ...series.map(s => s.name || "Series")];
rows.push(headers);
for(let i=0;i<xData.length;i++){
const row=[xData[i]];
series.forEach(s=>{
row.push((s.data && s.data[i]!==undefined) ? s.data[i] : "");
});
rows.push(row);
}
}else{
rows.push(["Period","Count"]);
const yData=series[0].data;
for(let i=0;i<xData.length;i++) rows.push([xData[i],yData[i]]); for(let i=0;i<xData.length;i++) rows.push([xData[i],yData[i]]);
} }
}
if(option.series[0].type==="pie"){ if(option.series[0].type==="pie"){
rows.push(["Name","Value","Percentage"]); rows.push(["Name","Value","Percentage"]);
const total=option.series[0].data.reduce((sum,d)=>sum+d.value,0); const total=option.series[0].data.reduce((sum,d)=>sum+d.value,0);
@@ -573,7 +774,7 @@ document.querySelectorAll(".export-btn").forEach(btn=>{
document.getElementById("channelSelect").addEventListener("change", async (e)=>{ document.getElementById("channelSelect").addEventListener("change", async (e)=>{
const channel = e.target.value; const channel = e.target.value;
const packetTypesData = await fetchPacketTypeBreakdown(channel); const packetTypesData = channel ? await fetchPacketTypeBreakdown(channel) : defaultPacketTypesData;
const formatted = packetTypesData const formatted = packetTypesData
.filter(d=>d.count>0) .filter(d=>d.count>0)
.map(d=>({ .map(d=>({
@@ -586,6 +787,9 @@ document.getElementById("channelSelect").addEventListener("change", async (e)=>{
chartPacketTypes = renderPieChart("chart_packet_types",formatted,"Packet Types (Last 24h)"); chartPacketTypes = renderPieChart("chart_packet_types",formatted,"Packet Types (Last 24h)");
}); });
document.getElementById("snapshotPacketsRange")?.addEventListener("change", renderSnapshotCharts);
document.getElementById("snapshotNodesGatewaysRange")?.addEventListener("change", renderSnapshotCharts);
// Kick everything off // Kick everything off
init(); init();
@@ -593,10 +797,14 @@ init();
async function loadConfigAndTranslations() { async function loadConfigAndTranslations() {
let langCode = "en"; let langCode = "en";
try { try {
if (window.site_config?.site) {
langCode = window.site_config.site.language || "en";
} else {
const resConfig = await fetch("/api/config"); const resConfig = await fetch("/api/config");
const cfg = await resConfig.json(); const cfg = await resConfig.json();
window.site_config = cfg; window.site_config = cfg;
langCode = cfg?.site?.language || "en"; langCode = cfg?.site?.language || "en";
}
} catch(err) { } catch(err) {
console.error("Failed to load /api/config:", err); console.error("Failed to load /api/config:", err);
window.site_config = { site: { language: "en" } }; window.site_config = { site: { language: "en" } };
+4
View File
@@ -354,6 +354,8 @@ async def graph_traceroute(request):
node_ids = set() node_ids = set()
for tr in traceroutes: for tr in traceroutes:
route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route) route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route)
if route is None:
continue
node_ids.add(tr.gateway_node_id) node_ids.add(tr.gateway_node_id)
for node_id in route.route: for node_id in route.route:
node_ids.add(node_id) node_ids.add(node_id)
@@ -379,6 +381,8 @@ async def graph_traceroute(request):
if tr.done and dest: if tr.done and dest:
continue continue
route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route) route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route)
if route is None:
continue
path = [packet.from_node_id] path = [packet.from_node_id]
path.extend(route.route) path.extend(route.route)
if tr.done: if tr.done:
+58 -1
View File
@@ -13,7 +13,7 @@ from meshtastic.protobuf.portnums_pb2 import PortNum
from meshview import database, decode_payload, store from meshview import database, decode_payload, store
from meshview.__version__ import __version__, _git_revision_short, get_version_info from meshview.__version__ import __version__, _git_revision_short, get_version_info
from meshview.config import CONFIG from meshview.config import CONFIG
from meshview.models import Node, NodePublicKey from meshview.models import DailySnapshot, Node, NodePublicKey
from meshview.models import Packet as PacketModel from meshview.models import Packet as PacketModel
from meshview.models import PacketSeen as PacketSeenModel from meshview.models import PacketSeen as PacketSeenModel
from meshview.radio.coverage import ( from meshview.radio.coverage import (
@@ -452,6 +452,59 @@ async def api_stats_count(request):
return web.json_response({"total_packets": total_packets, "total_seen": total_seen}) return web.json_response({"total_packets": total_packets, "total_seen": total_seen})
@routes.get("/api/snapshots/daily")
async def api_daily_snapshots(request):
length_raw = request.query.get("length")
length = None
if length_raw is not None:
try:
length = int(length_raw)
except ValueError:
return web.json_response({"error": "length must be an integer"}, status=400)
length = max(1, min(length, 3650))
try:
async with database.async_session() as session:
if length is not None:
rows = (
(
await session.execute(
select(DailySnapshot)
.order_by(DailySnapshot.snapshot_date.desc())
.limit(length)
)
)
.scalars()
.all()
)
rows = list(reversed(rows))
else:
rows = (
(
await session.execute(
select(DailySnapshot).order_by(DailySnapshot.snapshot_date.asc())
)
)
.scalars()
.all()
)
data = [
{
"date": row.snapshot_date.isoformat(),
"node_count": row.node_count,
"packet_count": row.packet_count,
"gateway_count": row.gateway_count,
"captured_at_us": row.captured_at_us,
}
for row in rows
]
return web.json_response({"data": data})
except Exception as e:
logger.error(f"Error in /api/snapshots/daily: {e}")
return web.json_response({"error": "Failed to fetch daily snapshots"}, status=500)
@routes.get("/api/edges") @routes.get("/api/edges")
async def api_edges(request): async def api_edges(request):
since = datetime.datetime.now() - datetime.timedelta(hours=12) since = datetime.datetime.now() - datetime.timedelta(hours=12)
@@ -480,6 +533,8 @@ async def api_edges(request):
route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route) route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route)
except Exception: except Exception:
continue continue
if route is None or tr.packet is None:
continue
path = [tr.packet.from_node_id] + list(route.route) path = [tr.packet.from_node_id] + list(route.route)
path.append(tr.packet.to_node_id if tr.done else tr.gateway_node_id) path.append(tr.packet.to_node_id if tr.done else tr.gateway_node_id)
@@ -781,6 +836,8 @@ async def api_traceroute(request):
# -------------------------------------------- # --------------------------------------------
for idx, tr in enumerate(traceroutes): for idx, tr in enumerate(traceroutes):
route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route) route = decode_payload.decode_payload(PortNum.TRACEROUTE_APP, tr.route)
if route is None:
continue
forward_list = list(route.route) forward_list = list(route.route)
reverse_list = list(route.route_back) reverse_list = list(route.route_back)
+9
View File
@@ -118,6 +118,15 @@ backup_dir = ./backups
backup_hour = 2 backup_hour = 2
backup_minute = 00 backup_minute = 00
# -------------------------
# Daily Snapshot Configuration
# -------------------------
[snapshot]
# Time to run daily network snapshot capture (24-hour format)
# Defaults are used if this section is omitted.
hour = 1
minute = 00
# ------------------------- # -------------------------
# Logging Configuration # Logging Configuration
+27
View File
@@ -129,6 +129,27 @@ async def daily_backup_at(hour: int = 2, minute: int = 0, backup_dir: str = ".")
await backup_database(database_url, backup_dir) await backup_database(database_url, backup_dir)
# -------------------------
# Daily snapshot scheduler
# -------------------------
async def daily_snapshot_at(hour: int = 1, minute: int = 0):
while True:
now = datetime.datetime.now()
next_run = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
if next_run <= now:
next_run += datetime.timedelta(days=1)
delay = (next_run - now).total_seconds()
cleanup_logger.info(f"Next daily snapshot scheduled at {next_run}")
await asyncio.sleep(delay)
try:
async with db_lock:
await mqtt_store.capture_daily_snapshot()
cleanup_logger.info("Daily snapshot captured successfully.")
except Exception as e:
cleanup_logger.error(f"Error capturing daily snapshot: {e}")
# ------------------------- # -------------------------
# Database cleanup using ORM # Database cleanup using ORM
# ------------------------- # -------------------------
@@ -290,6 +311,8 @@ async def main():
backup_dir = CONFIG.get("cleanup", {}).get("backup_dir", "./backups") backup_dir = CONFIG.get("cleanup", {}).get("backup_dir", "./backups")
backup_hour = get_int(CONFIG, "cleanup", "backup_hour", cleanup_hour) backup_hour = get_int(CONFIG, "cleanup", "backup_hour", cleanup_hour)
backup_minute = get_int(CONFIG, "cleanup", "backup_minute", cleanup_minute) backup_minute = get_int(CONFIG, "cleanup", "backup_minute", cleanup_minute)
snapshot_hour = get_int(CONFIG, "snapshot", "hour", 1)
snapshot_minute = get_int(CONFIG, "snapshot", "minute", 0)
logger.info(f"Starting MQTT ingestion from {CONFIG['mqtt']['server']}:{CONFIG['mqtt']['port']}") logger.info(f"Starting MQTT ingestion from {CONFIG['mqtt']['server']}:{CONFIG['mqtt']['port']}")
if cleanup_enabled: if cleanup_enabled:
@@ -300,6 +323,7 @@ async def main():
logger.info( logger.info(
f"Daily backups enabled: storing in {backup_dir} at {backup_hour:02d}:{backup_minute:02d}" f"Daily backups enabled: storing in {backup_dir} at {backup_hour:02d}:{backup_minute:02d}"
) )
logger.info(f"Daily snapshots enabled: capturing at {snapshot_hour:02d}:{snapshot_minute:02d}")
async with asyncio.TaskGroup() as tg: async with asyncio.TaskGroup() as tg:
tg.create_task( tg.create_task(
@@ -329,8 +353,11 @@ async def main():
) )
) )
tg.create_task(daily_snapshot_at(snapshot_hour, snapshot_minute))
if not cleanup_enabled and not backup_enabled: if not cleanup_enabled and not backup_enabled:
cleanup_logger.info("Daily cleanup and backups are both disabled by configuration.") cleanup_logger.info("Daily cleanup and backups are both disabled by configuration.")
cleanup_logger.info("Daily snapshots remain enabled by schedule configuration.")
# ------------------------- # -------------------------