Replace presentation-layer deduplication with collector-level approach:
- Add event_hash column to messages, advertisements, trace_paths, telemetry tables
- Handlers compute content hashes and skip duplicate events at insertion time
- Use 5-minute time buckets for advertisements and telemetry
- Include Alembic migration for schema changes
When multiple receiver nodes are running, the same mesh events (messages,
advertisements) are reported multiple times. This causes duplicate entries
in the Web UI.
Changes:
- Add hash_utils.py with deterministic hash functions for each event type
- Add `dedupe` parameter to messages and advertisements API endpoints (default: True)
- Update dashboard stats to use distinct counts for messages/advertisements
- Deduplicate recent advertisements and channel messages in dashboard
- Add comprehensive tests for hash utilities
Hash strategy:
- Messages: hash of text + pubkey_prefix + channel_idx + sender_timestamp + txt_type
- Advertisements: hash of public_key + name + adv_type + flags + 5-minute time bucket
Members can now have multiple associated nodes, each with a public_key
and node_role (e.g., 'chat', 'repeater'). This replaces the single
public_key field on members with a one-to-many relationship.
Changes:
- Add MemberNode model for member-node associations
- Update Member model to remove public_key, add nodes relationship
- Update Pydantic schemas with MemberNodeCreate/MemberNodeRead
- Update member_import.py to handle nodes list in seed files
- Update API routes to handle nodes in create/update/read operations
- Add Alembic migration to create member_nodes table and migrate data
- Update example seed file with new format
Replace the channel counts table with actual recent messages per channel:
- Added ChannelMessage schema for channel message summaries
- Dashboard API now fetches latest 5 messages for each channel with sender name lookups
- Network page displays messages grouped by channel with sender names and timestamps
- Only shows channels that have messages
The adv_type from the Advertisement record is often null, but the linked
Node has the correct adv_type. Now falls back to source_node.adv_type
when adv.adv_type is null.
- Swap Time and Type columns (Type now first)
- Add receiver_name and receiver_friendly_name to MessageRead schema
- Update messages API to fetch receiver node names and tags
- Make Receiver column a link showing name with public key prefix
- Rename Name column to Node
- Remove separate Public Key column
- Show name with public key prefix below (like Advertisements list)
- Add whitespace-nowrap to Last Seen column
- Join with Node table to get node names and tags for both source
and receiver nodes
- Display friendly_name (from tags), node_name, or advertised name
with priority in that order
- Show name with public key preview for both Node and Received By columns
- Remove Name and Type columns (usually null)
- Reorder columns: Node first, then Received By, then Time
- Link both Node and Received By to their node detail pages
- Show node name with public key preview when available
- Reduce hero section size and add stats bar with node/message counts
- Add new Advertisements page with public key filtering
- Update hero navigation buttons: Dashboard, Nodes, Advertisements, Messages
- Add Advertisements to main navigation menu
- Remove Hops column from messages list (always empty)
- Display full message text with proper multi-line wrapping
Internal database UUIDs (id, node_id, receiver_node_id) were being
exposed in API responses. These are implementation details that should
not be visible to API consumers. The canonical identifier for nodes
is the 64-char hex public_key.
Changes:
- Remove id, node_id from NodeTagRead, NodeRead schemas
- Remove id from MemberRead schema
- Remove id, receiver_node_id, node_id from MessageRead, AdvertisementRead,
TracePathRead, TelemetryRead schemas
- Update web map component to use public_key instead of member.id
for owner filtering
- Update tests to not assert on removed fields
- Return total_nodes, nodes_with_coords, and error in response
- Display meaningful messages when no nodes or no coordinates found
- Log API errors and node counts for debugging
- Use inline styles for marker colors instead of CSS classes for reliable rendering
- Center map on node locations when data is first loaded
- Refactor filter logic to separate recentering behavior
- Update legend to use inline styles
- Enhanced /map/data endpoint to include node role tag and member ownership
- Added client-side filtering for node type (chat, repeater, room)
- Added toggle to filter for infrastructure nodes only (role: infra)
- Added dropdown filter for member owner (nodes linked via public_key)
- Color-coded markers by node type with gold border for infrastructure
- Added legend showing marker types
- Dynamic count display showing total vs filtered nodes
Change debug logging to info level so contact processing is visible
in default log output. This helps verify that contact events are
being received and processed correctly.
Instead of sending all contacts in one MQTT message, the interface
now splits the device's contacts response into individual 'contact'
events. This is more consistent with other event patterns and makes
the collector simpler.
Interface changes:
- Add _publish_contacts() to split contacts dict into individual events
- Publish each contact as 'contact' event (not 'contacts')
Collector changes:
- Rename handle_contacts to handle_contact for single contact
- Simplify handler to process one contact per message
- Register handler for 'contact' events
The device sends contact entries with different field names than
originally expected:
- adv_name (not name) for the advertised node name
- type (numeric: 0=none, 1=chat, 2=repeater, 3=room) instead of node_type
Changes:
- Update handle_contacts to extract adv_name and convert numeric type
- Add NODE_TYPE_MAP for type conversion
- Always update node name if different (not just if empty)
- Add debug logging for node updates
- Update ContactInfo schema with actual device fields
Move _setup_event_subscriptions() from run() to connect() so that
event subscriptions are active before get_contacts() is called during
device initialization. Previously, CONTACTS events were lost because
subscriptions weren't set up until run() was called.
Trigger get_contacts() during receiver initialization to fetch the
device's contact database and broadcast CONTACTS events over MQTT.
This enables the collector to associate broadcast node names with
node records in the database.
Changes:
- Add get_contacts() abstract method to BaseMeshCoreDevice
- Implement get_contacts() in MeshCoreDevice using meshcore library
- Implement get_contacts() in MockMeshCoreDevice for testing
- Call get_contacts() in Receiver._initialize_device() after startup
- Replace JSON seed files with YAML format for better readability
- Auto-detect YAML primitive types (number, boolean, string) from values
- Add automatic seed import on collector startup
- Split lat/lon into separate tags instead of combined coordinate string
- Add PyYAML dependency and types-PyYAML for type checking
- Update example/seed and contrib/seed/ipnet with clean YAML format
- Update tests to verify YAML primitive type detection
- Add Member database model with name, callsign, role, description, contact, and public_key fields
- Add Member Pydantic schemas (MemberCreate, MemberUpdate, MemberRead, MemberList)
- Add members table to initial migration
- Add members API endpoints (GET/POST/PUT/DELETE /api/v1/members)
- Add member_import.py for importing from JSON files
- Update web layer to fetch members from API instead of file
- Add SEED_HOME setting (defaults to ./seed) for seed data files
- Add 'collector seed' command to import node_tags.json and members.json
- Rename tags.json to node_tags.json for consistency
- Move example seed data from example/data/* to example/seed/
- Update tests and configuration
Change tag import format from flat list with repeated public_keys to an
object keyed by public_key with nested tags. This makes the JSON more
intuitive and reduces redundancy.
New format supports both shorthand (string values) and full format
(with value and type):
{
"0123456789abcdef...": {
"friendly_name": "My Node",
"location": {"value": "52.0,1.0", "type": "coordinate"}
}
}
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Display friendly_name tags for nodes throughout web UI:
- nodes.html: Show friendly_name in node list table
- node_detail.html: Show in breadcrumb and page title
- network.html: Show in recent advertisements (24h stats)
- messages.html: Show sender friendly_name for direct messages
- map.py: Include friendly_name in map popup data
- API changes:
- dashboard.py: Look up friendly_name tags for recent advertisements
- messages.py: Look up sender friendly_name by pubkey_prefix
- Fix collector CLI import-tags command:
- Remove click.Path(exists=True) to allow optional file argument
- Add manual file existence check in function
- Add DATABASE_URL= to docker-compose.yml.example to prevent
host environment variable from overriding computed defaults
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add tag_import.py module with JSON file parsing and database upsert
- Convert collector CLI to group with subcommands for extensibility
- Add 'import-tags' command to import tags from JSON file
- Update docker-compose.yml.example with separated data directories:
- data/collector for tags.json
- data/web for members.json
- Add import-tags Docker service for easy containerized imports
- Add example data files in example/data/collector and example/data/web
- Add comprehensive test coverage (20 tests) for tag import
Add environment variable configuration for forwarding events to external
HTTP endpoints. Supports separate webhooks for advertisements, channel
messages, and direct messages with configurable timeouts, retries, and
authentication via X-Webhook-Secret header.
- Add webhook fields to CollectorSettings (config.py)
- Add create_webhooks_from_settings() to build webhooks from env vars
- Integrate WebhookDispatcher into Subscriber with background processing
- Update collector CLI to load and display webhook configuration
- Document webhook settings in README, AGENTS.md, .env.example
- Update docker-compose.yml.example with webhook env vars
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add --e2e flag to pytest to run e2e tests
- E2E tests skip by default with clear message
- Fix type annotations in webhook.py for mypy compliance
- Add proper type hints for comparison operations
- Add WebhookDispatcher for sending events to external services
- Webhook configuration loading from dict config
- JSONPath-like filter expression support for event filtering
- Async HTTP POST sending with httpx
- Retry logic with exponential backoff
- Comprehensive test suite
- Add health check infrastructure for Interface and Collector
- HealthReporter class for periodic status file updates
- CLI commands: meshcore-hub health interface/collector
- Updated Docker Compose to use CLI health checks
- File-based health status for non-HTTP components
- Update TASKS.md progress to 99% (218/221 tasks)
- Remaining 3 tasks are optional (docs/ directory)
Health Checks (6.3):
- Add is_healthy property and get_health_status() to Receiver/Sender
- Add is_healthy property and get_health_status() to Collector Subscriber
- Track device, MQTT, and database connection status
Documentation (6.5):
- Update README with Docker Compose profiles documentation
- Add serial device access instructions
- Update API documentation URLs and add health check info
CI/CD (6.6):
- Add .github/workflows/ci.yml for linting, testing, and building
- Add .github/workflows/docker.yml for Docker image builds
- Support Python 3.11 and 3.12 in CI matrix
- Configure Codecov for coverage reporting
End-to-End Testing (6.7):
- Add tests/e2e/ directory with Docker Compose test configuration
- Add e2e test fixtures with service health waiting
- Add comprehensive e2e tests for API, Web, and auth flows
Louis King