draft/chathistory support #974

New Issue

2026-10-05T04:56:28+02:00

adam commented

2026-10-05 04:56:28 +02:00

Now that the specification has a draft[0], can ZNC support chathistory as a server? I have a client that supports it and would very much be able to infinite scroll.

[0] https://ircv3.net/specs/extensions/chathistory

Now that the specification has a draft[0], can ZNC support chathistory as a server? I have a client that supports it and would very much be able to infinite scroll. [0] https://ircv3.net/specs/extensions/chathistory

adam commented

2026-10-05 04:56:28 +02:00

What is your idea where ZNC would get the old messages from? Do you want to disable buffer playback when a client connects to ZNC and instead the playback is made available via chathistory?

What is your idea where ZNC would get the old messages from? Do you want to disable buffer playback when a client connects to ZNC and instead the playback is made available via chathistory?

adam commented

2026-10-05 04:56:29 +02:00

I think this would behave in a similar manner to znc-playback module (https://github.com/jpnurmi/znc-playback / https://wiki.znc.in/Playback).

Thus if the capability is enabled, then we should prevent auto playback and allow the client to fetch the buffers as desired using the CHATHISTORY commands. The underlying data source will stay the same (CBuffer), if the upstream server also supports chathistory then we can forward the command upstream. I think we can treat forwarding chathistory to upstream server we can treat this as next steps and not part of initial support. At the moment only one IRCD supports chathistory (oragono).

I'd like to move away from the znc.in/playback batch identifier and switch to the chathistory one as it is standardised. This change can be separated as it isn't specific to the draft/chathistory capability, however it would need to be done as a dependency.

I think the following is needed for initial support:

Use chathistory batch type instead of znc.in/playback
Implement CHATHISTORY commands to client
- Will need support for standard replies, if any CMessage logic will need to change (I do not know if this is the case).
Expose capability draft/chathistory which would disable automatic playback

There are some additional things we can support, but as I read the spec I don't think they are mandatory and can be added after:

Support message-ids - support for msgid so that we can support chathistory commands with msgid parameter alongside time operations. This is optional as it requires upstream server to support message-ids which at this point is rate for the larger networks.
Support draft/event-playback capability - once enabled QUIT/PART/INVITE/KICK events should be played back in the history. Currently the module buffextras provides this functionality, however if I understand it correctly it serialises text into CBuffer. I think we will need to change CBuffer to be able to store other types of messages such as JOIN,PART,TAGMSG etc.
- Open question: Should this be part of ZNC core? I can see it may be undesired and potentially a breaking change as these events can fill limited buffers to hide actual messages.
Support forwarding chathistory to servers which support it natively

Technically unrelated, but I'd love to see:

Storing messages in external system, (sqlite per network?)
- Allow restarting ZNC without loosing messages
  - This limitation at the moment makes it hard for ZNC "operators" (those serving ZNC to multiple users). You don't know what users may have messages that can be silently lost to never be found again when you need to restart ZNC for patching etc.
- This can potentially make the log module less useful since the re-playable buffers can be used instead
- Prior art, oragono supports persisting messages in msql.

I think this would behave in a similar manner to znc-playback module (https://github.com/jpnurmi/znc-playback / https://wiki.znc.in/Playback). Thus if the capability is enabled, then we should prevent auto playback and allow the client to fetch the buffers as desired using the `CHATHISTORY` commands. The underlying data source will stay the same (`CBuffer`), if the upstream server also supports chathistory then we can forward the command upstream. I think we can treat forwarding chathistory to upstream server we can treat this as next steps and not part of initial support. At the moment only one IRCD supports chathistory (oragono). I'd like to move away from the `znc.in/playback` batch identifier and switch to the [`chathistory`](https://ircv3.net/specs/extensions/batch/chathistory-3.3) one as it is standardised. This change can be separated as it isn't specific to the `draft/chathistory` capability, however it would need to be done as a dependency. I think the following is needed for initial support: - [ ] Use [`chathistory`](https://ircv3.net/specs/extensions/batch/chathistory-3.3) batch type instead of `znc.in/playback` - [ ] Implement `CHATHISTORY` commands to client - Will need support for [standard replies](https://ircv3.net/specs/extensions/standard-replies), if any `CMessage` logic will need to change (I do not know if this is the case). - [ ] Expose capability `draft/chathistory` which would disable automatic playback There are some additional things we can support, but as I read the spec I don't think they are mandatory and can be added after: - [ ] Support `message-ids` - support for msgid so that we can support chathistory commands with `msgid` parameter alongside time operations. This is optional as it requires upstream server to support message-ids which at this point is rate for the larger networks. - [ ] Support `draft/event-playback` capability - once enabled QUIT/PART/INVITE/KICK events should be played back in the history. Currently the module buffextras provides this functionality, however if I understand it correctly it serialises text into CBuffer. I think we will need to change CBuffer to be able to store other types of messages such as JOIN,PART,TAGMSG etc. - Open question: Should this be part of ZNC core? I can see it may be undesired and potentially a breaking change as these events can fill limited buffers to hide actual messages. - [ ] Support forwarding chathistory to servers which support it natively Technically unrelated, but I'd love to see: - [ ] Storing messages in external system, (sqlite per network?) - Allow restarting ZNC without loosing messages - This limitation at the moment makes it hard for ZNC "operators" (those serving ZNC to multiple users). You don't know what users may have messages that can be silently lost to never be found again when you need to restart ZNC for patching etc. - This can potentially make the `log` module less useful since the re-playable buffers can be used instead - Prior art, oragono supports persisting messages in msql.

adam commented

2026-10-05 04:56:30 +02:00

Allow restarting ZNC without loosing messages

savebuff module somewhat addresses this

> Allow restarting ZNC without loosing messages savebuff module somewhat addresses this

adam commented

2026-10-05 04:56:31 +02:00

Thus if the capability is enabled, then we should prevent auto
playback and allow the client to fetch the buffers as desired using
the CHATHISTORY commands. The underlying data source will stay the
same (CBuffer),

Yes, CHATHISTORY allows the client to handle how messages should be
played back, for example to emulate infinite scrolling (typically with
LATEST and BEFORE), to fill gaps in their cache (typically with AFTER
and BETWEEN) or to search for a message (typically with AROUND).

if the upstream server also supports chathistory
then we can forward the command upstream. I think we can treat
forwarding chathistory to upstream server we can treat this as next
steps and not part of initial support. At the moment only one IRCD
supports chathistory (oragono).

Of course, CHATHISTORY may be useful for ZNC as a client, to retrieve
gaps in its logs (e.g. after network failures). However, I don't think
it is a valid use-case for a bouncer to forward requests to upstreams.
The main point of CHATHISTORY is for downstreams to navigate through
bouncer logs.

There are some additional things we can support, but as I read the
spec I don't think they are mandatory and can be added after:

Support message-ids - support for msgid so that we can
support chathistory commands with msgid parameter alongside time
operations. This is optional as it requires upstream server to
support message-ids which at this point is rate for the larger
networks.

msgid is indeed a non trivial feature, but it can be implemented
without upstream support: for example, messages ID could be the row ID
in a SQL database that stores the logs.

Other than these comments I completely agree.

> Thus if the capability is enabled, then we should prevent auto > playback and allow the client to fetch the buffers as desired using > the `CHATHISTORY` commands. The underlying data source will stay the > same (`CBuffer`), Yes, CHATHISTORY allows the client to handle how messages should be played back, for example to emulate infinite scrolling (typically with LATEST and BEFORE), to fill gaps in their cache (typically with AFTER and BETWEEN) or to search for a message (typically with AROUND). > if the upstream server also supports chathistory > then we can forward the command upstream. I think we can treat > forwarding chathistory to upstream server we can treat this as next > steps and not part of initial support. At the moment only one IRCD > supports chathistory (oragono). Of course, CHATHISTORY may be useful for ZNC as a client, to retrieve gaps in its logs (e.g. after network failures). However, I don't think it is a valid use-case for a bouncer to forward requests to upstreams. The main point of CHATHISTORY is for downstreams to navigate through bouncer logs. > There are some additional things we can support, but as I read the > spec I don't think they are mandatory and can be added after: > > - [ ] Support `message-ids` - support for msgid so that we can > support chathistory commands with `msgid` parameter alongside time > operations. This is optional as it requires upstream server to > support message-ids which at this point is rate for the larger > networks. msgid is indeed a non trivial feature, but it can be implemented without upstream support: for example, messages ID could be the row ID in a SQL database that stores the logs. Other than these comments I completely agree.

adam commented

2026-10-05 04:56:31 +02:00

msgid is indeed a non trivial feature, but it can be implemented
without upstream support: for example, messages ID could be the row ID
in a SQL database that stores the logs.

@hhirtz I don't think that's possible, the specifications states msgid value must be from the originating server, for which ZNC is not. ZNC wouldn't be able to enforce other uniqueness constraints for the upstream IRC network. This can create a confusing situation for clients because they may see msgid and have message-tags capability and expect commands like TAGMSG to work with message replies and reactions which ZNC won't be able to forward to the upstream server.

Message IDs spec:

The tag value MUST be a unique ID chosen by the originating server. Uniqueness in this context means that any other message transmitted on the entire network at any time MUST NOT share the same value

chathistory capability spec:

The msgid tag that identifies each individual message in a response MUST be the msgid tag as originally sent by the IRC server.

Meanwhile, bouncers implementing the server side of this specification may not be able to provide message IDs (when they are mediating for a server that does not support the message-tags capability). Therefore, clients with full support for message-tags MAY wish to implement fallback logic that relies only on server-time.

> msgid is indeed a non trivial feature, but it can be implemented > without upstream support: for example, messages ID could be the row ID > in a SQL database that stores the logs. @hhirtz I don't think that's possible, the specifications states msgid value must be from the originating server, for which ZNC is not. ZNC wouldn't be able to enforce other uniqueness constraints for the upstream IRC network. This can create a confusing situation for clients because they may see msgid and have message-tags capability and expect commands like TAGMSG to work with message replies and reactions which ZNC won't be able to forward to the upstream server. [Message IDs spec](https://ircv3.net/specs/extensions/message-ids): > The tag value MUST be a unique ID chosen by the originating server. Uniqueness in this context means that any other message transmitted on the entire network at any time MUST NOT share the same value [chathistory capability spec](https://ircv3.net/specs/extensions/chathistory): > The msgid tag that identifies each individual message in a response MUST be the msgid tag as originally sent by the IRC server. > Meanwhile, bouncers implementing the server side of this specification may not be able to provide message IDs (when they are mediating for a server that does not support the message-tags capability). Therefore, clients with full support for message-tags MAY wish to implement fallback logic that relies only on server-time.

adam commented

2026-10-05 04:56:32 +02:00

@hhirtz with such SQL ids, it's possible that different messages will clash:

Server without message id support sends a message
ZNC assigns it id 6346, and stores it. Sends that id to client, etc
Years later, server gains support of message ids
Server sends a message with id 6346. What ZNC is supposed to do with this message now?

@hhirtz with such SQL ids, it's possible that different messages will clash: 1. Server without message id support sends a message 2. ZNC assigns it id 6346, and stores it. Sends that id to client, etc 3. Years later, server gains support of message ids 4. Server sends a message with id 6346. What ZNC is supposed to do with this message now?

adam commented

2026-10-05 04:56:33 +02:00

It's possible for a bouncer to implement message-id even when upstreams
don't. Yes, it will lead to loss of client tags

when relaying messages from downstreams to upstreams (e.g.
@+reply=123123 should be dropped if the server doesn't have tag
support). This is OK, after all client tags are optional by design
since most IRC users don't use clients that support tags.
the other way around, when an upstream sends a message with an
ID unknown to ZNC, but then clients wouldn't have made sense of it
even if ZNC had relayed it raw.

Anyway, ZNC does not have to relay IDs from upstream. A firsthand
msgid impl would involve two things:

Craft an identifier that is unique for each message stored in ZNC logs,
Have a way to map the (eventual) upstream message ID with the
crafted, internal ID.

So in this example,

Server without message id support sends a message

ZNC assigns it id 6346, and stores it. Sends that id to client, etc

Years later, server gains support of message ids

Server sends a message with id 6346. What ZNC is supposed to do with this message now?

ZNC would store the message and make a unique ID for it. It can make
one from a SQL primary key, or generate a UUID or something.

When ZNC relays upstream messages to downstreams, it replaces the
upstream msgid with its crafted ID (or adds it if it's missing).

When ZNC relays downstream messages to upstream, ZNC replaces any
crafted ID with their upstream equivalent (or removes them if the server
doesn't support message tags).

It's possible for a bouncer to implement message-id even when upstreams don't. Yes, it will lead to loss of client tags - when relaying messages from downstreams to upstreams (e.g. `@+reply=123123` should be dropped if the server doesn't have tag support). This is OK, after all client tags are optional by design since most IRC users don't use clients that support tags. - the other way around, when an upstream sends a message with an ID unknown to ZNC, but then clients wouldn't have made sense of it even if ZNC had relayed it raw. Anyway, ZNC does not have to relay IDs from upstream. A firsthand msgid impl would involve two things: - Craft an identifier that is unique for each message stored in ZNC logs, - Have a way to map the (eventual) upstream message ID with the crafted, internal ID. So in this example, > 1. Server without message id support sends a message > 2. ZNC assigns it id 6346, and stores it. Sends that id to client, etc > 3. Years later, server gains support of message ids > 4. Server sends a message with id 6346. What ZNC is supposed to do with this message now? ZNC would store the message and make a unique ID for it. It can make one from a SQL primary key, or generate a UUID or something. When ZNC relays upstream messages to downstreams, it replaces the upstream msgid with its crafted ID (or adds it if it's missing). When ZNC relays downstream messages to upstream, ZNC replaces any crafted ID with their upstream equivalent (or removes them if the server doesn't support message tags).

adam commented

2026-10-05 04:56:33 +02:00

Here's a ZNC plugin for chathistory:

https://codeberg.org/hhirtz/znc-chathistory

Here's a ZNC plugin for chathistory: https://codeberg.org/hhirtz/znc-chathistory

adam commented

2026-10-05 04:56:34 +02:00

Here's a ZNC plugin for chathistory:

https://codeberg.org/hhirtz/znc-chathistory

Cool

Haven't read the code, but does AGPL there mean that the user needs to use only AGPL IRC clients to connect to ZNC with this module?

> Here's a ZNC plugin for chathistory: > > https://codeberg.org/hhirtz/znc-chathistory Cool Haven't read the code, but does AGPL there mean that the user needs to use only AGPL IRC clients to connect to ZNC with this module?

adam commented

2026-10-05 04:56:35 +02:00

No way or else google chrome would be AGPL at this point.

The module is AGPL because most of the code is ported from soju, but if this doesn't work out ill try to relicense under one that's compatible.

No way or else google chrome would be AGPL at this point. The module is AGPL because most of the code is ported from soju, but if this doesn't work out ill try to relicense under one that's compatible.

adam commented

2026-10-05 04:56:35 +02:00

The plugin has been relicensed under Apache 2.0:
https://codeberg.org/hhirtz/znc-chathistory/commit/f668ca7f80b9933479de9493ac03fc7652247de4

The plugin has been relicensed under Apache 2.0: https://codeberg.org/hhirtz/znc-chathistory/commit/f668ca7f80b9933479de9493ac03fc7652247de4

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/znc#974