Mitigating high IO of all modules with a common delayed-write abstraction #740

New Issue

2026-10-05T04:42:02+02:00

adam commented

2026-10-05 04:42:02 +02:00

In https://github.com/jpnurmi/znc-clientbuffer/issues/9#issuecomment-265964265, @psychon suggests:

(Does it make sense to make this change in ZNC instead, so that it always writes .registry-files in such a delayed way?)

I would argue that this seems like a common pattern and it should be abstracted away to the ZNC core. A good flow might be:

Change the default value of bWriteToDisk to false.
Read the NV registry in CModule’s constructor (already happens).
Since the destructor (with SaveRegistry) might not always be called (when not exiting cleanly), have a default of how often we dump the registry to the disk—if it is needed—say, once every 15 seconds. But, perhaps, let the author change it.
During SetNV(bWriteToDisk = true) (and DelNV(true), and ClearNV(true)), since the module author wants to write right away, do write right away.
During SetNV(bWriteToDisk = false) (the default), check if the last write to disk happened more than 15 seconds ago, and if so, write right away, and reschedule the automatic timer to now() + 15 seconds. If it happened during the last 15 seconds, save it only in memory, don’t touch the disk.
In the automatic more-or-less-every-15-seconds timer, SaveRegistry to the disk — ONLY if it was changed since the last SaveRegistry.

This minimizes IO and benefits all.

There’s this slight downside: if a module tries to save something with the new default bWriteToDisk == false, and the last SaveRegistry happened less than 15 seconds ago, and during those 15 seconds ZNC exits uncleanly (e.g. power loss or SIGKILL), this last write will be lost. These conditions seem improbable/rare enough to justify implementing this, given the benefits.

(Keep in mind, that module authors that want to save crucial data, can call SetNV with bWriteToDisk to be sure it gets written right away and with no potential data loss. Also, keep in mind, that currently, when bWriteToDisk == false, in the event of an unclean exit, all data will be lost, not just the last 15 seconds).

In https://github.com/jpnurmi/znc-clientbuffer/issues/9#issuecomment-265964265, @psychon suggests: > (Does it make sense to make this change in ZNC instead, so that it always writes .registry-files in such a delayed way?) I would argue that this seems like a common pattern and it should be abstracted away to the ZNC core. A good flow might be: 1. Change the default value of `bWriteToDisk` to `false`. 1. Read the NV registry in `CModule`’s constructor (already happens). 1. Since the destructor (with `SaveRegistry`) might not always be called (when not exiting cleanly), have a default of how often we dump the registry to the disk—**if it is needed**—say, once every 15 seconds. But, perhaps, let the author change it. 1. During `SetNV(bWriteToDisk = true)` (and `DelNV(true)`, and `ClearNV(true)`), since the module author wants to write right away, *do* write right away. 1. During `SetNV(bWriteToDisk = false)` (the default), check if the last write to disk happened more than 15 seconds ago, and **if so, write right away**, and reschedule the automatic timer to `now() + 15 seconds`. If it happened during the last 15 seconds, save it only in memory, don’t touch the disk. 1. In the automatic *more-or-less-every-15-seconds* timer, `SaveRegistry` to the disk — **ONLY if it was changed since the last `SaveRegistry`**. This minimizes IO and benefits all. There’s this slight downside: if a module tries to save something with the new default `bWriteToDisk == false`, **and** the last `SaveRegistry` happened less than 15 seconds ago, **and** during those 15 seconds ZNC exits uncleanly (e.g. power loss or `SIGKILL`), this last write will be lost. These conditions seem improbable/rare enough to justify implementing this, given the benefits. (Keep in mind, that module authors that want to save crucial data, can call `SetNV` with `bWriteToDisk` to be sure it gets written right away and with no potential data loss. Also, keep in mind, that currently, when `bWriteToDisk == false`, in the event of an unclean exit, **all** data will be lost, not just the last 15 seconds).

adam commented

2026-10-05 04:42:03 +02:00

Seems like the bounty-source plugin did something weird to this issue. 😅 Dunno why this doesn’t parse.

Seems like the bounty-source plugin did something weird to this issue. :sweat_smile: Dunno why this doesn’t parse.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/znc#740