9 Commits

Author SHA1 Message Date
Mosh
f8088c3381 Merge pull request #725 from ipfs/mishmosh-patch-1
Add guidance for implementers section to IPIP-0499 blog post
2026-03-16 22:52:37 +01:00
Mosh
0f9630e68e Add guidance for implementers section 2026-03-16 14:20:10 -04:00
Mosh
c6ea1d6591 Merge pull request #724 from ipfs/post/2026-03-reproducible-cids
Create 2026-03-reproducible-cids.md
2026-03-13 01:50:56 +01:00
Mosh
cc5c14d00f apply lidel suggestion
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2026-03-10 12:02:27 -04:00
Mosh
b53b1b01eb Update src/_blog/2026-03-reproducible-cids.md
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2026-03-10 12:02:08 -04:00
Mosh
a8918145fd Add canonicalUrl 2026-03-09 17:33:53 -04:00
github-actions[bot]
da9458a59e Optimised images with calibre/image-actions 2026-03-09 21:31:54 +00:00
Mosh
c24a89daff Add header image 2026-03-09 22:31:18 +01:00
Mosh
2a583f35f7 Create 2026-03-reproducible-cids.md 2026-03-09 17:29:09 -04:00
3 changed files with 55 additions and 2 deletions

View File

@@ -52,7 +52,7 @@ To address this critical need, we [created a mirror of Myanmar Wikipedia](https:
## How to help co-hosting this?
You can run your own IPFS node and co-host a subset of Wikipedia, or store a full copy.
You can run your own IPFS node and co-host a subset of Wikipedia, store a full copy, or even follow collaborative cluster to pull in future updates automatically.
It is also possible to donate co-hosting costs by pinning specific CID to a remote service.
@@ -71,6 +71,14 @@ Be wary that the English one is far bigger than other ones, and pinning it requi
The size of a specific mirror can be read with `ipfs files stat /ipfs/{cid}`.
### Collaborative cluster
This is an advanced option aimed at server administrators and power users. The `wikipedia` cluster includes all language versions and its size only grows over time.
$ ipfs-cluster-follow wikipedia run --init wikipedia.collab.ipfscluster.io
See _Instructions_ at [collab.ipfscluster.io](https://collab.ipfscluster.io#instructions).
### Donate remote pins
When co-hosting with your own IPFS node is not possible, one can still help by pinning snapshot CIDs to a remote pinning service.
@@ -104,4 +112,4 @@ Below are areas that could use a helping hand, and ideas looking for someone to
* **Improving the way ZIM is represented on IPFS.** When we store an original ZIM on IPFS, the DAG is produced by `ipfs add --cid-version 1`. This works fine, but with additional research on customizing DAG creation, we may improve deduplication and speed when doing range requests for specific bytes. There are different stages to explore here: if any of them sounds interesting to you, please comment in [distributed-wikipedia-mirror/issues/42](https://github.com/ipfs/distributed-wikipedia-mirror/issues/42).
* Stage 1: Invest some time to benchmark parameter space to see if low hanging fruits exists.
* Stage 2: Create a DAG builder that understands ZIM format and maximizes deduplication of image assets by representing them as sub-DAGs with dag-pb files.
* Stage 3: Research augmenting or replacing ZIM with [IPLD](https://ipld.io/). How can we maximize block deduplication across all snapshots and languages? How would an IPLD-based search index work?
* Stage 3: Research augmenting or replacing ZIM with [IPLD](https://ipld.io/). How can we maximize block deduplication across all snapshots and languages? How would an IPLD-based search index work?

View File

@@ -0,0 +1,45 @@
---
title: "IPIP-0499: Updating IPFS Standards for Consistent, Reproducible CIDs"
description: "IPIP-0499 makes IPFS CIDs deterministic and consistent across all implementations."
author: Michelle Lee
date: 2026-03-09
permalink: '/2026-03-reproducible-cids/'
canonicalUrl: https://ipfsfoundation.org/ipip-0499-updating-ipfs-standards-for-consistent-reproducible-cids/
header_image: '/2026-03-muted-teal-grid.png'
tags:
- ipfs
- cid
- ipip
---
# IPIP-0499: Updating IPFS Standards for Consistent, Reproducible CIDs
_Cross-posted from the [IPFS Foundation blog](https://ipfsfoundation.org/ipip-0499-updating-ipfs-standards-for-consistent-reproducible-cids/)._
If you've worked with IPFS, you know that a content identifier (CID) is supposed to be a fingerprint for your data: feed in the same data, get the same CID. In practice, that hasn't been true. The same file or directory uploaded with [Kubo](https://github.com/ipfs/kubo), [Helia](https://github.com/ipfs/helia), or [Singularity](https://github.com/data-preservation-programs/singularity) could produce three different CIDs, because each tool uses different settings for how to split files into chunks, how wide to build the internal DAG tree, and when to switch to a HAMT structure for large directories.
To fix this, [IPIP-0499](https://specs.ipfs.tech/ipips/ipip-0499) has been merged into the IPFS specification. It introduces two named configuration profiles — `unixfs-v1-2025` and `unixfs-v0-2015` — each defining a complete set of DAG construction parameters.
### Benefits: Predictable CIDs, Faster Verification
The most immediate benefit is that CIDs are now deterministic. Previously, comparing two CIDs for "the same file" could return a false negative simply because the tools that created them used different chunk sizes or DAG widths. With this update, identical input processed by any implementation conforming to the same profile will always produce the same CID. That's the behavior most developers intuitively expect, and it's what makes content-addressing useful as a verification primitive.
A second major gain is verification efficiency. Without profile guarantees, confirming that two CIDs represent the same content requires fetching the underlying data and computing and comparing their Merkle DAGs. With deterministic CIDs, you can compare the CIDs directly without fetching the data itself. This matters especially at scale.
Finally, `unixfs-v1-2025` is a more performant default. Switching from 256 KiB chunks to 1 MiB, and from 174 to 1024 links per DAG node, produces shallower trees: a 1 TiB file requires 3 levels of DAG traversal instead of 4, with roughly 4x fewer total nodes. That translates to faster random access and seeking in large files, and fewer CIDs being announced to public routing infrastructure like the Amino DHT.
### Guidance for Implementers
- Implementations *MUST* support the `unixfs-v1-2025` profile.
- The legacy `unixfs-v0-2015` profile is provided for backward compatibility and *MAY* be supported by implementations that need to produce CIDs matching historical Kubo output.
- Implementations *SHOULD* allow users to inspect default values and adjust configuration options related to CID generation.
You can read the full specification, including detailed parameter comparison tables, at [specs.ipfs.tech/ipips/ipip-0499](https://specs.ipfs.tech/ipips/ipip-0499/).
### Current Implementation Support
IPIP-499 is now supported across many implementations in Go ([kubo 0.40](https://github.com/ipfs/kubo/releases/tag/v0.40.0), [boxo 0.37](https://github.com/ipfs/boxo/releases/tag/v0.37.0), [go-ipfs-cmds 0.16](https://github.com/ipfs/go-ipfs-cmds/releases/tag/v0.16.0)) and JS ([helia/unixfs 7.0.3](https://www.npmjs.com/package/@helia/unixfs), [ipfs-unixfs-importer 16.1.1](https://github.com/ipfs/js-ipfs-unixfs/releases/tag/ipfs-unixfs-importer-16.1.1)).
### Acknowledgements
Special thanks to [@lidel](https://github.com/lidel/), [@2color](https://github.com/2color), and to everyone who contributed to the [forum discussion](https://discuss.ipfs.tech/t/should-we-profile-cids/18507/13) and [review](https://github.com/ipfs/specs/pull/499).

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB