Merge branch 'main' into ipfs-chromium-post

.forestry/front_matter/templates/blog-post.yml

@@ -154,6 +154,7 @@ pages:
 - src/_blog/2021-05-31-distributed-wikipedia-mirror-update.md
 - src/_blog/2022-12-07-testground-in-2022.md
 - src/_blog/2023-01-10-announcing-pin-tweet-to-ipfs.md
+- src/_blog/2023-01-26-announcing-durin.md
 - src/_blog/3s-studio-bringing-unreal-engine-to-ipfs.md
 - src/_blog/a-brave-new-wallet-the-future-of-the-browser-wallet.md
 - src/_blog/a-guide-to-ipfs-connectivity-in-web-browsers.md

src/.vuepress/theme/components/blog/Card.vue

@@ -39,6 +39,7 @@ export default {
         case 'News coverage':
         case 'Release notes':
         case 'Tutorial':
+        case 'Ecosystem content':
         case 'Video':
           return LinkCard
 

src/_blog/2023-01-26-announcing-durin.md

@@ -0,0 +1,82 @@
+---
+title: "Announcing Durin: a New Mobile App for the IPFS Network"
+description: "Durin is a native mobile application for iOS and Android that lets you read and share content on the IPFS network"
+date: 2023-05-11
+permalink: "/announcing-durin/"
+header_image: '/durin-featured-image.png'
+author: David Justice
+tags:
+- Durin
+- mobile
+- ios
+- android
+- app store
+- web3 storage
+- web3
+---
+
+Today we are excited to announce **Durin**, a native mobile application for [iOS](https://apps.apple.com/us/app/durin/id1613391995) and [Android](https://play.google.com/store/apps/details?id=ai.protocol.durin) built to give users a new way to read and share with IPFS. It also serves as a sandbox for the Browsers & Platforms team to experiment with IPFS in a mobile environment.
+
+## Background
+
+To date, it's been difficult to access, upload, and share IPFS content using a mobile device. This is for a number of reasons, one of which is that [Kubo](https://github.com/ipfs/kubo)(the initial implementation of the protocol) was simply not built with mobile in mind. The IPFS approach to P2P for many years was about running servers, but [that is changing](https://blog.ipfs.tech/2023-03-implementation-principles/). In the meantime, we wanted to provide a quick and easy way for users to access basic IPFS features on mobile and set up a testing ground for future explorations.
+
+## Accessing IPFS Content
+
+The transport-agnostic nature of IPFS content addresses means there are many ways to find and retrieve content on the IPFS public network. On a mobile device, the best balance of decentralization and device performance is to align with the network model of the device OS - transient connectivity.
+
+We do this in Durin by connecting to the IPFS network via multiple HTTP gateways. On app launch, Durin pings a list of public gateways, and determines which route is the most reliable and fastest way to access the network. This approach is functional but not optimal. We're working on specifications for multi-gateway connectivity patterns which balance a number of factors - such as verifiability guarantees, reader privacy, and not overloading gateways.
+
+<br>
+
+<img src="../assets/announcing-durin-ipfs/gateway-durin.png" alt="gateway list">
+
+IPFS addresses are not natively supported in most web browsers or any mobile operating systems today. Durin registers as an `ipfs` scheme handler so that addresses are handled when encountered in applications and on the web.
+
+On iOS Safari `ipfs://` protocol links will be redirected to Durin, where the app will translate and redirect the user to the fastest public gateway, making the content available on mobile. Unfortunately the auto-redirects do not work using Chrome's android app. They have not yet [implemented `registerProtocolHandler`](https://bugs.chromium.org/p/chromium/issues/detail?id=178097&q=protocol%20handler%20mobile&can=2)). 
+
+<br>
+
+<img src="../assets/announcing-durin-ipfs/durin-redirect.gif" alt="redirect functionality on mobile safari">
+
+## Sharing to IPFS from Mobile
+
+Mobile devices are transiently connected and low-powered, so they do not make good servers. For sharing files and data to IPFS, Durin uses a [pinning service](https://docs.ipfs.tech/concepts/persistence/#persistence-permanence-and-pinning) to do this on behalf of the user.
+
+We currently rely on [web3.storage](https://web3.storage/) for file uploads. `web3.storage` is a service that makes decentralized file storage accessible by hosting data on IPFS for the user, the way a web host does for HTTP today. NOTE: _Using a single service like this is not ideal, as users don’t hold those keys. We plan to experiment with approaches to ensuring maximal user ownership of their data while also providing remote storage and data availability._
+
+Durin also saves a local history of uploads already shared.
+
+<br>
+
+<img src="../assets/announcing-durin-ipfs/filelist-durin.png" alt="uploaded files list">
+
+Using a single remote service is a usable first step, but doesn't provide long term user control of the data published. We're looking at tighter integration with local OS data storage, local sharing between devices when possible, and pluggable remote service support.
+
+## Install Durin
+
+Durin is available now for mobile phones in the iOS app store and Google Play store.
+<br /> 
+<a href="https://apps.apple.com/us/app/durin/id1613391995" class="cta-button"> Get Durin in iOS App Store </a>
+<br />
+<a href="https://play.google.com/store/apps/details?id=ai.protocol.durin" class="cta-button"> Get Durin in Google Play Store</a>
+
+## The Future
+
+Durin is an experiment in learning how to expose and integrate IPFS features into mobile operating systems in ways which align optimally with those environments. We're trying out variety of ideas from contacts integration, photo sync & backup, filecoin storage, peer to peer bluetooth connectivity.
+
+We'd love to hear your ideas and feedback, and have you participate!
+
+* [ipfs-shipyard/durin on Github](https://github.com/ipfs-shipyard/durin)
+* [HackMd project document](https://hackmd.io/XtxGZoxqQ46X1GO7srrhMQ)
+* [Feedback link](https://github.com/ipfs-shipyard/durin/issues)
+
+Join the #browsers-and-platforms channel which is bridged across the [Filecoin Slack](https://filecoin.io/slack/), [IPFS Discord](https://discord.gg/vZTcrFePpt) and [Element/Matrix](https://matrix.to/#/#browsers-and-standards:ipfs.io).
+
+Checkout the IPFS Thing talk, discussing Durin's role and some future ideas for the app.
+
+<iframe width="560" height="315" src="https://www.youtube.com/embed/QkhnKm-fCs4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+
+## Shoutout
+
+Shout out to [Trigram](https://www.trigram.co/) for continued work on Durin. 

src/_blog/2023-how-to-hose-dynamic-content-on-ipfs.md

@@ -0,0 +1,505 @@
+---
+title: 'How to Host Dynamic Content on IPFS'
+description: 'This article presents a design for hosting dynamic content on IPFS using IPLD, IPNS, and DHT Provider Records.'
+author: tabcat
+date: 2023-05-17
+permalink: '/2023-how-to-host-dynamic-content-on-ipfs/'
+header_image: '/hosting-dynamic-content.png'
+tags:
+  - 'dynamic-content'
+  - 'hosting'
+  - 'ipld'
+  - 'ipns'
+  - 'dht'
+---
+
+The InterPlanetary File System (IPFS) is a distributed, peer-to-peer file system designed to make the web faster, safer, and more resilient. Although IPFS excels at hosting static content, hosting dynamic content remains a challenge. This article presents a design for hosting dynamic content on IPFS using InterPlanetary Linked Data (IPLD), InterPlanetary Name Service (IPNS), and DHT Provider Records.
+
+## Table of Contents
+<!-- TOC start -->
+
+- [Understanding Key Components](#understanding-key-components)
+  * [IPLD](#ipld)
+  * [IPNS](#ipns)
+  * [PeerID](#peerid)
+  * [Provider Records](#provider-records)
+- [Defining the Problem](#defining-the-problem)
+- [Achieving Dynamicity](#achieving-dynamicity)
+  * [Read and Write Steps](#read-and-write-steps)
+    + [Writing](#writing)
+    + [Reading](#reading)
+  * [Dynamic-Content IDs](#dynamic-content-ids)
+  * [Manifest Document](#manifest-document)
+- [Use-case: Edge-computed Applications](#use-case-edge-computed-applications)
+  * [Edge Devices](#edge-devices)
+  * [Pinning Servers](#pinning-servers)
+  * [Replication](#replication)
+- [Roadblocks and Workarounds](#roadblocks-and-workarounds)
+  * [No 3rd Party Publishing to DHT](#no-3rd-party-publishing-to-dht)
+  * [No Delegated Refreshing of IPNS OR Provider Records](#no-delegated-refreshing-of-ipns-or-provider-records)
+- [Example](#example)
+  * [Usage](#usage)
+    + [Clone the Repo](#clone-the-repo)
+    + [Install Packages](#install-packages)
+    + [Run Examples](#run-examples)
+  * [What's Happening?](#whats-happening)
+  * [Sample Outputs](#sample-outputs)
+- [Credits](#credits)
+- [Get Involved](#get-involved)
+- [FAQ](#faq)
+
+<!-- TOC end -->
+
+
+<br/>
+
+## Understanding Key Components
+
+### IPLD
+
+[IPLD](https://ipld.io/) is a data model for linking and addressing data across distributed systems. In IPFS, IPLD stores immutable data, providing [content-addressed storage](https://en.wikipedia.org/wiki/Content-addressable_storage). Data stored in IPLD has a unique [Content Identifier](https://docs.ipfs.tech/concepts/content-addressing/) (CID) derived from its content, ensuring data integrity.
+
+### IPNS
+
+[IPNS](https://docs.ipfs.tech/concepts/ipns/) is a decentralized naming system that allows you to create a mutable reference to an immutable CID. With IPNS, you can create a persistent address that always points to the latest version of your content, even as it changes over time.
+
+### PeerID
+
+A [Libp2p PeerID](https://docs.libp2p.io/concepts/fundamentals/peers/#peer-id) is a unique identifier for each node in the network, derived from a [public key](https://en.wikipedia.org/wiki/Public-key_cryptography). PeerIDs help find, identify, and communicate with other nodes.
+
+### Provider Records
+
+[Provider Records](https://docs.ipfs.tech/concepts/dht/) are a fundamental part of IPFS's Distributed Hash Table (DHT). When requesting IPFS content, a node queries the DHT for Provider Records associated with the requested CID. These records contain the PeerID of peers with the content, enabling the user to establish a connection and retrieve the data.
+
+---
+> **It's important to note that IPNS names and PeerIDs use the same [key structures](https://specs.ipfs.tech/ipns/ipns-record/#ipns-keys).**
+---
+
+<br/>
+
+## Defining the Problem
+
+Databases on IPFS have been gaining more attention recently. In essence, these database protocols use IPLD to store replica data.
+And they commonly use a real-time protocol like [Gossipsub](https://docs.libp2p.io/concepts/pubsub/overview/) with IPLD to sync database changes peer-to-peer.
+Using this design to create [local-first](https://www.inkandswitch.com/local-first/) databases looks quite promising.
+However, local-first databases are often highly [sharded](https://en.wikipedia.org/wiki/Partition_(database)) and run on end-user devices.
+
+This presents the problem of peers being few and unreliable to sync with.
+One solution is to add reliable database peers to the mix, either self-hosted or hosted by a service.
+There are two disadvantages to this approach:
+
+- Each project must build infra tooling
+- Users need a live instance of each database protocol used
+
+It would benefit all related protocols to have a general solution for asynchronous replication of dynamic content.<br/>
+*Think pinning layer for dynamic content.*
+
+This standardized layer would complement the app-specific protocols used for real-time replication.
+
+<br/>
+
+## Achieving Dynamicity
+
+Let's look at a replication algorithm for one of the first databases on IPFS, [OrbitDB](https://github.com/orbitdb). The algorithm is roughly as follows:
+
+  1. Join a shared pubsub channel for the database.
+  2. On seeing a new pubsub peer in the shared channel, attempt to join a direct pubsub channel ([ipfs-pubsub-1on1](https://github.com/ipfs-shipyard/ipfs-pubsub-1on1)).
+  3. On committing an update to the local replica, advertise replica root CIDs on each direct pubsub channel.
+  4. On receiving a replica root CIDs advertisement on a direct pubsub, traverse the remote replica for changes to merge.
+
+The design presented in this article works similarly but replaces pubsub with Provider Records and IPNS. Essentially, all parts of replication get encoded into ~persistent IPFS components.
+
+- Provider Records to find collaborators
+- IPNS to point to the latest version of a device's replica
+
+---
+> **Swapping pubsub for ~persistent components makes building on history without any collaborators online possible.**
+---
+
+The main contribution is the novel use of Provider Records.
+Instead of tying a CID to PeerIDs of nodes hosting that content, the records tie a "Dynamic-Content ID" to IPNS names.
+Each IPNS name resolves to the latest CID of a device's local replica.
+
+*Collaborating on dynamic content is possible without knowing any previous collaborators or needing them to be online as long as their replica data is kept available via a pinner.*
+
+If you are familiar with publishing Provider Records to the DHT, *you may have spotted a problem here*.
+The source of the problem is a check DHT servers do when receiving an `ADD_PROVIDER` query, addressed in [Roadblocks and Workarounds](#roadblocks-and-workarounds).
+
+<img src="https://raw.githubusercontent.com/tabcat/dynamic-content/master/.assets/dynamic-content-diagram.png" width="333">
+
+---
+> **The Merkle-DAGs built with IPLD provide a persistent and efficient layer for collaborators to sync.**
+---
+
+<br/>
+
+### Read and Write Steps
+
+Describes the process of reading/writing dynamic content to IPFS:
+
+#### Writing
+
+1. Make changes to the local replica
+2. Push replica data to the IPLD pinner
+3. Republish IPNS to point to new CID root
+4. Add IPNS key as a provider of the Dynamic Content's ID
+
+#### Reading
+
+1. Query the DHT for Providers of the Dynamic Content's ID
+2. Resolve providers' IPNS keys to CIDs
+3. Resolve CIDs to IPLD data
+4. Merge changes with the local replica
+
+---
+> **Titling this article 'Replication on IPFS' might have been more accurate, but 'Hosting Dynamic Content on IPFS' sounded waaay better.**
+---
+
+<br/>
+
+### Dynamic-Content IDs
+
+A Dynamic-Content ID (DCID) looks like a CID. Also, both DCIDs and CIDs reference and identify content on the DHT.
+
+*Where the two IDs differ is in their creation.*
+
+While CIDs come from the hash of some static content, DCIDs are a permutation of the CID of a manifest document.
+This immutable manifest document "describes" the dynamic content.
+
+As stated in the previous section, DCIDs identify unique dynamic content.
+They point to IPNS names by using Provider Records on the DHT.
+
+---
+> **Disclaimer: Dynamic-Content IDs, or DCIDs, only exist for the purpose of this article. It is not an official spec or part of IPFS. (expect a name change because I also hate "DCIDs" 🤢🤮)**
+---
+
+<br/>
+
+### Manifest Document
+
+Manifest documents, a term from OrbitDB, describe some unique dynamic content.
+Manifests are immutable and contain information like the protocol and parameter used.
+
+This document format is not formally specified, but included below is a specification for this article:
+
+**dag-cbor**
+```js
+// cbor type reference: https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1
+{
+  protocol: type 3
+  param: type 5
+}
+```
+
+`protocol`: a text string field containing a protocol id
+
+`param`: a key value map for exclusive use by the `protocol`
+
+```js
+// takes description of the dynamic content (protocol + params) 
+ // returns manifest (Block) and dynamic-content id (CID) 
+ export async function DynamicContent ( 
+   { protocol, param }: { protocol: string, param: any } 
+ ): 
+   Promise<{ id: CID, manifest: BlockView }> 
+ { 
+  
+   // create manifest 
+   const manifest = await Block.encode({ value: { protocol, param }, codec, hasher }) 
+  
+   // create dcid 
+   const dynamic = new TextEncoder().encode('dynamic') 
+   const bytes = new Uint8Array(dynamic.length + manifest.cid.multihash.digest.length) 
+   bytes.set(dynamic) 
+   bytes.set(manifest.cid.multihash.digest, dynamic.length) 
+   const dcid = CID.create( 
+     manifest.cid.version, 
+     manifest.cid.code, 
+     await hasher.digest(bytes) 
+   ) 
+  
+   return { id: dcid, manifest } 
+ } 
+ ```
+
+
+Above is a code block from the example attached to this article.
+It shows a manifest document "describing" the dynamic content using the `protocol` and `param` properties.
+It also shows the DCID derived from the manifest's CID.
+
+<br/>
+
+## Use-case: Edge-computed Applications
+
+This design is particularly useful when paired with local-first databases.
+These databases are partitioned (a.k.a. sharded) to only the interested parties.
+It's common for only a few collaborators to be a part of a database, and there may be long periods without any of them online.
+This context makes it challenging to build upon the history of collaborators, a challenge this design can potentially solve.
+
+### Edge Devices
+
+- Handle application logic and merging of replicas from other collaborators.
+- Consist of a network of potentially unreliable peers that may come online and go offline at various times.
+- Ensure the application history is available by commanding pinning servers.
+
+### Pinning Servers
+
+- Reliable storage servers that keep dynamic content available on IPFS.
+- Pin IPLD replicas, and refresh IPNS and Provider Records for clients.
+- Execute no app-specific code
+
+### Replication
+
+The design presented in this article is a replication protocol.
+However, it is not a real-time replication protocol.
+Applications with real-time features should include an app-specific replication protocol for use with other online collaborators.
+Combining two replication protocols with these properties results in preserved and real-time P2P applications.
+
+---
+> **Pinning servers, in this context, provide a general and reliable replication layer to fall back on when no other collaborators are online.**
+---
+
+<br/>
+
+## Roadblocks and Workarounds
+
+It should be clear now that using Provider Records this way was not intended.
+This brings us to the roadblock...
+
+### No 3rd Party Publishing to DHT
+
+[DHT servers validate that the PeerIDs inside received Provider Records match the PeerID of the node adding them.](https://github.com/libp2p/specs/tree/master/kad-dht#rpc-messages)
+
+This check makes adding Provider Records for multiple PeerIDs to the DHT difficult.
+Not great if you want to participate in multiple pieces of dynamic content as each will require its own IPNS name.
+A Libp2p node may only add its own PeerId as a provider. This PeerId is also known as the "self" key.
+
+There are two workarounds for now:
+
+1. Use the "self" key for IPNS, and have it point to a CID for a map(DCID -> root replica CID) for all relevant dynamic content.
+2. Spin up *ephemeral* libp2p nodes to refresh each IPNS name as a provider every [22hours](https://github.com/libp2p/specs/tree/master/kad-dht#content-provider-advertisement-and-discovery).
+
+### No Delegated Refreshing of IPNS OR Provider Records
+
+Delegated publishing of IPNS and Provider Records is necessary to realize the edge-computed applications use case.
+Unfortunately, there are no official plans to add this feature.
+
+<br/>
+
+## Example
+
+---
+> **USES HELIA 😲🤩 !!!! DHT IN 😵‍💫 JAVASCRIPT 😵‍💫 😵 !! DYNAMIC CONTENT ON IPFS!?🧐!?**
+---
+
+This example shows dynamic-content replication using IPLD, IPNS, and Provider Records. There are 3 [helia](https://github.com/ipfs/helia) (IPFS) nodes running in a single script, named `client1`, `client2`, and `server`. `client1` and `client2` dial `server` and use the `/ipfs/kad/1.0.0` protocol. After dialing, clients can add IPNS and Provider records to the DHT server. Clients also add IPLD data to `server` programmatically.
+
+![](../assets/hosting-dynamic-content-mermaid-3.png)
+
+---
+> **`client1`, `client2`, and `server ` are all in memory Helia nodes created by a single script.**
+
+> **IPLD data is added to the server by clients by accessing `server.blockstore.put` from within the script (programmatically). As opposed to using an HTTP API like in any real use-case.**
+---
+
+### Usage
+
+- Requires [npm and Node v18](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)
+
+#### Clone the Repo
+
+`git clone https://github.com/tabcat/dynamic-content.git`
+
+#### Install Packages
+
+`npm install`
+
+#### Run Examples
+
+There are two example scripts. One is interactive, meaning after the example runs, a REPL starts with global variables available to operate the replication manually.
+
+The scripts are `npm run example` and `npm run interactive`.
+
+**If something is broken please open an [issue](https://github.com/tabcat/dynamic-content/issues)!**
+
+<br/>
+
+### What's Happening?
+
+The example consists of 3 [Helia](https://github.com/ipfs/helia) nodes, named `client1`, `client2`, and `server`.
+The `server` represents a reliable machine used as a
+
+1. IPLD pinning server
+2. DHT server
+
+---
+> **IPNS and Provider records are both stored in the DHT.**
+---
+
+The clients are unreliable machines used to read and write dynamic content.
+In the example, `client1` does all the writing, and `client2` does all the reading.
+
+![](../assets/hosting-dynamic-content-mermaid-4.png)
+
+<br/>
+
+This is a very high overview of what's going on.
+Remember, this design uses only IPLD/IPNS/Provider Records.
+It may be helpful to read [index.ts](./src/index.ts) (~200 LOC) for clarity.
+
+### Sample Outputs
+
+In case you are unable to run the example, below shows all the output that would occur:
+
+<details>
+  <summary>`npm run example`</summary>
+  
+  ```sh
+$ npm run example
+
+> dynamic-content@1.0.0 example
+> npm run build && node dist/index.js
+
+
+> dynamic-content@1.0.0 build
+> tsc
+
+server is pinning ipld and serving dht ipns and provider records
+client1: online
+client1: added new values to set { nerf this }
+client1: set state: { nerf this }
+client1: encoded to raw data
+client1: pushed data to pinner
+client1: published ipns:12D3KooWRzE1FNCRXuz1C8Z3G8Q5oBg3C5nhKANSsFq377P1mWVn with value cid:bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq
+client1: advertised ipns:12D3KooWRzE1FNCRXuz1C8Z3G8Q5oBg3C5nhKANSsFq377P1mWVn as set provider
+client1: offline
+
+--- no peers online, Zzzzz ---
+
+client2: online
+dht query returned empty response
+client2: found ipns:12D3KooWRzE1FNCRXuz1C8Z3G8Q5oBg3C5nhKANSsFq377P1mWVn as set provider
+client2: resolved ipns:12D3KooWRzE1FNCRXuz1C8Z3G8Q5oBg3C5nhKANSsFq377P1mWVn to bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq
+client2: resolved ipfs:bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq to raw data
+client2: decoded raw data
+client2: added new values to set { nerf this }
+client2: set state: { nerf this }
+client2: offline
+  ```
+</details>
+
+<details>
+  <summary>`npm run interactive`</summary>
+   <br/>
+
+  The interactive example starts a REPL after the example has run. 
+
+  ```sh
+$ npm run interactive
+
+> dynamic-content@1.0.0 interactive
+> npm run build && node dist/interactive.js
+
+
+> dynamic-content@1.0.0 build
+> tsc
+
+server is pinning ipld and serving dht ipns and provider records
+client1: online
+client1: added new values to set { nerf this }
+client1: set state: { nerf this }
+client1: encoded to raw data
+client1: pushed data to pinner
+client1: published ipns:12D3KooWQXCo6Wzw7NmJRLC2peAX7fU6gHSydEKNAJfyfXCEwHFL with value cid:bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq
+client1: advertised ipns:12D3KooWQXCo6Wzw7NmJRLC2peAX7fU6gHSydEKNAJfyfXCEwHFL as set provider
+client1: offline
+
+--- no peers online, Zzzzz ---
+
+client2: online
+dht query returned empty response
+client2: found ipns:12D3KooWQXCo6Wzw7NmJRLC2peAX7fU6gHSydEKNAJfyfXCEwHFL as set provider
+client2: resolved ipns:12D3KooWQXCo6Wzw7NmJRLC2peAX7fU6gHSydEKNAJfyfXCEwHFL to bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq
+client2: resolved ipfs:bafyreihypffwyzhujryetatiy5imqq3p4mokuz36xmgp7wfegnhnjhwrsq to raw data
+client2: decoded raw data
+client2: added new values to set { nerf this }
+client2: set state: { nerf this }
+client2: offline
+
+--- interactive example ---
+
+client1: online
+client2: online
+
+  Usage:
+
+  globals
+
+  help: this message
+
+  client1: helia client node (sender)
+  client2: helia client node (receiver)
+  server: helia ipld/ipns pinner and dht server
+
+  // compare the 2 clients sets
+  set1: client1's set variable
+  set2: client2's set variable
+
+  await connect(<client>)       // connects client to server
+  await disconnect(<client>)    // disconnects client from server
+
+  await update(...<string[]>)   // create and publish changes from client1 - requires client1 to be connected
+  await sync()                  // syncs changes to client2 - requires client2 to be connected
+
+>
+  ```
+</details>
+
+---
+> **Note: in practice, the DHT queries related to the Dynamic Content's ID only need to be run initially. Afterward, a protocol meant for real-time replication with online collaborators can be used.**
+---
+
+<br/>
+
+## Credits
+
+Big thanks to [@autonome](https://github.com/autonome), [@SgtPooki](https://github.com/sgtpooki), and [@lidel](https://github.com/lidel) for help writing this article!
+
+Also thanks to [@willscott](https://github.com/willscott) for answering all my DHT questions in [#libp2p-implementers](https://app.element.io/#/room/#libp2p-implementers:ipfs.io)!
+
+<br/>
+
+## Get Involved
+
+Sound interesting? Get involved! Come [chat](https://matrix.to/#/#hldb:matrix.org)
+
+Have a question? Create an [issue](https://github.com/tabcat/dynamic-content/issues)
+
+[I](https://github.com/tabcat)'m implementing this in [tabcat/zzzync](https://github.com/tabcat/zzzync)
+
+<br/>
+
+## FAQ
+
+**Q**: Why not just share an IPNS name between devices to update?
+
+**A**: IPNS names are not built to handle concurrent writes and should not be extended to do so. They are signed, versioned documents that one device should be able to update. As shown here, they are essential for creating a system that can handle concurrent writes.
+
+<br/>
+
+**Q**: Isn't this going to be slow?
+
+**A**: This design complements real-time replication by providing a general and reliable layer to fall back to. It adds two steps on top of resolving a CID: 1) the DHT provider query and 2) the IPNS name resolutions.
+Developers must reason how to design replicas for efficient storage and replication over IPLD.
+
+<br/>
+
+**Q**: Provider Records do not support this use case. Could this affect DHT measurements?
+
+**A**: If this use case became prevalent, it could affect DHT measurements. Using Provider Records this way would make it look like the content providers are offline because the PeerIDs are used only for IPNS.
+
+<br/>
+
+**Q**: Could IPNS and Provider Records be swapped out for alternatives and achieve the same goal?
+
+**A**: Absolutely. The goal is to provide a general and reliable replication layer. Additionally, the more widespread the building blocks used, the more existing infrastructure can be leveraged.

src/_blog/2023-ipfs-thing-recap-content-routing.md

@@ -0,0 +1,94 @@
+---
+title: 'Recap: Content Routing (þing 2023)'
+description: 'A recap of the Content Routing track including summaries, links, and videos.'
+author: Masih Derkani
+date: 2023-05-15
+permalink: '/2023-ipfs-thing-content-routing-track/'
+header_image: '/ipfs-thing-2023-recap/content-routing/content-routing-recap-slides.png'
+tags:
+  - 'thing'
+  - 'þing'
+  - 'event'
+  - 'recap'
+  - 'track'
+  - 'content'
+  - 'routing'
+---
+
+The term "content" is ubiquitous in discussions about knowledge sharing, regardless of the platform used. IPFS takes this term to a new level by defining content as an immutable piece of information, identified by a cryptographic hash that defines its identity. Any change in the information results in a different identity, making the content immutable. This property has a subtle yet powerful advantage: a receiver of a piece of information can verify its authenticity based on its identifier. This simple concept leads to an important question: how can one locate shared content using its identity? 🤔 This is where "Content Routing" comes in.
+
+Content Routing is the crucial first step in exchanging content within the IPFS network. Once a Content Identifier (CID) is generated from a piece of information, Content Routing enables the information to be both discoverable and discovered. In other words, it involves telling the network, "Hey, I have content, and here is its CID," as well as answering peer questions such as "Who has this CID?".
+
+This seemingly simple yet paramount functionality enables the network to share immutable and verifiable pieces of information. Since the inception of IPFS as a protocol, Content Routing has taken various forms and utilized several techniques to fulfill its promise of sharing knowledge. It remains an essential component of the IPFS ecosystem, as evidenced by its dedicated track at IPFS þing 2023 in Brussels, Belgium, last month. 
+
+At IPFS þing 2022 a year ago, Content Routing was divided into two tracks: [Privacy](https://www.youtube.com/watch?v=VLU44qtXypE&list=PLuhRWgmPaHtTegfLTVFYtTtqTKQEtDvxW) and [Performance](https://www.youtube.com/watch?v=AWbobt9oHZ0&list=PLuhRWgmPaHtSF3oIY3TzrM-Nq5IU_RTXb). This year, both tracks were combined into one glorious Content Routing track that covered both areas. We had the privilege of hosting talks from community leaders who discussed the impressive improvements in performance and scalability of content routing systems, the privacy preservation techniques that cut across different systems, as well as community call-outs and discussions on how to get involved and build a better decentralized web together.
+
+The track offered a comprehensive view of the content routing evolution since the inception of IPFS and showcased the latest advancements in the IPFS ecosystem. It provided an overview of the [InterPlanetary Network Indexer (IPNI)](https://github.com/ipni) and explained how it enables the mass publication and lookup of content across hundreds of billions of CIDs. The latest developments in reader privacy preservation, a mechanism that allows private lookups of content on both the IPFS DHT and IPNI, were also presented.
+
+The rest of this blog post offers highlights, links, and a brief commentary on the talks.
+
+The full playlist of talks at the IPFS þing 2023 Content Routing track can be found [here](https://www.youtube.com/watch?v=oe7fjOl-q0s&list=PLuhRWgmPaHtRBWV3SvInC5ATS8aKV3lsW). To learn more about Content Routing, check out the previous tracks at [IPFS Camp 2022](https://www.youtube.com/watch?v=7nb5oEpURCU&list=PLuhRWgmPaHtRqhFZ-CAstJ0RIq7Vs-4eO) and the [IPFS YouTube channel](https://www.youtube.com/@IPFSbot/playlists).
+
+## Content Routing Track Introduction by Masih Derkani
+
+[Masih](https://derkani.org/) presented an overview of Content Routing as a concept, its evolution over time, along with the evolutionary trends of content routing in the IPFS ecosystem. The talk illustrated what routing content in the IPFS network looks like today and explained how the mesh of content providers of different sizes interconnects. It also showcased the sub-systems that enable content routing to "just work", regardless of where the data resides.
+
+@[youtube](oe7fjOl-q0s)
+
+## Opening the DHT to large content providers by Guillaume Michel
+
+How does a 1M x reduction in opened connections sound? That's right, providing data via the DHT is becoming much more efficient for large content providers thanks to "regions". [Gui](https://github.com/guillaumemichel) presented the latest research on how the DHT key space can be divided across regions to reduce the number of connections as well as messages sent to make content discoverable via the IPFS DHT.
+
+@[youtube](bXaL64fp55c)
+
+## IPNI: the InterPlanetary Network Indexer by Masih Derkani
+
+Talking of large content providers, IPNI, the InterPlanetary Network Indexer, is an alternative routing system designed from scratch to provide content by the bucket load. [Masih](https://derkani.org/) presented how IPNI achieves this by betting on storage becoming cheaper and using replicas to reduce the need for trust to provide single hop lookup for trillions of CIDs. He explained how IPNI handles changes in the subset of CIDs advertised by content providers in a super-efficient protocol. IPNI as a concept has been around for about a year; it is the same protocol that makes FileCoin content discoverable over the IPFS network. As a protocol, it has now grown large enough to deserve its own "InterPlanetary" acronym and a growing set of [specifications](https://github.com/ipni/specs).
+
+@[youtube](_EDJXeDtcX4)
+
+## cid.contact: one year on by Masih Derkani
+
+Having made the distinction between "protocol" and "implementation", [Masih](https://derkani.org/) presented a second talk on [`cid.contact`](https://cid.contact), the largest most mature IPNI cluster. `cid.contact` is built into [Kubo](https://github.com/ipfs/kubo) as a default routing system since version [`0.18.0`](https://github.com/ipfs/kubo/releases/tag/v0.18.0) and is the content router of choice for [Lassie](https://youtu.be/d5SzSm8NkUU) used by [Rhea](https://youtu.be/p89i9_AskIw). The talk covered the latest architecture of `cid.contact` and the newest features, such as cascading lookup over IPFS DHT and BitSwap, that make it a one-stop content router, tuned to find content no matter where it might be. `cid.contact` has ingested over 1.3 trillion CIDs from hundreds of providers, and just turned one this April. Happy 1st birthday! 🎂
+
+@[youtube](CPlOdNqJ8og)
+
+## IPFS Content Routing Workgroup, an introduction by Torfinn Olsen
+
+Ever wondered where the content routers meet? 🧙 Look no further; the Content Routing Workgroup is it! [Torfinn](https://github.com/TorfinnOlsen) provided an overview of what the workgroup aims for, how community decisions are made, and how things get prioritized in the pipeline. He presented the roadmap ahead for the workgroup and invited the community to join. The workgroup meetings are public and open to all. You can find recordings of the previous meetups [here](https://www.youtube.com/watch?v=LsCH8xw3__c&list=PLuhRWgmPaHtRP5lVouK_eqhC98xaej6Px). Whether it's the next big idea you'd like to propose or just to observe what content routers get up to all day, you are most welcome.
+
+@[youtube](MagS8ly_YXE)
+
+## DHT ~~Double Hashing~~ Reader Privacy Updates & Migration Plan by Yiannis Psaras
+
+It was at the first IPFS þing in Reykjavík where [Gui](https://github.com/guillaumemichel) presented the idea of [Double Hashing](https://www.youtube.com/watch?v=ZPIDU1-JnVc) in the context of Content Routing. Yep; we love hashes so much we're gonna do it twice! In this technique, rather than looking up a CID straight up, it is hashed again and its "double-hashed" value is the key that's used for lookup. In turn, the lookup results are then returned in encrypted form using the original CID as the encryption key. Pretty nifty, right?! Gui presented two follow-up talks on this at IPFS Camp 2022 further [explaining the core idea](https://youtu.be/VBlx-VvIZqU) and what [transitioning to it would mean for content routing](https://youtu.be/m-6_VZ8e1tk). At Brussels, [Yiannis](https://github.com/yiannisbot) walked us through the latest updates in the rollout of ~~Double Hashing~~ Reader Privacy to the IPFS DHT, one of the routing systems in use today. The initial phase of privacy preservation focuses on the "reader" side, where an external observer cannot know what a user is looking up without knowing the original CID. Later work will build on this to expand the privacy benefits to the "writer" side, i.e., content providers.
+
+@[youtube](FP4kKemco4w)
+
+## Double Hashing in IPNI: Reader Privacy at scale by Ivan Schasny
+
+Privacy preservation is a quality that cuts right across routing systems. ✂️ This means no matter how the content is advertised or found we _want to_ preserve the user's privacy. In this talk [Ivan](https://github.com/ischasny) walked us through what this means for IPNI and how it is changing the architecture of `cid.contact` to incorporate reader privacy at its very core: `cid.contact` is moving to _only_ store encrypted provider records which means even the servers do not know what CIDs are being looked up. He expanded on how this big change is being rolled out garcefully, in stages and what's to come in the near future. Watch the [`#ipni` channel on FileCoin Slack](https://filecoinproject.slack.com/archives/C02T827T9N0) for the latest updates.
+
+@[youtube](Q46zJ_mai2c)
+
+## Private data: state of the art by Ian Preston
+
+Taking things one step further on the privacy front, [Ian](https://peergos.org/about#ian_) and his team have been busy building privacy deep at the heart of [Peergos](https://peergos.org/). How does it work? The talk takes a deep dive into the Peergos architecture and how it utilizes `cryptree+`, BATs, and Capabilities to enable post-quantum ciphertext-level access control with improved metadata preservation and better performance. Ian walked us through the challenges they faced, such as garbage collection, and how the team overcame them to make application sandboxing a piece of cake. 🍰 As for the icing, check out Ian's slides shared right from Peergos [here](https://peergos.net/public/demo/talks/2023/ipfs-thing/private-data/web/index.html?open=true).
+
+@[youtube](HVyrVUI2-RA)
+
+## Content Advertisement Mirroring by Andrew Gillis
+
+As the adoption of IPNI as an alternative content routing protocol continues to grow, so does the need for scaling. 🚀 At IPFS Camp 2022, [Andrew](https://github.com/gammazero) presented how IPNI is [scaling the content routing](https://youtu.be/qaCB0UKqwAk). Building on top of previous work, this talk covered how the replication of content advertisements from providers is making ingestion (and re-ingestion) 5X faster. This means new IPNI instances can use alternative sources to build up their index records with much higher velocity, moving us closer to a federated mesh of IPNI instances that continue to maintain lookup latency in orders of a few milliseconds at 10^15 scale!
+
+The talk was followed by a discussion on a set of open questions as we scale the IPNI network. Take a look and get involved right from where we left off at the next Content Routing Workgroup meeting!
+
+@[youtube](6l0i8DjhpLg)
+
+## A Massive Shout-out
+
+It's great to see the IPFS community coming together and celebrating the latest advancements in the field. A big thank you to all who attended the track at Brussels and to the speakers who presented and helped generate questions. Last but not least, a massive shout-out to the community that tirelessly drives the vision (a better web for all) forward. 🙇
+
+And there's more to look forward to! The dates for [IPFS Camp 2023](https://lu.ma/ipfscamp23-prereg) in Bangalore, India have already been announced for early November. It's never too early to start preparing your talks and presentations. 😊
+
+See you on the decentralised web! ✊

src/_blog/ecosystemcontent.md

@@ -0,0 +1,14 @@
+---
+title: Ecosystem content
+type: Ecosystem content
+sitemap:
+  exclude: true
+data:
+- title: 'libp2p at IPFS þing 2023 Recap'
+  date: 2023-05-11
+  publish_date:
+  card_image: /blog-post-placeholder.png
+  path: https://blog.libp2p.io/2023-libp2p-IPFS-Thing-recap/
+  tags:
+    - libp2p
+---