adam/ipfs-blog
1
0
Fork 0
mirror of https://github.com/ipfs/ipfs-blog.git synced 2026-08-07 09:22:44 +02:00
Code Issues Packages Projects Releases Wiki Activity

Update src/_blog/2023-09-amino-refactoring.md

Browse Source 
Co-authored-by: Daniel Norman <1992255+2color@users.noreply.github.com>
This commit is contained in:
Yiannis Psaras
2023-09-26 19:20:55 +03:00
committed by GitHub
parent 3e7d4d54c0
commit eb7ec42ae3
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/_blog/2023-09-amino-refactoring.md
+1 -1
View File
@@ -54,7 +54,7 @@ The base premise of `ReprovideSweep` is that **all keys located in the *same key
Given that some large Content Providers are publishing way more CIDs than there are DHT Servers, by the [pigeonhole principle](https://en.wikipedia.org/wiki/Pigeonhole_principle) there must be DHT Servers that are allocated more than one Provider Record, by a particular Content Provider. The primary rationale is to send/re-provide all Provider Records allocated to the same DHT Server *******at once, instead of having to revisit the same server later on, re-establish a connection, and store the provider record*******.
However, because sending multiple Provider Records requires a new RPC causing a breaking change, it isn’t trivial to send all Provider Records exactly *at once.* That said, the most expensive part in a (Re)Provide operation is the DHT walk to discover the right DHT Servers to store the Provider Records on, as well as opening new connections to these peers. Once these peers are known, and a connection is already open, the Content Provider can simply reuse the same connection to send multiple individual `Provide` requests.
However, because sending multiple Provider Records requires a new RPC causing a breaking change, it isn’t trivial to send all Provider Records exactly *at once.* That said, the most expensive part in a (Re)Provide operation is the DHT walk to discover the right DHT Servers to store the Provider Records on, as well as opening new connections to these peers. Once these peers are known, and a connection is already open, the Content Provider can simply reuse the same connection to send multiple individual `Provide` requests, thereby avoiding breaking changes while still reaping performance gains.
The `go-libp2p-kad-dht` DHT implementation must keep track of the CIDs that must be republished every `Interval` (let’s assume that all Provider Records are republished at the same frequency). The Kademlia identifiers of the CIDs to republish must be arranged in a [binary trie](https://github.com/guillaumemichel/py-binary-trie) to allow for faster access. As each Provider Record is replicated on 20 different DHT Servers, 20 DHT Servers in a close locality are expected to store the same Provider Records (this is not 100% accurate, but suffices for our high-level description here - we’ll publish all the details in a subsequent post, when the solution is in production).