Merge branch 'main' into ipfs-chromium-post

src/_blog/2023-05-ipfs-unresponsive-nodes-incident.md

@@ -0,0 +1,118 @@
+---
+title: What happens when half of the network is down?
+description: "The IPFS DHT experienced a serious incident in the beginning of 2023, but users hardly noticed thanks to the power of a decentralized network!"
+author: Yiannis Psaras
+date: 2023-05-08
+permalink: '/2023-ipfs-unresponsive-nodes/'
+header_image: '/2023-05-ipfs-unresponsive-nodes-incident.jpeg'
+tags:
+  - 'dht'
+  - 'decentralization'
+  - 'resource manager'
+  - 'nodes'
+---
+
+It depends on what type of system/network you’re running. In 90% of networks, or networked systems, this is a grand-scale disaster. Alerts are popping up everywhere, engineers go far beyond “day-time work” to get things back to normal, customers are panicking and potentially leaving the platform and the customer care lines are on fire. Half of the network is a large fraction, but I would bet that the same would happen even when 10% or 20% of the network experiences an outage.
+
+It’s not like that when you run your services on a decentralized, distributed P2P network, such as IPFS! At the beginning of 2023, a critical component of the IPFS network, namely the public IPFS DHT, experienced a large-scale incident. *During this incident, [60% of the IPFS DHT Server nodes became unresponsive](https://github.com/protocol/network-measurements/blob/master/reports/2023/calendar-week-04/ipfs/plots/crawl-unresponsive.png).* Interestingly, **no content became unreachable and almost nothing in the network looked like the majority of the network was basically down**. We did observe a significant increase in the content routing/resolution latency (in the order of 25% initially), but this in no way reflected the scale of the event.
+
+In this blog post, we’ll go through the timeline of the event from “Detection” to “Root Cause Analysis” and give details about the engineering team’s response. A summarizing talk on the content of this blog post was given at [IPFS Thing 2023](https://2023.ipfs-thing.io/) and can be found [on YouTube](https://youtu.be/8cGEjdCfm14).
+
+## ❗️Detection: we've got a problem!
+
+> At the beginning of 2023, a critical component of the IPFS network, namely the public IPFS DHT, experienced a large-scale malfunction. *During this situation, [60% of the IPFS DHT Server nodes became unresponsive](https://github.com/protocol/network-measurements/blob/master/reports/2023/calendar-week-04/ipfs/plots/crawl-unresponsive.png).*
+> 
+
+Unresponsive here means that nodes would seem to be online, they would accept connections from other nodes, but they wouldn’t reply to requests. Basically, when a node would try to write to one of the unresponsive nodes, the unresponsive node would terminate the connection immediately.
+
+Given that these nodes seemed to be functional, they occupied several places in other nodes’ routing tables, when in fact they shouldn’t have.
+
+The problem came down to a misconfiguration of the go-libp2p resource manager - a new feature that shipped with `kubo-v0.17`. The problematic configuration which was applied manually (i.e. was not based on the default values of `kubo-v0.17`) was set to such values that any attempt to interact with the nodes would be flagged as a resource exhaustion event and would trigger the corresponding “defense” mechanism. In practice, this materialized as a connection tear-down. It is worth noting that `kubo` is the most prevalent IPFS implementation using the public IPFS DHT with ~80% of nodes in the DHT being `kubo` nodes (see most recent [stats](https://github.com/protocol/network-measurements/tree/master/reports/2023/calendar-week-17/ipfs#agent-version-analysis)).
+
+Content was still findable through kubo, so no alarms were raised. However, some of our research teams observed unusual error messages:
+
+```go
+> Application error 0x0 (remote): conn-22188077: system: cannot reserve inbound
+connection: resource limit exceeded
+```
+
+Since PUT and GET operations were completing successfully, the error didn’t seem like one that would trigger widespread panic. We were seeing slower performance than normal and had been investigating whether [recent changes with Hydra boosters](https://discuss.ipfs.tech/t/dht-hydra-peers-dialling-down-non-bridging-functionality-on-2022-12-01/15567) had bigger impacts than we were expecting. It was at this time that we had a physical meeting of our engineering teams and one of the items on the agenda was to figure out where this error was coming from.
+
+## ❓ Diagnosis: what was happening?
+
+We quickly realized that [there was a resource manager issue where the remote node was hitting a limit and closing the connection](https://github.com/libp2p/go-libp2p/issues/1928). After looking into the details of the resource manager and the error itself (i.e., `cannot reserve **in**bound connection`), we realized that the root cause of the issue was related to the remote node. It turned out that the resource manager was manually misconfigured by a very large percentage of nodes to values that were not in the default configuration by the “vanilla” version of the resource manager that shipped with `kubo-v0.17`.
+
+As mentioned earlier, the GET and PUT operations were completing successfully, so our next step was to identify the scale of the problem. Our main goals were to figure out:
+
+- what percentage of nodes in the network were affected
+- if there was a performance penalty in either the PUT or the GET operation, or both
+
+Through a combination of crawling the network and attempting connections to all ~50k DHT Server nodes (i.e., those that store and serve provider records and content), we found that close to 60% of the network had been affected by the misconfiguration. Clearly this was a very large percentage of the network, which made it urgent to look into the performance impact. We followed the below methodology:
+
+1. We wanted to figure out which buckets in the nodes’ routing tables did the affected nodes occupy. We found that they occupied the higher buckets of the nodes’ routing tables, which meant that most likely PUTs would get slower, but GETs should not be affected too much. This is because the DHT lookup from the GET operation terminates when it hits *one* of the 20 closest peers to the target key, while the PUT operation terminates when it has found *all* the 20 closest peers. Since a significant portion of the network was unresponsive, the PUT operation hit at least one unresponsive node, but the GET operation had good chances of finding at least one responsive node within the 20 closest.
+    
+    ![output.png](../assets/ipfs-unresponsive-nodes-incident/output.png)
+          <br>
+2. After further investigation and given the very large percentage of nodes that were affected by the resource manager misconfiguration, we started looking into the impact of the incident to the GET performance.
+         
+    A GET request that hits one of the affected, unresponsive nodes would get the connection shut down by the remote, but would get stuck there until it timed out, at which point it would re-issue the request to another peer. The relatively high concurrency factor of the IPFS DHT (`alpha = 10`) helps in this case, as it means that for any given request up to 10 concurrent requests can be in flight. This helps a lot even with a high percentage of unresponsive nodes as it means that at least one of the 10 peers contacted will respond.
+          <br>
+    > This is because the DHT lookup from the GET operation terminates when it hits one of the 20 closest peers to the target key, when the PUT operation terminates when it has found all the 20 closest peers.
+    >
+
+
+    <br>In the meantime, we estimated that a non negligible number of GET requests were hitting at least one unresponsive node during the lookup process. This event results in a timeout and significantly increases the request latency. There is a high probability that an unresponsive node is encountered during the last hops of the DHT walk because unresponsive peers are mostly present in higher buckets as the above figure shows.
+          <br>
+3. To quantify the impact, we crawled the network and gathered the PeerIDs of unresponsive nodes. We set up six kubo nodes in several locations around the globe and attempted to: i) publish content (PUT), and, ii) retrieve content (GET) for two cases: 1) when interacting with all nodes in the network, and, 2) when ignoring all responses from the unresponsive peers, whose PeerIDs we knew and were cross-checking with in real time.
+
+    - The results we found were as follows:
+      - The PUT operation was slowed down by approximately 10%
+                <br>
+        ![output2.png](../assets/ipfs-unresponsive-nodes-incident/output2.png)
+          <br>
+      - The GET operation was also disrupted (in contrast to our initial assumption) and was slowed down by approximately 15%, at times reaching closer to 20%.
+                <br>
+         ![output.png](../assets/ipfs-unresponsive-nodes-incident/output_1.png)
+          <br>
+4. We also experimented with even higher concurrency factors, in particular with `alpha = 20`, as a potential mitigation strategy. We repeated the same experiment with one extra set of runs: the case where we interact with all nodes in the network (i.e., we do not ignore unresponsive peers), but have higher concurrency factor.
+
+   <br>We found that the performance increased and went back to pre-incident levels. However, it was decided *not* to go down this path, as the increased concurrency factor would: i) increase significantly the overhead/traffic in the DHT network, and, ii) stick with nodes that do not upgrade later on (when the incident is resolved) giving a clear advantage advantage to those nodes.
+    
+
+## 🚑 Mitigation: how we stopped the bleeding.
+
+The team’s immediate focus became:
+
+1. [Adding/updating documentation on Kubo’s resource manager integration](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md)
+2. Triaging and responding to user questions/issues ([example](https://github.com/ipfs/kubo/issues/9432))
+3. Preparing a new kubo release (`v0.18.1`), where the default settings for the resource manager were set to more appropriate values. This reduced the likelihood that someone would need to adjust the resource manager configuration manually, thus avoiding the configuration “footguns”.    
+4. Encouraging as many nodes as possible to upgrade through public forums and direct relationships with known larger scale operators.
+
+In parallel, we kept monitoring the situation by instrumenting a PUT and GET measurement experiment that was running since before the `kubo-v0.18.1` update, when the affected nodes started updating gradually.
+
+`kubo-v0.18.1` was [released on the 2023-01-30](https://github.com/ipfs/kubo/releases/tag/v0.18.1) and within the first 10 days, more than 8.5k nodes updated to this release. Our monitoring software allowed us to have an accurate view of the state of the network and observed that the update to the new kubo release brought significant performance increase for the GET operation - more than 40% at the 95th percentile on a sample of ~2k requests, compared to the situation before the `kubo-v0.18.1` release.
+
+![output.png](../assets/ipfs-unresponsive-nodes-incident/output_2.png)
+
+We also monitored the situation compared to the pre-incident performance by running the experiment where we ignored the set of PeerIDs that were identified as affected by the misconfiguration. As a sample from more than 20k GET operations, in the figure below we show that the impact has reduced to ~5% (mid-February 2023).
+
+![output.png](../assets/ipfs-unresponsive-nodes-incident/output_3.png)
+
+## 🔧 Addressing the Root Cause 
+
+Our immediate actions managed to stop the bleeding and bring the network back to normal quickly. However, it was clear that we had to implement longer term fixes to protect the nodes’ routing tables from unresponsive peers and to avoid inadvertently making nodes unresponsive. Specifically this translated to:
+
+1. Revamping the Kubo resource manager UX to further reduce the likelihood of catastrophic misconfiguration. This was completed in [Kubo 0.19](https://github.com/ipfs/kubo/releases/tag/v0.19.0#improving-the-libp2p-resource-management-integration).
+2. Only adding peers to the routing table that are responsive requests [during the routing table refresh](https://github.com/libp2p/go-libp2p-kad-dht/pull/810) (done) and [upon adding a node to the routing table](https://github.com/libp2p/go-libp2p-kad-dht/issues/811) (in progress - targeting [Kubo 0.21 in May](https://github.com/ipfs/kubo/issues/9814)).
+
+## 📖 Lessons Learned
+
+In the days since, we have come away from this experience with several important learnings:
+
+🗒️ Significant fundamental changes to the codebase (such as retroactively adding resource accounting) is ripe for disruption. This increases the necessity for documentation, announcements, and clear recommendations to node operators.  
+
+⏱️ Monitoring software should always be in place to help identify such events from the start.
+
+📣 It is challenging to monitor and apply changes directly to the software that runs on nodes of a decentralized network. Well-established communication channels go a long way and help the engineering teams communicate directly with the community. In IPFS, we use a variety of channels including the Discord Server [[invite link](https://discord.gg/ipfs)], Filecoin Slack [[invite link](https://filecoin.io/slack)] (mostly in `#engres-ip-stewards` channel), the [Discourse discussion forum](https://discuss.ipfs.tech/), and the [blog](https://blog.ipfs.tech/).
+
+🚀 Last, but certainly not least, the decentralized, P2P nature of IPFS kept the network running with all important operations completing successfully (albeit slower than normal). It is exactly because of the structure of the network that there are no single points of failure and performance is not catastrophically disrupted even when more than half of the network nodes are essentially unresponsive.

src/_blog/2023-thing-web-track.md

@@ -0,0 +1,124 @@
+---
+title: 'Recap: IPFS on the Web (þing 2023)'
+description: 'Track recap with links and serious analysis for the IPFS on the Web track at IPFS þing 2023'
+author: Dietrich Ayala
+date: 2023-05-10
+permalink: '/2023-ipfs-thing-web-track/'
+header_image: '/ipfs-thing-2023-recap/ipfs-on-the-web-featured-image-2.jpg'
+tags:
+  - 'thing'
+  - 'þing'
+  - 'event'
+  - 'recap'
+  - 'track'
+  - 'web'
+---
+
+The world wide web is both the biggest deployment vector and least controllable surface for IPFS. There are opportunities and challenges with bringing IPFS support to rendering engines, browsers, gateway-served content, web apps, and browser extensions. It is these unique dynamics that motivated us to organize a dedicated content track for them at the annual gathering of IPFS implementers known as IPFS Thing.
+
+To catch you up to speed, the [*Browsers and the Web Platform* track at IPFS Thing 2022](https://2022.ipfs-thing.io/schedule/#Browsers-and-The-Web-Platform) was only a half day long with a small group of people. It had browser and gateway progress updates, some alternate visions of how a content-addressed web could work on desktop and mobile, and also some straight-up "stuff is hard still" talks. You can listen to [all of these talks](https://www.youtube.com/watch?v=_DGVa2CJjIc&list=PLuhRWgmPaHtTsL76nt_A6CPDe6lW7l6Sz) on YouTube.
+
+A few months later at [IPFS *Camp* 2022](https://2022.ipfs.camp/#Browsers-Platforms) we had a similar track with many more people and the tone changed a bit — we saw more working code, and even features shipped in products that were represented, and we covered platforms outside of web, like native mobile and space. You can watch the [full playlist of videos](https://www.youtube.com/watch?v=HhCHvuP5IJo&list=PLuhRWgmPaHtQohNbRjFJDS70WoElZ8ep5) on YouTube as well.
+
+This brings us to IPFS *Thing* 2023 that occurred last month. I had the privilege of broadening the lens even further than we experienced at the previous two gatherings. We examined the opportunities, challenges, products, protocols, and experiments in the intersection of these two distinct paradigms of HTTP and IPFS. This area of the IPFS ecosystem is changing so rapidly that [HTTP Gateways to the IPFS network](https://www.youtube.com/watch?v=p89i9_AskIw&list=PLuhRWgmPaHtTapMgLW7rRh92Tk8u7wip5) had a whole track to itself this year. This gave the Web track more room to move, so we were able to cover everything from naming systems to publishing pipelines and JS toolkits and more.
+
+In the rest of this blog post you'll find highlights, links, and commentary from the track lead (that's me, Dietrich) on why these talks were selected and why I think they're helping make a better web for us all.
+
+You can find the full video playlist for the IPFS on the Web track [here](https://www.youtube.com/watch?v=dn8PssXkRbY&list=PLuhRWgmPaHtQ-TO65P62tqfUM85HCIqSj), and I'll link each below as well.
+
+## IPFS on the Web in 2023 (so far) - Dietrich Ayala
+
+[Dietrich](https://metafluff.com/) gave a short overview of various initiatives and collaboration projects of the Browsers, Platforms & Standards team at Protocol Labs. It was a peek into the latest IPFS features in Brave Browser, early work into Chromium native support for IPFS, and various other work those weirdos are pushing forward.
+
+@[youtube](dn8PssXkRbY)
+
+## What Is The Web? - Robin Berjon
+
+Good morning. Have you had a coffee yet? Ok great because you're going to need it for this talk. [Robin](https://berjon.com/) sets the perspective for the day by asking us one of the most difficult questions: "What is the web, actually?" It's big. It's special. We complain about it. But we need to have language to describe what it is in order to talk about how it could grow and change. Warning: This talk begins at a point in history over 100 years ago!
+
+@[youtube](s878bm15mrk)
+
+## A better web: secure, private, p2p apps with user-owned data and identity - Ian Preston
+
+[Peergos](https://peergos.org/) has been building *your* private space online for half a decade, and it shows: [Ian](https://peergos.org/about#ian_) and team have built a mostly exilfiltration-proof application platform on IPFS which works in the browsers of today. The idea of web content that can't phone home might make you say "hmmm", but it could just be the antidote to the surveillance-is-required-to-pay-the-bills version of the web we have today.
+
+@[youtube](mSElk2jcFqY)
+
+## WNFS: Versioned and Encrypted Data on IPFS - Philipp Krüger
+
+It's the web. It's p2p. It's files. It's private. It's WNFS! [Philipp](https://irreactive.com/) walks us through how the WebNative Filesystem works, and how it works in browsers specifically. It's not easy, but none of us signed up for easy. That being said, you *can* sign up to join the [WNFS Working Group](https://github.com/wnfs-wg) today after watching this talk.
+
+@[youtube](LBMyRp4Ywew)
+
+## Content Based Addressing and the Web Security Model - Fabrice Desré
+
+Speaking of hard... have you ever decided that the problem you'd like to fix in the world is Google and Apple's stranglehold on our daily digital lives? That's what [Fabrice](https://github.com/fabricedesre) does with [Capyloon](https://capyloon.org/), a complete web-based mobile operating system. When you control the OS, you ~~control the world~~ can do veeeerrrry interesting things. Fabrice gave us a deep dive into the [origin security model](https://www.rfc-editor.org/rfc/rfc6454) today and how radically different it can be in a content-addressed world.
+
+@[youtube](H_1JVGDnctI)
+
+## Hello Helia - achingbrain
+
+Bye Felicia. Hello Helia. Thanks [achingbrain](https://github.com/achingbrain). Welcome to a new way to IPFS in JavaScript, on the web, or on the server... finally with nice things like DHT support. It should've been called banana. But we like it anyway.
+
+@[youtube](T_FlhkLSgH8)
+
+## JavaScript performance - how to wring the most out of your Helia deployment - achingbrain
+
+Hey it's [achingbrain](https://github.com/achingbrain) again, this time with a deep dive into Helia performance and optimizing for the environment you're deploying to. JavaScript is the *fastest* language for the environments it lives in which other languages can't even exist, so take that.
+
+@[youtube](zPeLYosZ3Ak)
+
+## Connecting everything, everywhere, all at once with libp2p - Prithvi Shahi
+
+First, the person who gives this talk is not Prithvi, it's Max. Second, Prithvi is a genius for thinking up a talk like this. Please enjoy all the transports everywhere all of the time.
+
+@[youtube](zPeLYosZ3Ak)
+
+## The Incredible Benefits of libp2p + HTTP - Marten Seemann & Marco Munizaga
+
+In the ageless words of Gandalf, the headmaster of Hogwarts: "be like water". Or something like that. Anyways, if you're familiar with the challenge of writing code for the web that has to pretend that it's not actually stuck in a web browser tab, but instead is actually connected to a global transport-agnostic peer-to-peer network, then you'll understand how important it is to make friends with your environment and use the *!#? out of what it gives you. This talk from Marten and Marco shows you how the changing landscape of the network layer of the web platform is allowing libp2p to operate unfettered while still stuck in a tab in a window in a browser on your computer on earth.
+
+@[youtube](Ixyo1G2tJZE)
+
+## The Name Name Service - Blaine Cook
+
+NNS.. NNS.. NNS... goes the beat. Now that you're bouncing your head, follow along as Blaine Cook shares his vision of how we solve one of the three hardest problems in computer science: how to make the perfect Hollandaise. Er, no that wasn't it. NAMING! Yes, that was it. The Name Name Service is a breathtakingly simple approach to flexible, veriable, integrate-able, old-world-compatible, and human-readable names for our digital things. Thank you Blaine.
+
+@[youtube](CHiCEd36KtI)
+
+## Building decentralized websites on IPFS - Ryan Shahine
+
+Decentralization is cool but it's so hard and you have to be super technical to even... wait, what? I can just... drag and drop? What I see is what I get?! With Portrait, yes. Ryan Shahine shares Portrait's slick and simple site builder for publishing your sites to IPFS. No-code sites for non-technical creators is such a wonderful thing to behold.
+
+@[youtube](TeFAHmzvIdg)
+
+## ODD.js, a technical overview - icidasset
+
+Oddly enough, DID you know UCAN build decentralized applications with that WNFS stuff we heard about earlier? In this preview of the final emergent form of a bunch of Fission's tools coming together like Voltron, you'll meet Odd.js - a toolkit for building applications that has all of the core bits you need, from identity to naming to storage to security. The only odd thing is that we didn't have this yet.
+
+@[youtube](ByQbY3lNAck)
+
+## IPFS native frontend development using Importmaps - Dilip Shukla
+
+Imagine if your CDN was a massive cooperative global distributed network that 1) wasn't a single company, and 2) pushed alllll the way up into your client-side build tooling, making your pages immediately available. WHAT YOUR DOM WAS ACTUALLY A DAG... ok maybe that's too far, but what Lagom is doing is finding exactly what the right balance is. In the future we'll look back and wonder why we didn't do this from the begining...
+
+@[youtube](4HY_7DxScMo)
+
+## Explorations into Decentralized Publishing - David Justice
+
+The Browsers, Platforms and Standards team wants to share our thoughts early and often. And we want to do it on IPFS. There are all kinds of ways to do it, but each have their trade-offs... but it's not clear what those are until you go make a bunch of mistakes. So we're going to make them for you. David Justice is working on approaches for building our team blog with some friends at Trigram and shares what the first stab at it looks like.
+
+@[youtube](fn5QNvRXMIo)
+
+## Thank you!
+
+Thanks to all the speakers for the day and also to the rad people who joined and asked great questions.
+
+IPFS Camp is already in planning for early November... in BANGALORE INDIA! So block your calendars and start thinking about the web you want to see exist, so you can submit a talk there. 😄
+
+Until then, come hang out in our superbridged megachannel:
+
+* #browsers-and-platforms on Filecoin Slack ([join](https://filecoin.io/slack))
+* #browsers-and-standards on Element/Matrix ([join](https://matrix.to/#/#browsers-and-standards:ipfs.io))
+* #browsers-and-standards on IPFS Discord ([join](https://discord.gg/ipfs))
+

src/_blog/newscoverage.md

@@ -4,6 +4,14 @@ type: News coverage
 sitemap:
   exclude: true
 data:
+- title: Brave announces automatic NFT backups and enhanced IPFS/Filecoin support in Brave Wallet
+  date: 2023-05-02
+  publish_date: 
+  path: https://brave.com/nft-pinning/
+  tags:
+  - NFTs
+  - Brave
+  - pinning
 - title: WebTransport in libp2p
   date: 2022-12-19
   publish_date: 

src/_blog/releasenotes.md

@@ -1,5 +1,12 @@
 ---
 data:
+    - title: 'Just released: Kubo 0.20.0!'
+      date: "2023-05-09"
+      publish_date: null
+      path: https://github.com/ipfs/kubo/releases/tag/v0.20.0
+      tags:
+        - go-ipfs
+        - kubo
     - title: 'Just released: Kubo 0.19.2!'
       date: "2023-05-03"
       publish_date: null

src/_blog/welcome-to-ipfs-news-193.md

@@ -0,0 +1,78 @@
+---
+title: Welcome to IPFS News 193!
+description: Featuring Bluesky, a recap of IPFS Thing 2023, Brave's enhanced IPFS support, content blocking in Kubo, and much more!
+author: ''
+date: 2023-05-09
+permalink: "/newsletter-193"
+translationKey: ''
+header_image: "/ipfsnews.png"
+tags:
+- newsletter
+---
+
+A lot has happened since the previous newsletter over a month ago. [IPFS Thing took place in Brussels](https://blog.ipfs.tech/2023-ipfs-thing-recap/), we created a [Bluesky](https://blog.ipfs.tech/2023-ipfs-on-bluesky/) account, [Brave released automatic NFT backups to IPFS](https://brave.com/nft-pinning/), [content blocking can now be enabled in Kubo](https://blog.ipfs.tech/2023-content-blocking-for-the-ipfs-stack/), plus so much more! Read on to catch up with what’s happened in the ecosystem over the last few weeks.
+
+## **Recap: IPFS Thing 2023 🔄**
+
+The IPFS implementers community recently gathered in Brussels, Belgium for the second year of [IPFS þing](https://2023.ipfs-thing.io/), an annual gathering dedicated to advancing IPFS implementation. With 12 tracks and over 75 talks, demos, and sessions, the 5-day summit that occurred in April 2023 was a showcase of recent advances across IPFS, a forum for sharing needs from the protocol, and an opportunity to chart new directions for the future of IPFS.
+
+[Read the recap on the blog for photos, videos, and summaries!](https://blog.ipfs.tech/2023-ipfs-thing-recap/)
+
+## **Brand New on IPFS ✨**
+
+**[IPFS is now on Bluesky!](https://blog.ipfs.tech/2023-ipfs-on-bluesky/)**
+
+* We’re excited to share that IPFS now has an official presence on [Bluesky](https://blueskyweb.xyz/)! We chose[ ](https://twitter.com/bluesky)Bluesky because it shares many of the same values and goals that the IPFS ecosystem has. Additionally, they actively utilize IPLD and content addressing. [Read more about it](https://blog.ipfs.tech/2023-ipfs-on-bluesky/)!
+
+**[Content Blocking for the IPFS stack is finally here!](https://blog.ipfs.tech/2023-content-blocking-for-the-ipfs-stack/)**
+
+* Traditionally, content blocking within the IPFS ecosystem has been performed only at the IPFS gateway level and directly in Nginx, using something called the "Badbits denylist" — but now it can be enabled in Kubo & other tools in the IPFS stack too! [Check out the blog post for more info.](https://blog.ipfs.tech/2023-content-blocking-for-the-ipfs-stack/)
+
+**[What happens when half of the network is down?](https://blog.ipfs.tech/2023-ipfs-unresponsive-nodes/)**
+
+* The IPFS DHT experienced a serious incident in the beginning of 2023, but users hardly noticed thanks to the power of a decentralized network. [Read all about it in a new incident report!](https://blog.ipfs.tech/2023-ipfs-unresponsive-nodes/)
+
+**[IPFS Principles](https://specs.ipfs.tech/architecture/principles/)**
+
+* As mentioned above, IPFS recently joined a new social media network called [Bluesky](https://blueskyweb.xyz/) because it shares many of the same values that the IPFS ecosystem has. But what are those values exactly? You can [read all about IPFS Principles in a new specs doc](https://specs.ipfs.tech/architecture/principles/) edited by[ Robin Berjon](https://twitter.com/robinberjon).
+
+**[Kubo 0.20.0](https://github.com/ipfs/kubo/releases/tag/v0.20.0)**
+
+* This update includes:
+    * Switch to `boxo/gateway` library
+    * Improved testing
+    * Trace Context support
+    * Removed legacy features
+
+**[Kubo 0.19.2](https://github.com/ipfs/kubo/releases/tag/v0.19.2)**
+
+**[Kubo 0.19.1](https://github.com/ipfs/kubo/releases/tag/v0.19.1)**
+
+
+## **Around the Ecosystem 🌎**
+
+* [Brave announces automatic NFT backups and enhanced Filecoin support in Brave Wallet](https://brave.com/nft-pinning/)
+    * We're excited to share that the latest version of Brave’s web browser introduces automatic NFT backups to IPFS. Brave Wallet users can avoid the permanent loss of NFT metadata and gain peace of mind thanks to this new feature. [Check it out!](https://brave.com/nft-pinning/)
+* [Introducing Lassie - a retrieval client for IPFS and Filecoin](https://blog.ipfs.tech/2023-introducing-lassie/)
+    * Lassie makes it easy to fetch your data from both the IPFS and Filecoin Network - it will find and fetch content over the best retrieval protocols available. [Read more about it on the IPFS blog](https://blog.ipfs.tech/2023-introducing-lassie/)!
+* [IPFS Implementations: It’s Definitely A Thing](https://blog.ipfs.tech/2023-03-implementation-principles/)
+    * In a new blog post,[ Robin Berjon](https://twitter.com/robinberjon) talks about how the world of IPFS implementations has diversified greatly over the past 9 months: “Springtime in the distributed hemisphere and we are frolicking across fields of tantalizing IPFS flowers.” [Read the entire blog post](https://blog.ipfs.tech/2023-03-implementation-principles/)!
+* [IPFS Open Metaverse Base Camp Accelerator](https://outlierventures.io/ipfs-open-metaverse-base-camp/)
+    * The latest cohort kicked-off on May 8, 2023. Co-delivered by Protocol Labs and Outlier Ventures, the program will run for 12 weeks and provide the teams in the cohort with the knowledge, networks, and capital they need to succeed as startups in Web3. Teams will pitch their products and services at Demo Day in August. [Visit the website to learn more!](https://outlierventures.io/ipfs-open-metaverse-base-camp/)
+
+
+## **IPFS Thing 2023 on YouTube 📺**
+
+All of the talks and presentations from this year’s gathering of the IPFS implementers community are now available on YouTube. If you weren’t able to attend, now is the perfect chance to catch up! Below you will find links to playlists for each content track:
+
+* [Opening & Keynotes](https://www.youtube.com/playlist?list=PLuhRWgmPaHtRnO5G2EF0RxYebcQzLDf5F)
+* [Community & Governance](https://www.youtube.com/playlist?list=PLuhRWgmPaHtTIFbOVO5YfXkoFg6wIGbBN)
+* [Integrating IPFS](https://www.youtube.com/playlist?list=PLuhRWgmPaHtTI0MS6ZjSJjBxZp7rcjSS_)
+* [Decentralized Compute & AI](https://www.youtube.com/playlist?list=PLuhRWgmPaHtQ_lKtbTR-vIW1LYuTjcaPw)
+* [HTTP Gateways](https://www.youtube.com/playlist?list=PLuhRWgmPaHtTapMgLW7rRh92Tk8u7wip5)
+* [Content Routing](https://www.youtube.com/playlist?list=PLuhRWgmPaHtRBWV3SvInC5ATS8aKV3lsW)
+* [Interplanetary Databases](https://www.youtube.com/playlist?list=PLuhRWgmPaHtTO8hr2CYiJPTSe7wybW_op)
+* [IPFS on the Web](https://www.youtube.com/playlist?list=PLuhRWgmPaHtQ-TO65P62tqfUM85HCIqSj)
+* [Data Transfer](https://www.youtube.com/playlist?list=PLuhRWgmPaHtS6WBDGK8oxcBHA6ILKatVk)
+* [IPFS Deployments & Operators](https://www.youtube.com/playlist?list=PLuhRWgmPaHtTYOY5l8nehP_Vt6Ek-svrp)
+* [Measuring IPFS](https://www.youtube.com/playlist?list=PLuhRWgmPaHtQkkbiq-PbIkt9_S2NjJz6x)