adam/ipfs-blog
1
0
Fork 0
mirror of https://github.com/ipfs/ipfs-blog.git synced 2026-07-05 01:11:02 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #123 from ipfs/staging

Browse Source 
Update fixed-width blog posts and video size issues
This commit is contained in:
Jessica Schilling
2021-04-08 15:51:12 -06:00
committed by GitHub
parent 6fa5aa840d 1e41c25541
commit 0eb8549673
6 changed files with 173 additions and 428 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/_blog/028-go-libp2p-6.0.0.md
+59 -183
View File
@@ -1,107 +1,66 @@
---
date: 2018-06-15
permalink: /39-go-libp2p-6-0-0/
permalink: "/39-go-libp2p-6-0-0/"
title: go-libp2p 6.0.0 released
description:
description:
author: Steven Allen
tags:
- 'libp2p'
---
- libp2p
---
# 🔦 go-libp2p 6.0.0 release notes
We're pleased to announce go-libp2p 6.0.0. This release includes a massive
refactor of go-libp2p that paves the way for new transports such as QUIC.
Unfortunately, as it is broad sweeping, there are some breaking changes,
_especially_ for maintainers of custom transports.
We're pleased to announce go-libp2p 6.0.0. This release includes a massive refactor of go-libp2p that paves the way for new transports such as QUIC. Unfortunately, as it is broad sweeping, there are some breaking changes, _especially_ for maintainers of custom transports.
Below, we cover the changes you'll likely care about. For convenience, we've
broken this into a section for users and transport authors/maintainers. However,
transport maintainers should really read both sections.
Below, we cover the changes you'll likely care about. For convenience, we've broken this into a section for users and transport authors/maintainers. However, transport maintainers should really read both sections.
## 💁 For Users
Libp2p users should be aware of a few major changes.
- Guarantees and performance concerning connect/disconnect notification
processing have improved.
- Handling of half-closed streams has changed (READ THIS SECTION).
- Some constructors and method signatures have changed slightly.
* Guarantees and performance concerning connect/disconnect notification processing have improved.
* Handling of half-closed streams has changed (READ THIS SECTION).
* Some constructors and method signatures have changed slightly.
### Dialing And Source Addresses
We've improved the logic that selects the source address when dialing. In the
past, you may have run into an issue where you couldn't dial non-local nodes
when listening on 127.0.0.1 as opposed to 0.0.0.0. This happened because
go-libp2p would randomly pick the source address from the set of addresses on
which the node was listening. We did this to ensure that the other end could
dial us back at the same address. Unfortunately, one can't use 127.0.0.1 as a
source address when dialing any non-local address so outbound dials failed.
We've improved the logic that selects the source address when dialing. In the past, you may have run into an issue where you couldn't dial non-local nodes when listening on 127.0.0.1 as opposed to 0.0.0.0. This happened because go-libp2p would randomly pick the source address from the set of addresses on which the node was listening. We did this to ensure that the other end could dial us back at the same address. Unfortunately, one can't use 127.0.0.1 as a source address when dialing any non-local address so outbound dials failed.
go-libp2p now tries to be smarter about this and avoids picking source addresses
that have no route to the destination address.
go-libp2p now tries to be smarter about this and avoids picking source addresses that have no route to the destination address.
### Bandwidth Metrics
To start out on an unhappy note, bandwidth metrics are now less accurate. In the
past, transports returned "minimal" connections (e.g., a TCP connection) so we
could wrap these transport connections in "metrics" connections that counted
every byte sent and received.
To start out on an unhappy note, bandwidth metrics are now less accurate. In the past, transports returned "minimal" connections (e.g., a TCP connection) so we could wrap these transport connections in "metrics" connections that counted every byte sent and received.
Unfortunately, now that we've moved encryption and multiplexing down into the
transport layer, the connection we're wrapping has significantly more
under-the-covers overhead.
Unfortunately, now that we've moved encryption and multiplexing down into the transport layer, the connection we're wrapping has significantly more under-the-covers overhead.
However, we do hope to improve this and get even _better_ bandwidth metrics than
we did before. See [libp2p/go-libp2p-transport#31][] for details.
[libp2p/go-libp2p-transport#31]: https://github.com/libp2p/go-libp2p-transport/issues/31
However, we do hope to improve this and get even _better_ bandwidth metrics than we did before. See [libp2p/go-libp2p-transport#31](https://github.com/libp2p/go-libp2p-transport/issues/31) for details.
### Notifications
This release brings performance improvements and simple ordering guarantees for connect/disconnect events:
1. For any given connection/stream, libp2p will wait for all connect/open event
handlers to finish exit before triggering a disconnect/close event for the
connection/stream.
2. When a user calls the `Close` (or `Reset`) method on a connection or stream,
go-libp2p will process the close event asynchronously (i.e., not block the
call to `Close`). Otherwise, a call to `Close` from within a connect event
handler would deadlock.
1. For any given connection/stream, libp2p will wait for all connect/open event handlers to finish exit before triggering a disconnect/close event for the connection/stream.
2. When a user calls the `Close` (or `Reset`) method on a connection or stream, go-libp2p will process the close event asynchronously (i.e., not block the call to `Close`). Otherwise, a call to `Close` from within a connect event handler would deadlock.
3. Unless otherwise noted, events will be handled in parallel.
What does this mean for end users? Well:
1. Reference counting connections to a peer using connect/disconnect events
should "just work" and should never go negative.
2. Under heavy connect/disconnect loads, connecting to new peers should be
faster (usually).
1. Reference counting connections to a peer using connect/disconnect events should "just work" and should never go negative.
2. Under heavy connect/disconnect loads, connecting to new peers should be faster (usually).
For those interested in the history of this issue, ...
In the past, (dis)connect and stream open/close notifications have been a bit of
a pain point. For a long time, they were fired off in parallel and one could, for
example, process a disconnect notification before a connect notification (we had
to support _negative_ ref-counts in several places to account for this).
In the past, (dis)connect and stream open/close notifications have been a bit of a pain point. For a long time, they were fired off in parallel and one could, for example, process a disconnect notification before a connect notification (we had to support _negative_ ref-counts in several places to account for this).
After no end of trouble, we finally "fixed" this by synchronizing notification
delivery. We still delivered notifications to all notifiees in parallel, we just
processed the events in series.
After no end of trouble, we finally "fixed" this by synchronizing notification delivery. We still delivered notifications to all notifiees in parallel, we just processed the events in series.
Unfortunately, under heavy connect/disconnect load, new connections could easily
get stuck on open behind a queue of connect events all being handled in series.
In theory, these events should have been handled quickly but in practice, it's
very hard to avoid locks _entirely_ (bitswap's event handlers were especially
Unfortunately, under heavy connect/disconnect load, new connections could easily get stuck on open behind a queue of connect events all being handled in series. In theory, these events should have been handled quickly but in practice, it's very hard to avoid locks _entirely_ (bitswap's event handlers were especially
problematic).
Worse, this serial delivery guarantee didn't actually provide us with an
_in-order_ delivery guarantee as it was still possible for a disconnect to
happen before we even _started_ to fire the connect event. The situation was
slightly better than before because the events couldn't overlap but still far
from optimal.
Worse, this serial delivery guarantee didn't actually provide us with an _in-order_ delivery guarantee as it was still possible for a disconnect to happen before we even _started_ to fire the connect event. The situation was slightly better than before because the events couldn't overlap but still far from optimal.
However, this has all been resolved now. From now on, you'll never receive a
disconnect event before a connect event.
However, this has all been resolved now. From now on, you'll never receive a disconnect event before a connect event.
### Conn.GetStreams Signature Change
@@ -117,65 +76,39 @@ To:
GetStreams() []Stream
```
Listing the streams on an open connection should never involve IO or do anything
that can fail so we removed this error to improve usability.
Listing the streams on an open connection should never involve IO or do anything that can fail so we removed this error to improve usability.
### Libp2p Constructor
If you're not already doing so, you should be using the `libp2p.New` constructor
to make your libp2p nodes. This release brings quite a few new options to the
libp2p constructor so if it hasn't been flexible enough for you in the past, I
recommend that you try again. A simple example can be found in the
[echo][example:echo] example.
If you're not already doing so, you should be using the `libp2p.New` constructor to make your libp2p nodes. This release brings quite a few new options to the libp2p constructor so if it hasn't been flexible enough for you in the past, I recommend that you try again. A simple example can be found in the [echo](https://github.com/libp2p/go-libp2p/tree/master/examples/echo) example.
Given this work and in an attempt to consolidate all of our configuration logic
in one place, we've removed all default transports from go-libp2p-swarm.
Given this work and in an attempt to consolidate all of our configuration logic in one place, we've removed all default transports from go-libp2p-swarm.
TL;DR: Please use the libp2p constructor.
### Zombie Streams
From this release on, when you're done with a stream, you must either call
`Reset` on it (in case of an error) or close it and read an EOF (or some other
error). Otherwise, libp2p can't determine if the stream is _actually_ closed and
will hang onto it indefinitely.
From this release on, when you're done with a stream, you must either call `Reset` on it (in case of an error) or close it and read an EOF (or some other error). Otherwise, libp2p can't determine if the stream is _actually_ closed and will hang onto it indefinitely.
To make properly closing streams a bit easier, we've added two methods to
[go-libp2p-net][]: `AwaitEOF` and `FullClose`.
To make properly closing streams a bit easier, we've added two methods to [go-libp2p-net](https://github.com/libp2p/go-libp2p-net): `AwaitEOF` and `FullClose`.
- `AwaitEOF(stream)` tries to read a single byte from the stream. If `Read`
returns an EOF, `AwaitEOF` returns success. Otherwise, if `Read` either reads
some data or returns some other error, `AwaitEOF` resets the stream and returns
an error. To avoid waiting indefinitely, `AwaitEOF` resets the stream
unconditionally after 1 minute.
- `FullClose(stream)` is a convenience function that closes the stream and then
* `AwaitEOF(stream)` tries to read a single byte from the stream. If `Read` returns an EOF, `AwaitEOF` returns success. Otherwise, if `Read` either reads some data or returns some other error, `AwaitEOF` resets the stream and returns an error. To avoid waiting indefinitely, `AwaitEOF` resets the stream unconditionally after 1 minute.
* `FullClose(stream)` is a convenience function that closes the stream and then
calls `AwaitEOF` on it.
Like with libp2p notifications, this issue has a bit of history...
In the beginning, libp2p assumed that calling `Close` on a stream would close
the stream for both reading and writing. Unfortunately, _none_ of our stream
multiplexers actually behaved this way. In practice, `Close` always closed the
stream for writing.
In the beginning, libp2p assumed that calling `Close` on a stream would close the stream for both reading and writing. Unfortunately, _none_ of our stream multiplexers actually behaved this way. In practice, `Close` always closed the stream for writing.
After realizing this, we made two changes:
1. We accepted the fact that `Close` only closed the stream for writing.
2. We added a `Reset` method for killing the stream (closing it in both
directions, throwing away any buffered data).
2. We added a `Reset` method for killing the stream (closing it in both directions, throwing away any buffered data).
However, we ran into a bit of a snag because we try to track open streams and
need some way to tell when a stream has been closed. In the past this was easy:
when the user calls `Close` on the stream, stop tracking it. However, now that
`Close` only closes the stream for writing, we still _technically_ needed to
track it until the _other_ end closed the stream as well. Unfortunately, without
actually reading from the stream, we have no way of knowing about this.
Therefore, if the user calls `Close` on a stream and then walks away, we'd have
However, we ran into a bit of a snag because we try to track open streams and need some way to tell when a stream has been closed. In the past this was easy: when the user calls `Close` on the stream, stop tracking it. However, now that `Close` only closes the stream for writing, we still _technically_ needed to track it until the _other_ end closed the stream as well. Unfortunately, without actually reading from the stream, we have no way of knowing about this. Therefore, if the user calls `Close` on a stream and then walks away, we'd have
to hang onto the stream indefinitely.
Our solution was to simply stop tracking streams once they were closed for
writing. This wasn't the _correct_ behavior but it avoided leaking memory in the
common case:
Our solution was to simply stop tracking streams once they were closed for writing. This wasn't the _correct_ behavior but it avoided leaking memory in the common case:
1. The user calls `Close` and drops all references to the stream.
2. The other end calls `Close` without writing any additional data.
@@ -185,125 +118,68 @@ common case:
However, this meant that:
1. The list of "open" streams was technically incomplete.
2. If the other side either failed to call `Close` or tried to send data before
closing, the stream would remain "open" (until the connection was closed).
2. If the other side either failed to call `Close` or tried to send data before closing, the stream would remain "open" (until the connection was closed).
In this release, we've changed this behavior. Now, when you `Close` a stream for
writing, libp2p _continues_ to track it. We only stop tracking it when either:
In this release, we've changed this behavior. Now, when you `Close` a stream for writing, libp2p _continues_ to track it. We only stop tracking it when either:
1. You call `Reset` (throwing away the stream).
2. You finish reading any data off of it and observe either an EOF or an error.
This way, we never "forget" about open streams or leave them in a half-forgotten
state.
This way, we never "forget" about open streams or leave them in a half-forgotten state.
In the future, I'd like to add a `CloseAndForget` method to streams that:
1. Closes the stream (sends an EOF).
2. Tells the swarm to stop tracking the stream.
3. Tells the stream muxer to stop tracking the stream and throw away any data
the other side may send (possibly resetting the stream on unexpected data).
3. Tells the stream muxer to stop tracking the stream and throw away any data the other side may send (possibly resetting the stream on unexpected data).
However:
1. This would likely require modifying our stream muxers which may not be
feasible.
2. Explicitly waiting for an EOF is still the correct thing to do unless you
really don't care if the operation succeeded.
1. This would likely require modifying our stream muxers which may not be feasible.
2. Explicitly waiting for an EOF is still the correct thing to do unless you really don't care if the operation succeeded.
## 📞 For Transport Maintainers
For transport maintainers, quite a bit has changed. Before this change,
transports created simple, unencrypted, stream connections and it was the job of
the libp2p Network (go-libp2p-swarm) to negotiate security, multiplexing, etc.
For transport maintainers, quite a bit has changed. Before this change, transports created simple, unencrypted, stream connections and it was the job of the libp2p Network (go-libp2p-swarm) to negotiate security, multiplexing, etc.
However, when attempting to add support for the QUIC protocol, we realized that
this was going to be a problem: QUIC already handles authentication and
encryption (using TLS1.3) and multiplexing. After much debate, we inverted our
current architecture and made transports responsible for encrypting/multiplexing
their connections (before returning them).
However, when attempting to add support for the QUIC protocol, we realized that this was going to be a problem: QUIC already handles authentication and encryption (using TLS1.3) and multiplexing. After much debate, we inverted our current architecture and made transports responsible for encrypting/multiplexing their connections (before returning them).
To make this palatable, we've also introduced a new ["upgrader"
library][go-libp2p-transport-upgrader] for upgrading go-multiaddr-net
connections/listeners to full libp2p transport connections/listeners. Transports
that don't support encryption/multiplexing out of the box can expect to have an
upgrader passed into the constructor.
To make this palatable, we've also introduced a new ["upgrader" library](https://github.com/libp2p/go-libp2p-transport-upgrader) for upgrading go-multiaddr-net connections/listeners to full libp2p transport connections/listeners. Transports that don't support encryption/multiplexing out of the box can expect to have an upgrader passed into the constructor.
To get a feel for how this new transport system works, take a look at the TCP
and WebSocket transports and the transport interface documentation:
To get a feel for how this new transport system works, take a look at the TCP and WebSocket transports and the transport interface documentation:
- [TCP Transport][go-tcp-transport]
- [WebSocket Transport][go-ws-transport]
- [Transport Interface][doc:go-libp2p-transport]
* [TCP Transport](https://github.com/libp2p/go-tcp-transport)
* [WebSocket Transport](https://github.com/libp2p/go-ws-transport)
* [Transport Interface](https://godoc.org/github.com/libp2p/go-libp2p-transport)
### Deprecated Packages
This release sees the deprecation of a few packages:
- [go-peerstream][] has been deprecated and all functionality has been merged
into [go-libp2p-swarm][]. [go-peerstream][] was written as a general-purpose
(not libp2p specific) listener, connection, and stream manager. However, this
package caused more problems than it solved and was incompatible with the new
transport interface.
- [go-libp2p-interface-conn][] has been deprecated. These interfaces to bridge
the gap between transport-level connections and [go-libp2p-net][] connections
however, now that transport connections are fully multiplexed/encrypted, this
is no longer needed.
- [go-libp2p-conn][] has also been deprecated and most of the functionality has
been moved to [go-libp2p-transport-upgrader][]. This package used to provide
connection "upgrade" logic for upgrading transport-level connections to
[go-libp2p-interface-conn][] connections however, transport-level connections
now provide the required functionality out of the box.
* [go-peerstream](https://github.com/libp2p/go-peerstream) has been deprecated and all functionality has been merged into [go-libp2p-swarm](https://github.com/libp2p/go-libp2p/go-libp2p-swarm). [go-peerstream](https://github.com/libp2p/go-peerstream) was written as a general-purpose (not libp2p specific) listener, connection, and stream manager. However, this package caused more problems than it solved and was incompatible with the new transport interface.
* [go-libp2p-interface-conn](https://github.com/libp2p/go-libp2p-interface-conn) has been deprecated. These interfaces to bridge the gap between transport-level connections and [go-libp2p-net](https://github.com/libp2p/go-libp2p-net) connections however, now that transport connections are fully multiplexed/encrypted, this is no longer needed.
* [go-libp2p-conn](https://github.com/libp2p/go-libp2p-conn) has also been deprecated and most of the functionality has been moved to [go-libp2p-transport-upgrader](https://github.com/libp2p/go-libp2p-transport-upgrader). This package used to provide connection "upgrade" logic for upgrading transport-level connections to [go-libp2p-interface-conn](https://github.com/libp2p/go-libp2p-interface-conn) connections however, transport-level connections now provide the required functionality out of the box.
### Testing
We've moved `GenSwarmNetwork` in [go-libp2p-netutil][] to `GenSwarm` in
[go-libp2p-swarm/testing][] because:
We've moved `GenSwarmNetwork` in [go-libp2p-netutil](https://github.com/libp2p/go-libp2p/go-libp2p-netutil) to `GenSwarm` in [go-libp2p-swarm/testing](https://github.com/libp2p/go-libp2p/go-libp2p-swarm/testing) because:
1. The swarm duplicated this exact function for its own tests.
2. The swarm couldn't depend on [go-libp2p-netutil][] because
[go-libp2p-netutil][] depends on [go-libp2p-swarm][].
2. The swarm couldn't depend on [go-libp2p-netutil](https://github.com/libp2p/go-libp2p/go-libp2p-netutil) because [go-libp2p-netutil](https://github.com/libp2p/go-libp2p/go-libp2p-netutil) depends on [go-libp2p-swarm](https://github.com/libp2p/go-libp2p/go-libp2p-swarm).
We've also added a new transport test suit
[go-libp2p-transport/test][]. If you implement a new transport, please consider
testing against these suite. If you find a bug in an existing transport, please
consider adding a test to this suite.
We've also added a new transport test suite [go-libp2p-transport/test](https://github.com/libp2p/go-libp2p-transport/test). If you implement a new transport, please consider testing against these suite. If you find a bug in an existing transport, please consider adding a test to this suite.
### go-addr-util
In go-addr-util, we've removed the `SupportedTransportStrings` and
`SupportedTransportProtocols` transport registries and the associated
`AddTransport` function. These registries were updated by `init` functions in
packages providing transports and were used to keep track of known transports.
In go-addr-util, we've removed the `SupportedTransportStrings` and `SupportedTransportProtocols` transport registries and the associated `AddTransport` function. These registries were updated by `init` functions in packages providing transports and were used to keep track of known transports.
However, _importing_ a transport doesn't mean any libp2p nodes have been
configured to actually _use_ that transport. Therefore, in the new go-libp2p,
it's go-libp2p-swarm's job to keep track of which transports are supported
(i.e., which transports have been registered with the swarm).
However, _importing_ a transport doesn't mean any libp2p nodes have been configured to actually _use_ that transport. Therefore, in the new go-libp2p, it's go-libp2p-swarm's job to keep track of which transports are supported (i.e., which transports have been registered with the swarm).
We've also removed the associated `AddrUsable`, `FilterUsableAddrs`, and
`AddrUsableFunc` functions.
We've also removed the associated `AddrUsable`, `FilterUsableAddrs`, and `AddrUsableFunc` functions.
### Pluggable Security Transports
This release brings a new pluggable security transport framework. Implementing a
new security framework is now as simple as:
This release brings a new pluggable security transport framework. Implementing a new security framework is now as simple as:
1. Implement the interfaces defined in [go-conn-security][].
2. Pass it into the libp2p constructor using the `Security` option.
[go-conn-security]: https://github.com/libp2p/go-conn-security
[go-libp2p-conn]: https://github.com/libp2p/go-libp2p-conn
[go-libp2p-interface-conn]: https://github.com/libp2p/go-libp2p-interface-conn
[go-libp2p-net]: https://github.com/libp2p/go-libp2p-net
[go-libp2p-netutil]: https://github.com/libp2p/go-libp2p/go-libp2p-netutil
[go-libp2p-swarm]: https://github.com/libp2p/go-libp2p/go-libp2p-swarm
[go-libp2p-swarm/testing]: https://github.com/libp2p/go-libp2p/go-libp2p-swarm/testing
[go-libp2p-transport]: https://github.com/libp2p/go-libp2p-transport
[go-libp2p-transport/test]: https://github.com/libp2p/go-libp2p-transport/test
[go-libp2p-transport-upgrader]: https://github.com/libp2p/go-libp2p-transport-upgrader
[go-peerstream]: https://github.com/libp2p/go-peerstream
[go-tcp-transport]: https://github.com/libp2p/go-tcp-transport
[go-ws-transport]: https://github.com/libp2p/go-ws-transport
[example:echo]: https://github.com/libp2p/go-libp2p/tree/master/examples/echo
[doc:go-libp2p-transport]: https://godoc.org/github.com/libp2p/go-libp2p-transport
1. Implement the interfaces defined in [go-conn-security](https://github.com/libp2p/go-conn-security).
2. Pass it into the libp2p constructor using the `Security` option.
src/_blog/035-go-ipfs-0-4-18.md
+67 -175
View File
@@ -1,94 +1,59 @@
---
date: 2018-11-07
permalink: /53-go-ipfs-0-4-18/
permalink: "/53-go-ipfs-0-4-18/"
title: go-ipfs 0.4.18 released
description:
description:
author: Steven Allen, Erik Ingenito
tags:
- 'go-ipfs'
- 'QUIC'
- 'pubsub'
- 'Gossipsub'
- 'IPFS Web UI'
---
- go-ipfs
- QUIC
- pubsub
- Gossipsub
- IPFS Web UI
---
[go-ipfs 0.4.18](https://dist.ipfs.io/#go-ipfs) has been released! This is one of largest go-ipfs releases to date; 3 months in the making. _Thanks to all our contributors for your awesome work!_
## ✨ Highlights
The headline features this release are:
- **[Experimental QUIC support](#quic)** - for faster and more efficient peer connections, better handling of lossy
networks and improved NAT traversal.
- **[Gossipsub pubsub routing algorithm](#pubsub)** - dramatically more efficient pubsub on IPFS, along with signed
messages
- **[Updated WebUI](#webui)** - a big update to the IPFS WebUI with expanded features and information across the board
- **[Enhanced `p2p`, `cid` and `add`](#commands-changes)** - refactors and new features to a number of IPFS
* [**Experimental QUIC support**](#quic) - for faster and more efficient peer connections, better handling of lossy networks and improved NAT traversal.
* [**Gossipsub pubsub routing algorithm**](#pubsub) - dramatically more efficient pubsub on IPFS, along with signed messages
* [**Updated WebUI**](#webui) - a big update to the IPFS WebUI with expanded features and information across the board
* [**Enhanced `p2p`, `cid` and `add`**](#commands-changes) - refactors and new features to a number of IPFS
commands.
- **[Performance](#performance)** - numerous performance and efficiency improvements in a number of IPFS subsystems.
IPFS is faster, smaller and more reliable.
- **[And a lot more!](#refactors-and-endeavors)**
* [**Performance**](#performance) - numerous performance and efficiency improvements in a number of IPFS subsystems. IPFS is faster, smaller and more reliable.
* [**And a lot more!**](#refactors-and-endeavors)
## 🏃 QUIC
First up, on the networking front, this release introduced experimental
support for the QUIC protocol. QUIC is a new UDP-based network transport that
solves many of the long standing issues with TCP.
First up, on the networking front, this release introduced experimental support for the QUIC protocol. QUIC is a new UDP-based network transport that solves many of the long standing issues with TCP.
For us, this means (eventually):
- **Fewer local resources** - TCP requires a file-descriptor per connection while
QUIC (and most UDP based transports) can share a single file descriptor
between all connections. This should allow us to dial faster and keep more
connections open.
- **Faster connection establishment** - When client authentication is included, QUIC
has a three-way handshake like TCP. However, unlike TCP, this handshake brings
us all the way from 0 to a fully encrypted, authenticated, and
multiplexed connection. In theory (not yet in practice), this should
significantly reduce the latency of DHT queries which will improve a number of IPFS operations like adding and
getting large volumes of data.
- **Behaves better on lossy networks** - When multiplexing multiple requests over a
single TCP connection, a single dropped packet will bring the entire
connection to a halt while the packet is re-transmitted. However, because QUIC
handles multiplexing internally, dropping a single packets affects only the
related stream.
- **Better NAT traversal** - NAT hole-punching is significantly easier and, in
many cases, more reliable with UDP than with TCP.
* **Fewer local resources** - TCP requires a file-descriptor per connection while QUIC (and most UDP based transports) can share a single file descriptor between all connections. This should allow us to dial faster and keep more connections open.
* **Faster connection establishment** - When client authentication is included, QUIC has a three-way handshake like TCP. However, unlike TCP, this handshake brings us all the way from 0 to a fully encrypted, authenticated, and multiplexed connection. In theory (not yet in practice), this should significantly reduce the latency of DHT queries which will improve a number of IPFS operations like adding and getting large volumes of data.
* **Behaves better on lossy networks** - When multiplexing multiple requests over a single TCP connection, a single dropped packet will bring the entire connection to a halt while the packet is re-transmitted. However, because QUIC handles multiplexing internally, dropping a single packets affects only the related stream.
* **Better NAT traversal** - NAT hole-punching is significantly easier and, in many cases, more reliable with UDP than with TCP.
However, we still have a long way to go. While we encourage users to test this,
the IETF QUIC protocol is still being actively developed and _will_ change. You can find instructions for enabling it [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#QUIC).
However, we still have a long way to go. While we encourage users to test this, the IETF QUIC protocol is still being actively developed and _will_ change. You can find instructions for enabling it [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#QUIC).
## 📨 Pubsub
go-ipfs now supports the [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) routing algorithm and message signing.
The gossipsub routing algorithm is _significantly_ more efficient than the
current floodsub routing algorithm. Even better, it's fully backwards compatible
so you can enable it and still talk to nodes using the floodsub algorithm. You
can find instructions to enable gossipsub in go-ipfs
[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#gossipsub).
The gossipsub routing algorithm is _significantly_ more efficient than the current floodsub routing algorithm. Even better, it's fully backwards compatible so you can enable it and still talk to nodes using the floodsub algorithm. You can find instructions to enable gossipsub in go-ipfs [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#gossipsub).
Messages are now **signed by their authors**. While signing is now
enabled by default, strict signature verification has not been and will not be
for at least one release (probably multiple) to avoid breaking existing
applications. You can read about how to configure this feature
[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#message-signing).
Messages are now **signed by their authors**. While signing is now enabled by default, strict signature verification has not been and will not be for at least one release (probably multiple) to avoid breaking existing applications. You can read about how to configure this feature [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#message-signing).
## 🎛 Commands Changes
In terms of new toys, this release introduces 1) the`ipfs cid` subcommand for
working with CIDs, 2) a completely refactored `ipfs p2p` command, 3) streaming name
resolution, and 4) inline block support.
In terms of new toys, this release introduces 1) the`ipfs cid` subcommand for working with CIDs, 2) a completely refactored `ipfs p2p` command, 3) streaming name resolution, and 4) inline block support.
#### 1. `ipfs cid`
The new `ipfs cid` command allows users to both inspect CIDs and convert them
between various formats and versions. Here are some examples:
The new `ipfs cid` command allows users to both inspect CIDs and convert them between various formats and versions. Here are some examples:
```sh
# Print out the CID metadata (prefix)
@@ -106,28 +71,19 @@ bafybeicg2rebjoofv4kbyovkw7af3rpiitvnl6i7ckcywaq6xjcxnc2mby
#### 2. `ipfs p2p`
The refactored `ipfs p2p` command allows forwarding TCP streams through two IPFS
nodes from one host to another. It's `ssh -L` but for IPFS.
**It's still experimental** but we don't expect too many breaking changes after this
point (it will very likely be stabilized in the next release).
The refactored `ipfs p2p` command allows forwarding TCP streams through two IPFS nodes from one host to another. It's `ssh -L` but for IPFS. **It's still experimental** but we don't expect too many breaking changes after this point (it will very likely be stabilized in the next release).
Here's a quick summary of the breaking changes in this release:
- We don't stop listening for local (forwarded) connections after accepting a
single connection.
- `ipfs p2p stream ls` output now returns more useful output, first address is
always the initiator address.
- `ipfs p2p listener ls` is renamed to `ipfs p2p ls`
- `ipfs p2p listener close` is renamed to `ipfs p2p close`
- Protocol names have to be prefixed with `/x/` and are now just passed to
libp2p as handler name. Previous version did this 'under the hood' and with
`/p2p/` prefix. There is a `--allow-custom-protocol` flag which allows you
to use any libp2p handler name.
- `ipfs p2p listener open` was renamed to `ipfs p2p listen`
- `ipfs p2p stream dial` got renamed to `ipfs p2p forward`
* We don't stop listening for local (forwarded) connections after accepting a single connection.
* `ipfs p2p stream ls` output now returns more useful output, first address is always the initiator address.
* `ipfs p2p listener ls` is renamed to `ipfs p2p ls`
* `ipfs p2p listener close` is renamed to `ipfs p2p close`
* Protocol names have to be prefixed with `/x/` and are now just passed to libp2p as handler name. Previous version did this 'under the hood' and with `/p2p/` prefix. There is a `--allow-custom-protocol` flag which allows you to use any libp2p handler name.
* `ipfs p2p listener open` was renamed to `ipfs p2p listen`
* `ipfs p2p stream dial` got renamed to `ipfs p2p forward`
You can find documentation [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-p2p),
but here's a quick example of connecting the WebUI of a remote IPFS node:
You can find documentation [here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-p2p), but here's a quick example of connecting the WebUI of a remote IPFS node:
```sh
# On the 'remote' IPFS host
@@ -146,34 +102,21 @@ QmSiXjrYwLmDhRvAb3vK2TUP8W2pTDd34MhgCwpanVjdNT
#### 3. `ipfs name resolve` streaming response
There is now a new flag for `ipfs name resolve` - `--stream`. When the command
is invoked with the flag set, it will start returning results as soon as they
are discovered in the DHT and other routing mechanisms. This enables certain
applications to start prefetching/displaying data while the discovery is still
running. Note that this command will likely return many outdated records
before it finding and returning the latest. However, it will always return
_valid_ records (even if a bit stale).
There is now a new flag for `ipfs name resolve` - `--stream`. When the command is invoked with the flag set, it will start returning results as soon as they are discovered in the DHT and other routing mechanisms. This enables certain applications to start prefetching/displaying data while the discovery is still running. Note that this command will likely return many outdated records before it finding and returning the latest. However, it will always return _valid_ records (even if a bit stale).
#### 4. `ipfs add` block inlining
In the previous release, we added support for extracting blocks inlined
into CIDs. In this release, we've added support for creating these CIDs. You can
now run `ipfs add` with the `--inline` flag to inline blocks less than or equal
to 32 bytes in length into a CID, instead of writing an actual block. This
should significantly reduce the size of filesystem trees with many empty
directories and tiny files.
In the previous release, we added support for extracting blocks inlined into CIDs. In this release, we've added support for creating these CIDs. You can now run `ipfs add` with the `--inline` flag to inline blocks less than or equal to 32 bytes in length into a CID, instead of writing an actual block. This should significantly reduce the size of filesystem trees with many empty directories and tiny files.
## 🌐 WebUI
This release includes the latest, very shiny [updated webui](https://github.com/ipfs-shipyard/ipfs-webui). You can view it by
installing go-ipfs and visiting http://localhost:5001/webui. It deserves its own release note - oh look, it [got
one](../assets/51-js-ipfs-0-33/#web-ui-2-0)! Here's a peek:
This release includes the latest, very shiny [updated webui](https://github.com/ipfs-shipyard/ipfs-webui). You can view it by installing go-ipfs and visiting http://localhost:5001/webui. It deserves its own release note - oh look, it [got one](../assets/51-js-ipfs-0-33/#web-ui-2-0)! Here's a peek:
![Screenshot of the status page](../assets/ipfs-webui-status.png)
| Files | Explore | Peers | Settings |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------- |
| ![Screenshot of the file browser page](../assets/ipfs-webui-files.png) | ![Screenshot of the IPLD explorer page](../assets/ipfs-webui-explore.png) | ![Screenshot of the swarm peers map](../assets/ipfs-webui-peers.png) | ![Screenshot of the settings page](../assets/ipfs-webui-settings.png) |
| Files | Explore | Peers | Settings |
| --- | --- | --- | --- |
| | | | |
Kudos and thanks to the webui team! 👏
@@ -183,9 +126,7 @@ This release includes some significant performance improvements, both in terms o
### Resource Utilization
In this release, we've (a) fixed a slow memory leak in libp2p and (b)
significantly reduced the allocation load. Together, these should improve both
memory and CPU usage. How much you ask? Glad you asked.
In this release, we've (a) fixed a slow memory leak in libp2p and (b) significantly reduced the allocation load. Together, these should improve both memory and CPU usage. How much you ask? Glad you asked.
![gc-latency](../assets/035-go-ipfs-0-4-18-gc-latency.png)
@@ -193,133 +134,84 @@ Above is a graph of time our IPFS gateway nodes spend in GC. See the awesome loo
![cpu-load](../assets/035-go-ipfs-0-4-18-cpu-load.png)
This graph of time spent in execution has greater variability but the improvement is still pretty clear. You want the
light blue one.
This graph of time spent in execution has greater variability but the improvement is still pretty clear. You want the light blue one.
#### Datastructures
We've changed two of our most frequently used datastructures, CIDs and
Multiaddrs, to reduce allocation load.
We've changed two of our most frequently used datastructures, CIDs and Multiaddrs, to reduce allocation load.
First, we now store CIDs _encoded_ as strings, instead of decoded in structs
(behind pointers). In addition to being more compact, our `Cid` type is now a
valid `map` key so we no longer have to encode CIDs every time we want to use
them in a map/set. Allocations when inserting CIDs into maps/sets was showing up
as a significant source of allocations under heavy load so this change should
improve memory usage.
First, we now store CIDs _encoded_ as strings, instead of decoded in structs (behind pointers). In addition to being more compact, our `Cid` type is now a valid `map` key so we no longer have to encode CIDs every time we want to use them in a map/set. Allocations when inserting CIDs into maps/sets was showing up as a significant source of allocations under heavy load so this change should improve memory usage.
Second, we've changed many of our multiaddr parsing/processing/formatting
functions to allocate less. Much of our DHT related-work includes processing
multiaddrs so this should reduce CPU utilization when heavily using the DHT.
Second, we've changed many of our multiaddr parsing/processing/formatting functions to allocate less. Much of our DHT related-work includes processing multiaddrs so this should reduce CPU utilization when heavily using the DHT.
#### Streams and Yamux
Streams have always plagued us in terms of memory utilization. This was
partially solved by introducing the connection manager, keeping our maximum
connection count to a reasonable number but they're still a major memory sink.
Streams have always plagued us in terms of memory utilization. This was partially solved by introducing the connection manager, keeping our maximum connection count to a reasonable number but they're still a major memory sink.
This release sees two improvements on this front:
1. A memory [leak in identify](https://github.com/libp2p/go-libp2p/issues/419)
has been fixed. This was slowly causing us to leak connections (locking up
the memory used by the connections' streams).
2. Yamux streams now use a buffer-pool backed, auto shrinking read buffer.
Before, this read buffer would grow to its maximum size (a few megabytes) and
never shrink, but **now** these buffers shrink as they're emptied and **free up space efficiently**.
1. A memory [leak in identify](https://github.com/libp2p/go-libp2p/issues/419) has been fixed. This was slowly causing us to leak connections (locking up the memory used by the connections' streams).
2. Yamux streams now use a buffer-pool backed, auto shrinking read buffer. Before, this read buffer would grow to its maximum size (a few megabytes) and never shrink, but **now** these buffers shrink as they're emptied and **free up space efficiently**.
### Bitswap Performance
Bitswap will now pack _multiple_ small blocks into a single message thanks to
[ipfs/go-bitswap#5](https://github.com/ipfs/go-bitswap/pull/5). While this won't
help when transferring large files (with large blocks), this should help when
transferring many tiny files.
Bitswap will now pack _multiple_ small blocks into a single message thanks to [ipfs/go-bitswap#5](https://github.com/ipfs/go-bitswap/pull/5). While this won't help when transferring large files (with large blocks), this should help when transferring many tiny files.
## 🛠 Refactors and Endeavors
This release saw yet another commands-library refactor, work towards the
CoreAPI, and the first step towards reliable base32 CID support.
This release saw yet another commands-library refactor, work towards the CoreAPI, and the first step towards reliable base32 CID support.
### Commands Lib
We've completely refactored our commands library (again). While it still needs
quite a bit of work, it now requires significantly less boilerplate and should
be significantly more robust. The refactor immediately found two broken tests
and probably fixed quite a few bugs around properly returning and handling
errors.
We've completely refactored our commands library (again). While it still needs quite a bit of work, it now requires significantly less boilerplate and should be significantly more robust. The refactor immediately found two broken tests and probably fixed quite a few bugs around properly returning and handling errors.
### CoreAPI
CoreAPI is a new way to interact with IPFS from Go. While it's still not
final, most things you can do via the CLI or HTTP interfaces can now be done
through the new API.
CoreAPI is a new way to interact with IPFS from Go. While it's still not final, most things you can do via the CLI or HTTP interfaces can now be done through the new API.
Currently only the Go implementation exists, but there are plans to expose the new API via HTTP soon.
We are also looking into creating an RPC interface to this API which could help performance in some use cases.
Currently only the Go implementation exists, but there are plans to expose the new API via HTTP soon. We are also looking into creating an RPC interface to this API which could help performance in some use cases.
You can track progress in https://github.com/ipfs/go-ipfs/issues/4498
### CIDv1/Base32 Migration
We're continuing work to upgrade our default CID format to Base32 while preserving compatibility with existing CIDs. We
need this change to improve the security of IPFS content in browsers. Currently, IPFS is usually used in browsers by browsing to `https://SOME_GATEWAY/ipfs/CID/...`. There are two significant drawbacks to this approach:
We're continuing work to upgrade our default CID format to Base32 while preserving compatibility with existing CIDs. We need this change to improve the security of IPFS content in browsers. Currently, IPFS is usually used in browsers by browsing to `https://SOME_GATEWAY/ipfs/CID/...`. There are two significant drawbacks to this approach:
1. From a browser security standpoint, all IPFS "sites" will live under the same
origin (SOME_GATEWAY).
2. From a UX standpoint, this doesn't feel very "native" (even if the gateway is
a local IPFS node).
To fix the security issue, we intend to switch IPFS gateway links
`https://ipfs.io/ipfs/CID` to `https://CID.ipfs.dweb.link`. This way, the CID
will be a part of the
["origin"](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin) so
each IPFS website will get a separate security origin.
To fix the security issue, we intend to switch IPFS gateway links `https://ipfs.io/ipfs/CID` to `https://CID.ipfs.dweb.link`. This way, the CID will be a part of the ["origin"](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin) so each IPFS website will get a separate security origin.
To fix the UX issue, we've been working on adding support for `ipfs://CID/...`
to web browsers through our
[ipfs-companion](https://github.com/ipfs/ipfs-companion/) add-on and some new,
experimental extension APIs from Mozilla. This has the same effect of putting
the CID in the URL origin but has the added benefit of looking "native".
To fix the UX issue, we've been working on adding support for `ipfs://CID/...` to web browsers through our [ipfs-companion](https://github.com/ipfs/ipfs-companion/) add-on and some new, experimental extension APIs from Mozilla. This has the same effect of putting the CID in the URL origin but has the added benefit of looking "native".
Unfortunately, origins must be _case insensitive_. Currently, the most common CIDs
are _CIDv0_ CIDs (those starting with `Qm`) which are _always_ base58
encoded and are therefore case-sensitive.
Unfortunately, origins must be _case insensitive_. Currently, the most common CIDs are _CIDv0_ CIDs (those starting with `Qm`) which are _always_ base58 encoded and are therefore case-sensitive.
Fortunately, CIDv1 (the latest CID format) supports arbitrary bases using the
[multibase](https://github.com/multiformats/multibase/) standard. Unfortunately,
IPFS has always treated equivalent CIDv0 and CIDv1 CIDs as distinct. This means
that files added with CIDv0 CIDs (the default) can't be looked up using the
equivalent CIDv1.
Fortunately, CIDv1 (the latest CID format) supports arbitrary bases using the [multibase](https://github.com/multiformats/multibase/) standard. Unfortunately, IPFS has always treated equivalent CIDv0 and CIDv1 CIDs as distinct. This means that files added with CIDv0 CIDs (the default) can't be looked up using the equivalent CIDv1.
This release makes some significant progress towards solving this issue by
introducing two features:
This release makes some significant progress towards solving this issue by introducing two features:
(1) The previous mentioned `ipfs cid base32` command for converting a CID to a
case insensitive encoding required by domain names. This command converts an existing base58 CIDv0 to
a CIDv1 encoded using base32.
(1) The previous mentioned `ipfs cid base32` command for converting a CID to a case insensitive encoding required by domain names. This command converts an existing base58 CIDv0 to a CIDv1 encoded using base32.
(2) A hack to allow locally looking up blocks associated with a CIDv0 CID using
the equivalent CIDv1 CID (or the reverse). This hack will eventually
be replaced with a multihash indexed blockstore, which is agnostic to both the
CID version and multicodec content type.
(2) A hack to allow locally looking up blocks associated with a CIDv0 CID using the equivalent CIDv1 CID (or the reverse). This hack will eventually be replaced with a multihash indexed blockstore, which is agnostic to both the CID version and multicodec content type.
## 📋 Full Changelog
As always, you can find the full (massive) changelog over at ipfs/go-ipfs's Github
repository:
https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#go-ipfs-changelog-1
As always, you can find the full (massive) changelog over at ipfs/go-ipfs's Github repository: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#go-ipfs-changelog-1
## 🙌 Contributing
Would you like to help contribute to the go-ipfs project?
- Join us on Github at https://github.com/ipfs/go-ipfs where you can find out more about the project.
- Check the issues with the help wanted label at the Ready column in our waffle board - https://waffle.io/ipfs/go-ipfs?label=help%20wanted
- Join an IPFS All Hands, introduce yourself and let us know where you would like to contribute or any cool demos of what you've built - https://github.com/ipfs/pm/#all-hands-call
- Join the discussion at http://discuss.ipfs.io/ and help users finding their answers.
- Join the [Go Core Dev Team Weekly Sync](https://github.com/ipfs/pm/issues/674) 🙌🏽 and be part of the Sprint action!
* Join us on Github at https://github.com/ipfs/go-ipfs where you can find out more about the project.
* Check the issues with the help wanted label at the Ready column in our waffle board - https://waffle.io/ipfs/go-ipfs?label=help%20wanted
* Join an IPFS All Hands, introduce yourself and let us know where you would like to contribute or any cool demos of what you've built - https://github.com/ipfs/pm/#all-hands-call
* Join the discussion at http://discuss.ipfs.io/ and help users finding their answers.
* Join the [Go Core Dev Team Weekly Sync](https://github.com/ipfs/pm/issues/674) 🙌🏽 and be part of the Sprint action!
## ⁉️ Do you have questions?
The best place to ask your questions about IPFS, how it works and what you can do with it, is at [discuss.ipfs.io](https://discuss.ipfs.io). We are also available at the #ipfs channel on Freenode.
Thanks!
Thanks!
src/_blog/058-ipfs-desktop-0-9.md
+10 -10
View File
@@ -1,13 +1,13 @@
---
date: 2019-09-19
permalink: /2019-09-19-ipfs-desktop-0-9/
permalink: "/2019-09-19-ipfs-desktop-0-9/"
title: IPFS Desktop 0.9 released
description:
description:
author: Henrique Dias
tags:
- 'IPFS Desktop'
---
- IPFS Desktop
---
IPFS Desktop has come a long way to reach its current form. We are here to celebrate and announce the [release of 0.9.0](https://github.com/ipfs-shipyard/ipfs-desktop/releases) and share the story of this application and, of course, all the exciting new features.
## Once upon a time...
@@ -46,14 +46,14 @@ Adding `ipfs` command line tools to your system was actually introduced in 0.8.0
Other notable features from previous releases include:
- **Handle `ipfs://`, `ipns://` and `dweb:` links**. If you have IPFS Desktop installed and you click on [ipns://ipfs.io](ipns://ipfs.io) in your browser or any other application, that request will go through be redirected to your own gateway via the app, or to the public gateway if yours is not online.
- **Easily add files to IPFS**. Just drag and drop them to the application icon, either on the menubar for macOS users, or the shortcut icon for Windows users.
- **Add screenshots to IPFS**. You can click on 'Take Screenshot' on the app's menu or enable the global shortcut on "Settings". After taking a screenshot, a shareable link will be copied to your clipboard!
- **Download any CID content**. If you would like to download the contents of some CID or IPFS/IPNS path, you can copy it, go to the dropdown menu and select 'Download Hash'. Then, just pick wherever you want to write those files to! This is also available through a global shortcut.
* **Handle `ipfs://`, `ipns://` and `dweb:` links**. If you have IPFS Desktop installed and you click on [ipns://ipfs.io](ipns://ipfs.io) in your browser or any other application, that request will go through be redirected to your own gateway via the app, or to the public gateway if yours is not online.
* **Easily add files to IPFS**. Just drag and drop them to the application icon, either on the menubar for macOS users, or the shortcut icon for Windows users.
* **Add screenshots to IPFS**. You can click on 'Take Screenshot' on the app's menu or enable the global shortcut on "Settings". After taking a screenshot, a shareable link will be copied to your clipboard!
* **Download any CID content**. If you would like to download the contents of some CID or IPFS/IPNS path, you can copy it, go to the dropdown menu and select 'Download Hash'. Then, just pick wherever you want to write those files to! This is also available through a global shortcut.
You can also take a look at this video:
<iframe width="2000" height="600" src="https://www.youtube-nocookie.com/embed/-7jAIVeg2vQ" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/-7jAIVeg2vQ" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
# Let's talk about future...
@@ -65,4 +65,4 @@ Is there any feature you would like to see on IPFS Desktop? If so, please [creat
# Don't forget...
Don't forget to [install IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop/releases) today and be part of the decentralized Internet! We also recommend you to install our browser extension, [IPFS Companion](https://github.com/ipfs-shipyard/ipfs-companion/#install), which gives your browser superpowers. Both of these applications work well together. While IPFS Desktop makes your OS InterPlanetary, IPFS Companion connects to your node on your browser, enabling further integration on your Internet interactions to the IPFS world.
Don't forget to [install IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop/releases) today and be part of the decentralized Internet! We also recommend you to install our browser extension, [IPFS Companion](https://github.com/ipfs-shipyard/ipfs-companion/#install), which gives your browser superpowers. Both of these applications work well together. While IPFS Desktop makes your OS InterPlanetary, IPFS Companion connects to your node on your browser, enabling further integration on your Internet interactions to the IPFS world.
src/_blog/071-ipfs-in-web-browsers.md
+18 -18
View File
@@ -1,14 +1,14 @@
---
date: 2019-10-08
permalink: /2019-10-08-ipfs-browsers-update/
permalink: "/2019-10-08-ipfs-browsers-update/"
title: IPFS Browser Update
description:
description:
author: Dietrich Ayala
header_image: /071-ipfs-in-web-browsers-header-image.png
header_image: "/071-ipfs-in-web-browsers-header-image.png"
tags:
- 'browsers'
---
- browsers
---
Decentralization can feel slow in providing user-centric approaches to the internet's biggest problems. Right now, software that is fully in the control of the user is often too technical, too fragile, and too time-consuming to be the default choice.
But we're making headway. Today, we'd like to share some collaborations the IPFS project has had in the works for a while, which bring us a few steps closer to making unmediated access to information _just work_... by solving that "last mile" problem and _integrating IPFS directly into web browsers_.
@@ -17,7 +17,7 @@ But we're making headway. Today, we'd like to share some collaborations the IPFS
![Stages of browser integration](../assets/071-ipfs-in-web-browsers-stages-of-browser-integration.png)
The path to a truly decentralized web is a long one. For over 30 years the browser has been a [_client_](<https://en.wikipedia.org/wiki/Client_(computing)>) – but a foundational concept in P2P systems is that a participant is both a client _and a [server](<https://en.wikipedia.org/wiki/Server_(computing)>)\_. Web browser vendors and web standards bodies have not designed for this architectural shift, so we're breaking it down into steps.
The path to a truly decentralized web is a long one. For over 30 years the browser has been a [_client_](https://en.wikipedia.org/wiki/Client_(computing)) – but a foundational concept in P2P systems is that a participant is both a client _and a_ [_server_](https://en.wikipedia.org/wiki/Server_(computing)). Web browser vendors and web standards bodies have not designed for this architectural shift, so we're breaking it down into steps.
From the beginning, IPFS had an HTTP gateway. The gateway lets HTTP clients like web browsers publish to and read from the IPFS network. Now there are [lots of different public HTTP gateways to the IPFS network](https://ipfs.github.io/public-gateway-checker/), and the gateway we run at ipfs.io serves over _five million_ requests per day.
@@ -43,9 +43,9 @@ But that isn't all. One of the biggest barriers to even just experimenting with
These powerful APIs enable the js-ipfs node embedded in the browser to provide a _true_ P2P experience without the need for an external daemon:
- The embedded HTTP Gateway removes reliance on public gateways, by connecting over HTTP to a local IPFS node
- TCP transport improves connectivity - not only by enabling transport interoperability with go-ipfs nodes, but also makes direct browser-to-browser communication within the same network possible
- UDP sockets enable DNS-based service discovery of go-ipfs in LAN, and we are working on additional browser-to-browser discovery methods that work in offline environments
* The embedded HTTP Gateway removes reliance on public gateways, by connecting over HTTP to a local IPFS node
* TCP transport improves connectivity - not only by enabling transport interoperability with go-ipfs nodes, but also makes direct browser-to-browser communication within the same network possible
* UDP sockets enable DNS-based service discovery of go-ipfs in LAN, and we are working on additional browser-to-browser discovery methods that work in offline environments
By default, our browser extension still expects [IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop#ipfs-desktop) to be installed, however you can select the experimental option shown below in order to run _a standalone IPFS node_ in Brave itself. Mind this is an _early preview_ of both a full JS IPFS node and a gateway running in a browser extension. Performance and features will improve over time.
@@ -53,7 +53,7 @@ By default, our browser extension still expects [IPFS Desktop](https://github.co
At IPFS Camp earlier this year, Brave engineer Jocelyn Liu demoed some of these features at the science fair, including one-click install of Companion, the embedded gateway, and also talks about what's to come: tackling the current connectivity limitations such as the lack of DHT support in js-ipfs, and also plans Brave has for the design of the address bar when loading IPFS content.
<iframe width="2000" height="600" src="https://www.youtube-nocookie.com/embed/oMqe9LfnIDU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/oMqe9LfnIDU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
## Opera
@@ -67,11 +67,11 @@ Earlier this year [Opera announced they were working on support for IPFS](https:
We've been working with Mozilla on initial APIs since 2018, when the [libdweb project](https://github.com/mozilla/libdweb) emerged, providing browser extension APIs for many of the primitives a P2P system needs: filesystem access, TCP, UDP and protocol registration and handling. Earlier in 2018 the [ipfs:// scheme was whitelisted in Firefox](https://blog.mozilla.org/addons/2018/01/26/extensions-firefox-59/), so now with libdweb we were able to experiment with a proper [ipfs:// protocol handler](https://github.com/ipfs-shipyard/ipfs-companion/pull/533), which we demoed at Lab Day in August 2018:
<iframe width="2000" height="600" src="https://www.youtube-nocookie.com/embed/fS8pLOQdOoM" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/fS8pLOQdOoM" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
We used yet more APIs in libdweb to implement [local discovery and TCP transport](https://github.com/ipfs-shipyard/ipfs-companion/pull/553):
<iframe width="2000" height="600" src="https://www.youtube-nocookie.com/embed/FRzyWUXIyeo" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/FRzyWUXIyeo" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
Unfortunately, libdweb is still an experimental set of APIs - not included in Firefox yet. We're hoping to see more progress soon from Mozilla on shipping libdweb APIs in Firefox for at least some extensions this year!
@@ -91,10 +91,10 @@ Web browsers have been solely a client for over 30 years - putting all power in
Consider this a sneak peek. There's more on the way:
- In Brave, the gateway is, well... a gateway to even more: We're working on local discovery (mDNS) of IPFS nodes, and also Brave-to-Brave connectivity for embedded nodes.
- We're continuing our experiments with [libdweb](https://github.com/mozilla/libdweb) in Firefox, both on desktop and Android.
- [Puma Browser](https://www.pumabrowser.com/), the first mobile browser with support for [Coil's micropayments for publishers](https://www.grantfortheweb.org/), is interested in IPFS support.
- Keeping a close eye on [Bundled HTTP Exchanges](https://wicg.github.io/webpackage/draft-yasskin-wpack-bundled-exchanges.html) and ways IPFS could provide decentralized alternative to centralized HTTP CDNs.
- We're exploring what a native Chromium implementation might look like...
* In Brave, the gateway is, well... a gateway to even more: We're working on local discovery (mDNS) of IPFS nodes, and also Brave-to-Brave connectivity for embedded nodes.
* We're continuing our experiments with [libdweb](https://github.com/mozilla/libdweb) in Firefox, both on desktop and Android.
* [Puma Browser](https://www.pumabrowser.com/), the first mobile browser with support for [Coil's micropayments for publishers](https://www.grantfortheweb.org/), is interested in IPFS support.
* Keeping a close eye on [Bundled HTTP Exchanges](https://wicg.github.io/webpackage/draft-yasskin-wpack-bundled-exchanges.html) and ways IPFS could provide decentralized alternative to centralized HTTP CDNs.
* We're exploring what a native Chromium implementation might look like...
If you're an experienced Chromium developer or mobile app developer and you're interested in working on some of these projects, contact Dietrich Ayala on [Twitter](https://twitter.com/dietrich), the [fediverse](https://mastodon.social/@dietrich), [Secure Scuttlebutt](@vB/yLGZ7vBGgPMTG5NEczZc6ufr0YqhxRScOk6jPi60=.ed25519) or [LinkedIn](https://www.linkedin.com/in/dietrich).
If you're an experienced Chromium developer or mobile app developer and you're interested in working on some of these projects, contact Dietrich Ayala on [Twitter](https://twitter.com/dietrich), the [fediverse](https://mastodon.social/@dietrich), [Secure Scuttlebutt](@vB/yLGZ7vBGgPMTG5NEczZc6ufr0YqhxRScOk6jPi60=.ed25519) or [LinkedIn](https://www.linkedin.com/in/dietrich).
src/_blog/077-collaborative-clusters.md
+19 -42
View File
@@ -1,65 +1,42 @@
---
date: 2020-01-09
permalink: /2020-01-09-collaborative-clusters/
permalink: "/2020-01-09-collaborative-clusters/"
title: Announcing collaborative clusters
description:
description:
author: Hector Sanjuan
header_image: /077-collaborative-clusters-header-image.png
header_image: "/077-collaborative-clusters-header-image.png"
tags:
- 'IPFS Cluster'
- 'collaborative cluster'
- IPFS Cluster
- collaborative cluster
---
We are very excited to announce the first set of public _collaborative clusters_ and the first iteration of the [collaborative clusters website](https://collab.ipfscluster.io).
We are very excited to announce the first set of public _collaborative
clusters_ and the first iteration of the
[collaborative clusters website](https://collab.ipfscluster.io).
_Collaborative clusters_ are an easy way to join and help improving distribution and data availability of specific datasets in the IPFS Network.
_Collaborative clusters_ are an easy way to join and help improving distribution
and data availability of specific datasets in the IPFS Network.
Using IPFS Cluster's [latest release (0.12.1)](https://cluster.ipfs.io/news/0.12.0_release/) we have set up the first of those archives:
Using IPFS Cluster's
[latest release (0.12.1)](https://cluster.ipfs.io/news/0.12.0_release/) we
have set up the first of those archives:
* Filecoin cluster: which will be used to pin Filecoin parameters and Filecoin objects.
* Spanish books from the Gutenberg Project: a collection of Spanish literature from the [Gutenberg Project](http://www.gutenberg.org/).
* IPFS Websites: a list with a few of the IPFS-universe websites (such as ipfs.io, libp2p.io).
- Filecoin cluster: which will be used to pin Filecoin parameters and
Filecoin objects.
- Spanish books from the Gutenberg Project: a collection of Spanish literature
from the [Gutenberg Project](http://www.gutenberg.org/).
- IPFS Websites: a list with a few of the IPFS-universe websites (such as ipfs.io, libp2p.io).
Users can join these clusters by running a single `ipfs-cluster-follow` command. You will need to have enough space available in your IPFS repository (check the size row for each cluster on the [website](https://collab.ipfscluster.io)).
Users can join these clusters by running a single `ipfs-cluster-follow`
command. You will need to have enough space available in your IPFS repository
(check the size row for each cluster on the
[website](https://collab.ipfscluster.io)).
Instructions on how to setup and join these and, in the future, other
collaborative clusters can be found at
[https://collab.ipfscluster.io](https://collab.ipfscluster.io). You can stop
and re-start replication whenever you want. Also, here's a quick video to show
how easy it is:
Instructions on how to setup and join these and, in the future, other collaborative clusters can be found at [https://collab.ipfscluster.io](https://collab.ipfscluster.io). You can stop and re-start replication whenever you want. Also, here's a quick video to show how easy it is:
<iframe src="https://asciinema.org/a/yV2Bk4nlrPAQ6MQ4w6z3ea0uZ/iframe" style="width: 737px; height: 509px; overflow: hidden;" scrolling="no"></iframe>
<p class="powered">asciicast powered by <a href="https://asciinema.org/" target="_top">asciinema</a></p>
We hope that collaborative clusters will allow the community to participate in
the distribution and seeding of data they care about, enabling easy "mirroring"
on the IPFS network, along with unlocking _Content-Distribution-Network (CDN)_
properties.
We hope that collaborative clusters will allow the community to participate in the distribution and seeding of data they care about, enabling easy "mirroring" on the IPFS network, along with unlocking _Content-Distribution-Network (CDN)_ properties.
## Creating your own Clusters
Collaborative clusters are no different from normal IPFS Clusters, with the exception that
they include a list of _trusted peers_ (peers that can modify the cluster pinset).
Collaborative clusters are no different from normal IPFS Clusters, with the exception that they include a list of _trusted peers_ (peers that can modify the cluster pinset).
The process of setting up one of these and letting other peers easily join as
followers is documented at
[https://cluster.ipfs.io/documentation/collaborative/](https://cluster.ipfs.io/documentation/collaborative/).
The process of setting up one of these and letting other peers easily join as followers is documented at [https://cluster.ipfs.io/documentation/collaborative/](https://cluster.ipfs.io/documentation/collaborative/).
## About the IPFS Cluster project
The [IPFS Cluster project](https://cluster.ipfs.io) provides data
orchestration across a swarm of IPFS daemons by allocating, replicating and
tracking a global pinset distributed among multiple peers.
The [IPFS Cluster project](https://cluster.ipfs.io) provides data orchestration across a swarm of IPFS daemons by allocating, replicating and tracking a global pinset distributed among multiple peers.
For full documentation on how to setup and operate clusters, see
[https://cluster.ipfs.io/documentation](https://cluster.ipfs.io/documentation).
For full documentation on how to setup and operate clusters, see [https://cluster.ipfs.io/documentation](https://cluster.ipfs.io/documentation).
src/assets/111485037-90aa5d80-8736-11eb-9fbf-44e254214d1d.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 238 KiB