adam/ipfs-blog
1
0
Fork 0
mirror of https://github.com/ipfs/ipfs-blog.git synced 2026-08-02 06:53:03 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #233 from TheDiscordian/staging

Browse Source 
Blog Post: A Guide to IPFS Connectivity in Web Browsers
This commit is contained in:
Jessica Schilling
2021-06-10 12:34:29 -06:00
committed by GitHub
parent 832e008fdc 391b88effe
commit 0bc7ea7df7
2 changed files with 427 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.forestry/front_matter/templates/blog-post.yml
+1
View File
@@ -150,6 +150,7 @@ fields:
target="_blank">file an issue</a> with the details.
pages:
- src/_blog/2021-05-31-distributed-wikipedia-mirror-update.md
- src/_blog/a-guide-to-ipfs-connectivity-in-web-browsers.md
- src/_blog/announcing-js-ipfs-0.53.0.md
- src/_blog/audius-uses-ipfs-for-content-streaming-storage-to-empower-artists-creators-worldwide.md
- src/_blog/congrats-gitcoin-grants-round-9-winners.md
src/_blog/a-guide-to-ipfs-connectivity-in-web-browsers.md
+426
View File
@@ -0,0 +1,426 @@
---
title: A Guide to IPFS Connectivity in Web Browsers
description: A minimal chat example using js-ipfs in the browser.
author: Discordian
date: 2021-06-21
permalink: "/2021-06-21-guide-to-ipfs-connectivity-in-browsers/"
translationKey: ''
header_image: ''
tags:
- libp2p
- browsers
- js-ipfs
---
# A Guide to IPFS Connectivity in Web Browsers
We see a lot of questions about how to get started with using js-ipfs in the browser. I'm going to demonstrate a minimal chat example in js-ipfs entirely in the browser. It uses WebRTC to achieve browser-to-browser connectivity where possible, and a circuit relay to connect browser nodes where not. Message passing is done with [libp2p](https://docs.libp2p.io)'s [pubsub](https://docs.libp2p.io/concepts/publish-subscribe/).
### Getting the Code
You can see the live demo [here](https://ipfs.io/ipfs/bafybeia5f2yk6td7ciroeped2uwfivo333b524t3zmoderfhl3xn7wi7aa/). If you'd like a local copy you can edit yourself, you can download the whole directory using IPFS:
ipfs get bafybeia5f2yk6td7ciroeped2uwfivo333b524t3zmoderfhl3xn7wi7aa
Then simply open `index.html` in your web browser and you'll immediately begin automatically connecting to nodes and looking for peers!
You can also fork [TheDiscordian/browser-ipfs-chat](https://github.com/TheDiscordian/browser-ipfs-chat) on Github, and it'll be ready to test right away! If you want to deploy your own version, simply edit `index.html` and follow the setup information below.
The libraries used in this example are [js-ipfs](https://github.com/ipfs/js-ipfs/blob/master/docs/BROWSERS.md) and bootstrap (just their minified css). If you want a newer version of js-ipfs, feel free to download [this one here](https://cdn.jsdelivr.net/npm/ipfs/dist/index.min.js) to use the latest version available 😃.
Let's take a look at how this works.
# 📖 Table of Contents
* [🪐 Peer Discovery and Connectivity](#🪐-peer-discovery-and-connectivity)
* [🐳 Docker (optional)](#🐳-docker-optional)
* [Create Volume](#create-volume)
* [Configure Domain](#configure-domain)
* [Running the Container](#running-the-container)
* [🌟 WebRTC-Star](#🌟-webrtc-star)
* [Usage](#usage)
* [Setup](#setup)
* [⚡ p2p-circuit](#⚡-p2p-circuit)
* [Usage](#usage-2)
* [Setup](#setup-2)
* [Nginx Setup](#nginx-setup)
* [Advertising](#advetising)
* [🌐 Communication](#🌐-communication)
* [📰 PubSub](#📰-pubsub)
* [⚠️ Possible Browser Pitfalls](#⚠️-possible-browser-pitfalls)
* [Staying Connected to Peers](#staying-connected-to-peers)
* [Staying Connected to the Circuit Relay](#staying-connected-to-the-circuit-relay)
* [🎉 Conclusion](#🎉-conclusion)
# 🪐 Peer Discovery and Connectivity
In a browser discovering and connecting to peers can be very hard, as we can't listen for new peers, and we don't have access to the DHT. In order to have the best experience working in a browser, it's important to understand how to find peers, and stay connected with them.
The chat example achieves this in 2 ways. Using WebRTC-Star we achieve direct browser-to-browser communication, and with a circuit relay, we have a relay in the middle. The chat application has a status indicator in the top-left to let you know too what kind of connection you have. Green means you're connected to the relay, even if it's via another peer, yellow mean you're only seeing direct peers, and red means you have no peers (at least none using the chat application).
![BrowserIPFSNetworkGraph](https://ipfs.io/ipfs/QmX2og5BKJCMVaebEm9ZGsACEYExoGqxhJjePKNc2mZ2pE "BrowserIPFSNetworkGraph_Transparent.png")
🌟 The diagram above demonstrates what a 3 user network can look like. It's worth noting that the browser nodes can communicate with go-ipfs as well, so BrowserC doesn't have to be a browser at all, but instead could be a go-ipfs node!
## 🐳 Docker (optional)
If you don't want to use docker, skip to [**WebRTC-Star**](#🌟-webrtc-star).
After this section we'll go over what WebRTC-Star and circuit-relay do, and how to set them up. However if you'd like to quickly roll your own kit using docker, I've prepared an image you can use. It might not be the best long-term solution, but it should be great if you want to quickly get rolling and experiment.
### Create Volume
First create a volume to store long-term data like keys, and node data.
```bash
docker volume create ipfs_bundle
```
### Configure Domain
You need a domain, and SSL to use this kit with browser nodes. There are two options below, one will run certbot, and automatically grab a certificate for the provided domain name. The other option won't handle SSL for you, and instead you'll have to reverse proxy port 9091 to 9090 (SSL), and port 4011 to 4430 (SSL).
When you execute either commands your IPFS node will also be setup for the first time giving you information such as it's PeerID, and circuit-relay addresses. Take note of these, you'll want to edit them into the chat client so you can use your own node (see [WebRTC-Star#Usage](#usage) and [p2p-circuit#Usage](#usage-2) for usage examples, or edit `index.html`, and change my node's multiaddresses out for your own).
#### Certbot
Ensure port 80 is open, follow checklist below, then run the following command:
```bash
docker run --mount source=ipfs_bundle,destination=/root -p 9091:9091 -p 4011:4011 -p 9090:9090 -p 4430:4430 -p 80:80 -it trdiscordian/ipfsbundle certbot DOMAIN.COM
```
#### No Cerbot (SSL Disabled)
If you do this option, the container won't handle SSL at all, and you'll have to reverse proxy port 9091 to 9090 (SSL), and port 4011 to 4430 (SSL).
```bash
docker run --mount source=ipfs_bundle,destination=/root -p 9091:9091 -p 4011:4011 -it trdiscordian/ipfsbundle DOMAIN.COM
```
**📝 Checklist**
* Replace `DOMAIN.COM` with your domain
* Ensure the domain is correctly pointing to the machine you're running the container on (subdomains work fine too)
### Running the Container
Once you're configured, running the container is simple. Ensure at minimum ports 4430 and 9090 are forwarded.
```bash
docker run --mount source=ipfs_bundle,destination=/root -p 9091:9091 -p 4011:4011 -p 9090:9090 -p 4430:4430 -it trdiscordian/ipfsbundle
```
🎉 You should now be able to use this machine as both a WebRTC-Star node and a p2p-circuit node.
## 🌟 WebRTC-Star
We can use [WebRTC-Star](https://github.com/libp2p/js-libp2p-webrtc-star) nodes to help discover other peers we can connect with directly browser-to-browser. I find it easy to think of it as similar to [STUN](https://en.wikipedia.org/wiki/STUN), if you're already familiar with that concept. Effectively each connecting node will be given a WebRTC-Star [multiaddress](https://docs.libp2p.io/concepts/addressing/) that other nodes can use to discover and connect to your browser directly. Meaning if you peer with someone using the star node, and the star node goes offline, you remain connected!
### Usage
Connecting to a star node is quite simple:
```javascript
ipfs = await Ipfs.create({
repo: 'ok' + Math.random(), // random so we get a new peerid every time, useful for testing
config: {
Addresses: {
Swarm: [
'/dns4/star.thedisco.zone/tcp/9090/wss/p2p-webrtc-star',
'/dns6/star.thedisco.zone/tcp/9090/wss/p2p-webrtc-star'
]
},
}});
```
### Setup
Please note that this example uses my own star nodes, however those won't necessarily always be accessible there. Currently it's important to find a reliable star node, or host your own. You can host your own quite simply by following the instructions [here](https://github.com/libp2p/js-libp2p-webrtc-star#rendezvous-server-aka-signaling-server) for a native setup and [here](https://github.com/libp2p/js-libp2p-webrtc-star/blob/master/DEPLOYMENT.md) for a docker container which includes Nginx (for SSL). If you opt for the native setup, we cover the Nginx reverse proxy process and SSL cert retrieval later in this post.
🚀 This is a very clean and effective method of P2P communications, however sometimes NATs get in the way. For that, we use [p2p-circuit](https://docs.libp2p.io/concepts/circuit-relay/) to get around that.
## ⚡ p2p-circuit
p2p-circuit is really useful for peers behind tricky NATs (or a VPN, or anything really). I find the relaying of p2p-circuit to be similar to [TURN](https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT), so it's easy to think of it that way if you're already familiar with it.
### Usage
Once all the services for p2p-circuit are put together, connecting to the node can be achieved a couple of ways. First, to connect on startup to _only_ our node(s):
```javascript
ipfs = await Ipfs.create({
config: {
Bootstrap: [
'/dns6/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt',
'/dns4/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt'
]
}});
```
Or we can add our own after, then manually initiate the connection:
```javascript
await ipfs.bootstrap.add('/dns6/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt');
await ipfs.swarm.connect('/dns6/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt');
await ipfs.bootstrap.add('/dns4/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt');
await ipfs.swarm.connect('/dns4/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt');
```
If you're looking to do your own client, without copying the example, ensure you're also communicating with the announce channel, which is described under "Advertising". The relevant code in the chat demo is this (simplified):
```javascript
var ipfs; // store the IPFS node you're using in this variable
// processes a circuit-relay announce over pubsub
async function processAnnounce(addr) {
// get our peerid
me = await ipfs.id();
me = me.id;
// not really an announcement if it's from us
if (addr.from == me) {
return;
}
// if we got a keep-alive, nothing to do
if (addr == "keep-alive") {
console.log(addr);
return;
}
peer = addr.split("/")[9];
console.log("Peer: " + peer);
console.log("Me: " + me);
if (peer == me) { // return if the peer being announced is us
return;
}
// get a list of peers
peers = await ipfs.swarm.peers();
for (i in peers) {
// if we're already connected to the peer, don't bother doing a
// circuit connection
if (peers[i].peer == peer) {
return;
}
}
// log the address to console as we're about to attempt a connection
console.log(addr);
// connection almost always fails the first time, but almost always
// succeeds the second time, so we do this:
try {
await ipfs.swarm.connect(addr);
} catch(err) {
console.log(err);
await ipfs.swarm.connect(addr);
}
}
// process announcements over the relay network, and publish our own
// keep-alives to keep the channel alive
await ipfs.pubsub.subscribe("announce-circuit", processAnnounce);
setInterval(function(){ipfs.pubsub.publish("announce-circuit", "peer-alive");}, 15000);
```
### Setup
Like the star nodes, it'll be important to host your own things as mine could go offline at any moment.
For the purposes of this example, you'll need to do a few things on a server hosting your own [go-ipfs](https://github.com/ipfs/go-ipfs) node. You'll also need a working Nginx install setup, which will be used for SSL which is a requirement for browsers.
First configure the Go node, enabling [WebSocket](https://en.wikipedia.org/wiki/WebSocket) support, and designate it as a relay so we can communicate with it from a browser by editing `~/.ipfs/config` to add the following settings:
```json
{
"Addresses": {
"Swarm" : [
"/ip4/0.0.0.0/tcp/4011/ws",
"/ip6/::/tcp/4011/ws"
]
},
"Swarm": {
"EnableRelayHop": true
}
}
```
Restart your go-ipfs node however you normally do (possibly `systemctl --user restart ipfs`), and we're mostly setup! We've enabled regular WebSockets with relaying support, however we need secure WebSockets otherwise browsers won't connect to us.
### Nginx Setup
This setup is similar for WebRTC-Star, you just need to set it up as a different site, on a different port, with a new upstream name (instead of `ipfs`, try something like `star`).
First obtain and install [Certbot](https://certbot.eff.org/docs/install.html). Then edit the following file with your domain name, and port, then copy it to `/etc/nginx/sites-available/ipfs`.
```nginx
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream ipfs {
server 127.0.0.1:4011;
}
server {
server_name ipfs.YOURDOMAIN.COM;
listen 4430 ssl;
ssl_certificate /etc/letsencrypt/live/ipfs.YOURDOMAIN.COM/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ipfs.YOURDOMAIN.COM/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://ipfs;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
}
}
```
So in this example you can see we're accepting ssl on port 4430, this is our "wss port" (WebSocket Secure), and then passing it to the unsecured port locally on 4011, this is our "ws port". So if we want to connect to this node from a browser, we'd use port 4430.
After, run the following:
```bash
sudo systemctl stop nginx
sudo certbot -d ipfs.YOURDOMAIN.COM --standalone # Edit ipfs.YOURDOMAIN.COM to the domain you want a cert for
sudo ln -s /etc/nginx/sites-available/ipfs /etc/nginx/sites-enabled/ipfs
sudo systemctl start nginx
```
🎉 Nginx is now operating as a reverse-proxy, giving you secured WebSockets!
### Advertising
Using p2p-circuit can be a bit tricky. Once we connect to the relay from a browser, we're not advertising that we're able to be reached through it! For this purpose, I've created a Python script that runs alongside go-ipfs which advertises the browser js-ipfs peers it encounters over [PubSub](https://docs.libp2p.io/concepts/publish-subscribe/) with a p2p-circuit [multiaddress](https://docs.libp2p.io/concepts/addressing/).
You can find the Python script [here](https://gist.github.com/TheDiscordian/51962fea72f8d5a5c3bba79dd7009e1c). It can be run with a simple `python ipfs_peeradvertiser.py`. However, ensure you first edit `CIRCUIT` with your own node's information, or you won't announce the peers correctly, and they won't know how to use your relay to connect to other peers.
You can retrieve your own circuit info quite easily. Simply run `ipfs id` on your go-ipfs node, to get your PeerID, then form the circuit URL like so:
/dns6/ipfs.YOURDOMAIN.COM/tcp/4430/p2p/YOUR_PEERID/p2p-circuit/p2p/
You should see here where you simply fill out your domain name you got the SSL cert for, as well as your node's PeerID. For the script, the leading and trailing slash are required, too.
⚠️ **Notice** ⚠️
Ensure you specify dns6 or dns4, depending on if you're forming an IPv6 or IPv4 address. **It's important to ensure you use dns, otherwise browser nodes likely won't be able to connect.** Also note the port 4430, if you used a different one, you'll need to specify that.
# 🌐 Communication
Whew so you made it this far, you might be wondering "what is communication like?", well luckily the answer is it's _very_ easy in comparison to finding the peers, with only minor pitfalls. We're going to simply cover how we're using [PubSub](https://docs.libp2p.io/concepts/publish-subscribe/) in the chat example, and exactly what pitfalls were found while it was developed.
## 📰 PubSub
Using PubSub we're able to subscribe to topics, and retrieve any messages posted to those topics. In js-ipfs, we can set a callback function, which gets called whenever a message is received:
```javascript
function echo(msg) {
msg = new TextDecoder().decode(msg.data);
console.log(msg);
}
await ipfs.pubsub.subscribe("example_topic", echo);
```
Publishing is just as easy too:
```javascript
await ipfs.pubsub.publish("example_topic", "Hello world!");
```
This is effectively what the chat demo is doing. It's subscribing to a global topic (named "discochat-global"), and simply relaying the messages people type around over PubSub.
## ⚠️ Possible Browser Pitfalls
So let's say you've done everything correctly. You're able to find peers using WebRTC-Star and p2p-circuit, awesome! However you might find your connections expire, and you're unable to restore them. I'm not completely sure what causes this behaviour (probably some browser policy), however we can do our best to mitigate these issues!
### Staying Connected to Peers
We stay connected to peers in a couple ways. The first way is more direct, and that's by subscribing to and sending a "keepalive" announcement over `discochat-keepalive` every 4 seconds:
```javascript
setInterval(function(){sendmsg("1", prefix+"keepalive");}, 4000);
setInterval(checkalive, 1000);
```
This should help ensure we give peers looking to chat a high priority. Additionally, we report over `announce-circuit` every 15 seconds to make sure we keep a connection to the circuit relay so we can connect to peers stuck behind a NAT. That's accomplished like so:
```javascript
// process announcements over the relay network, and publish our own keep-alives to keep the channel alive
await ipfs.pubsub.subscribe("announce-circuit", processAnnounce);
setInterval(function(){ipfs.pubsub.publish("announce-circuit", "peer-alive");}, 15000);
```
🌟 A simplified version of `processAnnounce` is found under [p2p-circuit#Usage](#usage).
The Python script on the circuit relay will report a keepalive every 4 seconds. You may have noticed we're reporting "peer-alive" instead of "keep-alive", this is to separate peer requests from relay requests, to make it easier to tell when we no longer see a relay.
### Staying Connected to the Circuit Relay
Outside of the simplified version of `processAnnounce`, in the real version there are a couple variables used for tracking keep-alive and peer-alive. These are `lastAlive` and `lastPeer`, respectively. We even track the last time we bootstrapped via `lastBootstrap`. Using all this, we can display the yellow status when we're only connected to peers (tracked via `lastPeer`), and if we don't see a keep-alive for 35 seconds (and we haven't attempted a bootstrap in 60 seconds), we can attempt to re-connect to the bootstrap relay (and display a red status). This is accomplished like so:
```javascript
const bootstraps = [
'/dns6/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt',
'/dns4/ipfs.thedisco.zone/tcp/4430/wss/p2p/12D3KooWChhhfGdB9GJy1GbhghAAKCUR99oCymMEVS4eUcEy67nt'
];
var lastAlive = 0; // last keep-alive we saw from a relay
var lastPeer = 0; // last keep-alive we saw from another peer
var lastBootstrap = 0; // used for tracking when we last attempted to bootstrap (likely to reconnect to a relay)
// if reconnect is true, it'll first attempt to disconnect from the bootstrap nodes
async function dobootstrap(reconnect) {
now = new Date().getTime();
if (now-lastBootstrap < 60000) { // don't try to bootstrap again if we just tried within the last 60 seconds
return;
}
lastBootstrap = now;
for (i in bootstraps) {
if (reconnect) {
try {
await ipfs.swarm.disconnect(bootstraps[i]);
} catch (e) {
console.log(e);
}
} else {
await ipfs.bootstrap.add(bootstraps[i]);
}
await ipfs.swarm.connect(bootstraps[i]);
}
}
// check if we're still connected to the circuit relay
function checkalive() {
now = new Date().getTime();
if (now-lastAlive >= 35000) {
if (now-lastPeer >= 35000) {
document.getElementById("status-ball").style.color = "red";
} else {
document.getElementById("status-ball").style.color = "yellow";
}
dobootstrap(true); // let's try to reconnect
} else {
document.getElementById("status-ball").style.color = "lime";
}
}
setInterval(checkalive, 1000);
```
🌟 The above should be used with the full version of `processAnnounce` as it relies on `lastAlive` and `lastPeer`, which aren't updated in the simplified version.
# 🎉 Conclusion
I hope this was informative enough to get rolling. If you were successful in following this entire guide, you now have the ability to deploy powerful IPFS apps that run entirely in the browser, and leverage decentralised p2p whenever you can! I've selected some helpful resources and shared them below for further reading:
* [js-ipfs/docs/BROWSERS.md](https://github.com/ipfs/js-ipfs/blob/master/docs/BROWSERS.md)
* [js-ipfs/docs/CONFIG.md](https://github.com/ipfs/js-ipfs/blob/master/docs/CONFIG.md)
* [js-ipfs/docs/core-api](https://github.com/ipfs/js-ipfs/tree/master/docs/core-api)
* [js-ipfs/examples/circuit-relaying](https://github.com/ipfs/js-ipfs/tree/master/examples/circuit-relaying)
* [js-libp2p-webrtc-star](https://github.com/libp2p/js-libp2p-webrtc-star)