diff --git a/.gitignore b/.gitignore index eb66e38..562c145 100644 --- a/.gitignore +++ b/.gitignore @@ -36,3 +36,6 @@ vendor # generated js contrib/static/nntpchan.js contrib/static/miner-js.js + +#docs trash +doc/.trash diff --git a/README.md b/README.md index d028fc2..a850018 100644 --- a/README.md +++ b/README.md @@ -3,14 +3,11 @@ NNTPChan **NNTPChan** (previously known as overchan) is a decentralized imageboard that uses the [NNTP protocol](https://en.wikipedia.org/wiki/Network_News_Transfer_Protocol) (network-news protocol) to synchronize content between many different servers. It utilizes cryptographically signed posts to perform optional/opt-in decentralized moderation. -This repository contains resources used by the core daemon which is located on [github](https://github.com/majestrate/srndv2) (for now) along with general documentation, [here](doc/) +This repository contains resources used by the core daemon which is located on [GitHub](https://github.com/majestrate/srndv2) (for now) along with general documentation, [here](doc/) ##Getting started -This is a step-by-step guide for getting up and running with NNTPChan. - -1. [Compile the NNTPChan server](doc/build.md) -2. [Configuring your NNTPChan node](doc/config.md) +[This](doc) is a step-by-step guide for getting up-and-running with NNTPChan as well as documentation for developers wwho want to either work on NNTPChan directly or use NNTPChan in their aplications with the API. ##Bugs and issues @@ -29,7 +26,7 @@ Below is a list of known NNTPChan nodes: Need help? Join us on IRC. 1. [freenode: #nntpchan](https://webchat.freenode.net/?channels=#nntpchan) -2. [rizon: #nntpchan](https://qchat.rizon.net/?channels=#nntpchan) - Most activce +2. [rizon: #nntpchan](https://qchat.rizon.net/?channels=#nntpchan) - Most active ##Donations diff --git a/doc/.trash/Configuring Postgres database b/doc/.trash/Configuring Postgres database new file mode 100644 index 0000000..aaafbc5 --- /dev/null +++ b/doc/.trash/Configuring Postgres database @@ -0,0 +1,6 @@ +##Installing Postgres + +Postgres on Debian (as root) + + # install as root + apt-get install --no-install-recommends postgresql postgresql-client diff --git a/doc/.trash/Configuring Redis database b/doc/.trash/Configuring Redis database new file mode 100644 index 0000000..b703bad --- /dev/null +++ b/doc/.trash/Configuring Redis database @@ -0,0 +1,5 @@ +##Installing Redis + +Redis 3.x or higher is required, [stable release](http://download.redis.io/releases/redis-stable.tar.gz) recommended + +* See http://redis.io/download diff --git a/doc/.trash/configuration-files.md b/doc/.trash/configuration-files.md new file mode 100644 index 0000000..0e01b52 --- /dev/null +++ b/doc/.trash/configuration-files.md @@ -0,0 +1,9 @@ +Configuration Files +=================== + +In NNTPChan's stack there are currently 2 configuration files. + +All config files are auto generated with sane defaults if the files are not present. + +1. [srnd.ini](srnd.md) is for the core daemon +2. [feeds.ini](feeds.md) is for all your nntp outfeeds diff --git a/doc/README.md b/doc/README.md index a15707d..2bf09f9 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,12 +1,17 @@ NNTPChan Documentation ====================== +Hey, welcome to the documentation. This will help you use and develop with NNTPChan. + ##End-user related 1. [Building NNTPChan](building.md) - Building the source code 2. [Setting up NNTPChan](setting-up.md) - Configuring the node 3. [Running NNTPChan](running.md) - Running the node for the first time +4. [Managing your NNTPChan node with the CLI](cli.md) - Manage many aspects of your node via the command-line interface +5. [Configuring your news reader for NNTPChan](extras/configure-newsreader.md) - Setup Mozilla Thunderbird or Pan to send and receive articles from your node. ##Developer related -1. [Protocol](protocol.md) +1. [Protocol](developer/protocol.md) - NNTPChan's protocol specification +2. [JSON-RPC API](developer/api.md) - NNTPChan's JSON-RPC API diff --git a/doc/building.md b/doc/building.md index ec76253..b5e7f62 100644 --- a/doc/building.md +++ b/doc/building.md @@ -1,13 +1,14 @@ -Building the NNTPChan server -============================ +Building NNPTChan +================= -This document will help you setup the NNTPChan server from the source code. +This document will help you build the NNTPChan software from the source code. ##Requirements NNTPChan can run on the following operating systems: * Linux + * Instructions are available for Debian and Trisquel. * FreeBSD Dependancies: @@ -32,6 +33,21 @@ Install the Go programming language version _1.6_ from the [Go website](https:// sudo apt-get update sudo apt-get --no-install-recommends install imagemagick libsodium-dev ffmpeg sox build-essential git ca-certificates +###Installing Redis + +Run this to install Redis - if you want to use Redis. + + su + apt-get install redis-server + +###Installing Postgres (WIP) + +Run this to install Postgres - if you want to use Postgres. + + # install as root + su + apt-get install --no-install-recommends postgresql postgresql-client + ###Get the NNTPChan source git clone https://github.com/majestrate/nntpchan --depth=1 @@ -47,6 +63,55 @@ If you want to build without support for Redis then build with the `--no-redis` ./build.sh --no-redis -##Now let's setup NNTPChan +##Trisquel instructions (WIP) -Check out [Configuring NNTPChan](config.md). +These are installation instructions for Trisquel. + +###Install Go + +Run this to install Go. + + sudo apt-get update + sudo apt-get install golang-1.6 + +###Installing dependancies (standard) + + sudo apt-get --no-install-recommends install imagemagick libsodium-dev sox build-essential git ca-certificates + +###Installing dependancies (`ffmpeg` is not available in Trisquel repos - there must be compiled) + +This will install `ffmpeg` to `/usr/local/bin/ffmpeg`: + + git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg --depth=1 + cd ffmpeg + ./configure --disable-yasm + make + sudo make install + rm -rf ffmpeg + +###Installing Redis + +Run this to install Redis - if you want to use Redis. + + sudo apt-get install redis-server + +###Installing Postgres (WIP) + +Run this to install Postgres - if you want to use Postgres. + + sudo apt-get install --no-install-recommends postgresql postgresql-client + +###Get the NNTPChan source + + git clone https://github.com/majestrate/nntpchan --depth=1 + cd nntpchan + +###Now compile! + +If you want to compile with Redis support (recommended - Redis is easy to use) then run: + + ./build.sh + +If you want to build without support for Redis then build with the `--no-redis` flag: + + ./build.sh --no-redis diff --git a/doc/cli.md b/doc/cli.md index 5c4ea32..089d845 100644 --- a/doc/cli.md +++ b/doc/cli.md @@ -6,32 +6,32 @@ To rebuild all thumbnails run: - srndv2 tool rethumb + ./srndv2 tool rethumb ##Generate a new tripcode keypair (prints to stdout) - srndv2 tool keygen + ./srndv2 tool keygen ##Add a public key to moderation trust Where `publickey` is the public key to be added. - srndv2 tool mod add publickey + ./srndv2 tool mod add publickey ##Remove a public key from moderation trust Where `publickey` is the public key to be removed. - srndv2 tool mod del publickey + ./srndv2 tool mod del publickey ##Add a new NNTP user Where `username` is the username and `password` is the user's password for the new uer. - srndv2 tool nntp add-login username password + ./srndv2 tool nntp add-login username password ##Remove an existing NNTP user Where `username` is the username of the user to be deleted. - srndv2 tool nntp del-login username + ./srndv2 tool nntp del-login username diff --git a/doc/configuration.md b/doc/configuration.md deleted file mode 100644 index 29fe67b..0000000 --- a/doc/configuration.md +++ /dev/null @@ -1,9 +0,0 @@ -# Configuration # - -In nntpchan's stack there are currently 2 configuration files. - -All config files are auto generated with sane defaults if the files are not present. - -[feeds.ini](feeds.md) is for all your nntp outfeeds - -[srnd.ini](srnd.md) is for the core daemon diff --git a/doc/database.md b/doc/database.md deleted file mode 100644 index 5f2245e..0000000 --- a/doc/database.md +++ /dev/null @@ -1,5 +0,0 @@ -You have 2 choices for database backend: - -* [postgres](postgres.md) - -* [redis](redis.md) diff --git a/doc/postgres.md b/doc/database/postgres/configure-postgres.md similarity index 62% rename from doc/postgres.md rename to doc/database/postgres/configure-postgres.md index aa26a9e..c870c6a 100644 --- a/doc/postgres.md +++ b/doc/database/postgres/configure-postgres.md @@ -1,11 +1,11 @@ +Configuring Postgres database +============================= -Postgres on Debian (as root) +These are instructions for setting up NNTPChan with Postgres as the data-storage system. - # install as root - apt-get install --no-install-recommends postgresql postgresql-client +##Configuring Postgres - -Setting up postgres (as root) +Setting up postgres (as root): # become postgres user su postgres @@ -21,8 +21,6 @@ You'll get a prompt, enter the following: For demo purposes we'll use these credentials. These are default values, please change them later. -## important +###Important -these credentials assume you are going to run using a user called `srnd`, if your username you plan to run the daemon as is different please change `srnd` to your username. - -Now run it, [next](running.md) +These credentials assume you are going to run using a user called `srnd`, if your username you plan to run the daemon as is different please change `srnd` to your username. diff --git a/doc/database/redis/configure-redis.md b/doc/database/redis/configure-redis.md new file mode 100644 index 0000000..6dd67a4 --- /dev/null +++ b/doc/database/redis/configure-redis.md @@ -0,0 +1,20 @@ +Configuring Redis database +========================== + +These are instructions for setting up NNTPChan with Redis as the data-storage system. + +##Configuring Redis + +In `srnd.ini` the database sections should look like this: + + [database] + type=redis + schema=single + host=localhost + port=6379 + user= + password= + +##Securing Redis (optional) + +Read [Securing Redis](securing-redis.md) for adding password authentication to your Redis server. diff --git a/doc/securing-redis.md b/doc/database/redis/securing-redis.md similarity index 100% rename from doc/securing-redis.md rename to doc/database/redis/securing-redis.md diff --git a/doc/api.md b/doc/developer/api.md similarity index 85% rename from doc/api.md rename to doc/developer/api.md index 36530eb..905c5e8 100644 --- a/doc/api.md +++ b/doc/developer/api.md @@ -1,4 +1,5 @@ -# nntpchan jsonrpc api +NNTPChan JSON-RPC API +===================== To enable set the following in `srnd.ini` section `frontend` diff --git a/doc/protocol.md b/doc/developer/protocol.md similarity index 99% rename from doc/protocol.md rename to doc/developer/protocol.md index 00c8928..bb3b8ff 100644 --- a/doc/protocol.md +++ b/doc/developer/protocol.md @@ -1,6 +1,8 @@ Protocol ======== +Documentation of the NNTPChan protocol. + ##Preface NNTPChan is a newsgroup meant to be served on web frontends in an effort to create a decentralized imageboard. Moderation takes place on each frontend itself. Message and image transport is using MIME multipart messages and Base64 as encoding for Images. All messages need to be valid NNTP messages, the transport of messages need to follow NNTP specifications. It is possible to use an existing NNTP daemon like INN or to implement the NNTP sync part as well. diff --git a/doc/extras/configure-newsreader.md b/doc/extras/configure-newsreader.md new file mode 100644 index 0000000..c2ca2e3 --- /dev/null +++ b/doc/extras/configure-newsreader.md @@ -0,0 +1,4 @@ +Configuring your newsreader +=========================== + +WIP. Let's document both Thunderbird and Pan. diff --git a/doc/feeds.md b/doc/feeds.md index efd60ea..faf8dca 100644 --- a/doc/feeds.md +++ b/doc/feeds.md @@ -1,10 +1,11 @@ -# feeds.ini # +`feeds.ini` +=========== -## Peering ## +##Peering In order to actually be distributed, you need another person to sync posts with, otherwise what's the point right? -### Peering over the regular internet with tls ### +###Peering over the regular internet with TLS Requirements: @@ -53,7 +54,7 @@ If Alice owns `nntp.alice.net` and Bob owns `nntp.bob.com` and are both using po overchan.* = 1 ctl = 1 -each side's `certs` directory contains 2 files: +Each side's `certs` directory contains 2 files: * overchan-alice-nntp.alice.net.crt (alice's certificate) * overchan-bob-nntp.bob.com.crt (bob's certificate) @@ -63,7 +64,7 @@ Alice keeps `overchan-alice-nntp.alice.net.key` secret and never shares it Bob keeps `overchan-bob-nntp.bob.com.key` secret and never shares it -### Peering Authentication with passwords ### +###Peering Authentication with passwords adding / removing credentials via the command line: @@ -91,7 +92,7 @@ exmaple feeds.ini: ctl = 1 -### Peering over tor ### +###Peering over Tor Install tor @@ -125,9 +126,9 @@ Then to peer with someone over tor add this to you feeds.ini ctl=1 -## Options ## +##Options -#### You need one connection and one settings block for each connection #### +####You need one connection and one settings block for each connection Here is an example entry in feeds.ini @@ -161,7 +162,7 @@ Proxy settings, straight forward. Supported proxy types are `socks4a` and `none` [them.onion] -nntp synchronization settings +NNTP synchronization settings overchan=1 diff --git a/doc/redis.md b/doc/redis.md deleted file mode 100644 index 37590b1..0000000 --- a/doc/redis.md +++ /dev/null @@ -1,26 +0,0 @@ -# configuring redis database backend - - -0) Install redis - -Redis 3.x or higher is required, [stable release](http://download.redis.io/releases/redis-stable.tar.gz) recommend - -* see http://redis.io/download - - - -1) Configuration - -In srnd.ini the database sections should look like this: - - [database] - type=redis - schema=single - host=localhost - port=6379 - user= - password= - -2) Run the daemon - -* see the [next step](running.md) diff --git a/doc/running.md b/doc/running.md index a7ac3be..ef77f97 100644 --- a/doc/running.md +++ b/doc/running.md @@ -1,10 +1,10 @@ -Running -======= +Running NNTPChan +================ -Once you have [built NNTPChan](building.md) and have done [the initial setup you](setting-up.md) you can start NNTPChan. +Once you have [built NNTPChan](building.md) and done [the initial setup you](setting-up.md) you can start NNTPChan. -You can now start the NNTP server (srndv2) by running: +You can now start the NNTPChan node (srndv2) by running: ./srndv2 run -Now you can check out the web-interface by navigating to 127.0.0.1:18000 (default address - unless you have changed it in your `srnd.ini`) or you can [configure your newsreader](). +Now you can check out the web-interface by navigating to 127.0.0.1:18000 (default address - unless you have changed it in your `srnd.ini`) or you can [configure your newsreader](extras/configure-newsreader.md). diff --git a/doc/setting-up.md b/doc/setting-up.md index b726a0d..4c4127a 100644 --- a/doc/setting-up.md +++ b/doc/setting-up.md @@ -1,5 +1,5 @@ -Configuring NNTPChan server -=========================== +Configuring NNTPChan +==================== This document provides a step-by-step guide to configurin your NNTPChan node. @@ -7,11 +7,13 @@ This document provides a step-by-step guide to configurin your NNTPChan node. You can configure NNTPChan via the web-interface by navigating your browser to http://127.0.0.1:18000. +
+ ###Selecting your data-storage system ![Image 1](http://i.imgur.com/l9iiXxB.png) -First your will be asked what data-storage system you would like to use. We support Redis and PostgreSQL. +First your will be asked what data-storage system you would like to use. We support Redis and Postgres. Below we have configuration information for both Redis and Postgres.
@@ -19,24 +21,26 @@ First your will be asked what data-storage system you would like to use. We supp ![Image 2](http://i.imgur.com/HDp4Ddf.png) -If you have chosen Redis then fill in the fields below: +**First** [Install and Configure Redis database](database/redis/configure-redis.md). +**Then** fill in the fields below: * **Hostname or IP Address** - This is the hostname or IP address of your Redis server (I would run it locally on 127.0.0.1 to be safe). * **Port number** - The port that your Redis server is running on. -* **Password** - Optional authentication password for Redis ([Setting up a Redis password](securing-redis.md)). +* **Password** - Optional authentication password for Redis ([Setting up a Redis password](database/redis/securing-redis.md)).
-####PostgreSQL configuration +####Postgres configuration ![Image 3](http://i.imgur.com/WPXedZB.png) -If you have chosen PostgreSQL then fill in the fields below: +**First** [Install and Configure Postgres database](database/postgres/configure-postgres.md). +**Then** fill in the fields below: -* **Hostname or IP Address** - This is the hostname or IP address of your PostreSQL server (I would run it locally on 127.0.0.1 to be safe). -* **Port number** - This is the port that your PostgreSQL server is running on. -* **Username** - The username for PostgreSQL. -* **Password** - The password for PostgreSQL. +* **Hostname or IP Address** - This is the hostname or IP address of your Postres server (I would run it locally on 127.0.0.1 to be safe). +* **Port number** - This is the port that your Postgres server is running on. +* **Username** - The username for Postgres. +* **Password** - The password for Postgres.
@@ -73,4 +77,10 @@ NNTPChan needs to know the paths to the listed programs on your system. ##Manual configuration (WIP) -FIXME +Check out the following in order: + +1. Setting up data-storage system (choose i or ii) + 1. [Setting up using Postgres](database/postgres/configure-postgres.md) + 2. [Setting up using Redis](database/redis/configure-redis.md) +2. [Setting up NNTPChan system](srnd.md) +3. [Setting up feeds](feeds.md) diff --git a/doc/srnd.md b/doc/srnd.md index 5574a34..784280a 100644 --- a/doc/srnd.md +++ b/doc/srnd.md @@ -1,4 +1,122 @@ `srnd.ini` ========== -`srnd.ini` is the configuration file for your NNTPChan node. +`srnd.ini` is the configuration file for your NNTPChan node. Some configuration can be done initially through the web-interface (for now hopefully) but there is more that can be done by tweaking your `srnd.ini` file. + +Below is the default state of the file. We will explain each section below. + +```` +[nntp] +instance_name=test.srndv2.tld +bind=127.0.0.1:1199 +sync_on_start=1 +allow_anon=0 +allow_anon_attachments=0 +allow_attachments=1 +require_tls=1 +anon_nntp=0 + +[pprof] +enable=0 +bind=127.0.0.1:17000 + +[crypto] +tls-keyname=overchan +tls-hostname=!!put-hostname-or-ip-of-server-here +tls-trust-dir=certs + +[articles] +store_dir=articles +incoming_dir=/tmp/articles +attachments_dir=webroot/img +thumbs_dir=webroot/thm +convert_bin=/usr/bin/convert +ffmpegthumbnailer_bin=/usr/bin/ffmpeg +sox_bin=/usr/bin/sox +compression=0 + +[database] +type=redis +schema=single +host=localhost +port=6379 +user +password + +[cache] +type=file + +[frontend] +enable=1 +allow_files=1 +regen_on_start=0 +regen_threads=1 +bind=[::]:18000 +name=web.srndv2.test +webroot=webroot +prefix=/ +static_files=contrib +templates=contrib/templates/default +translations=contrib/translations +locale=en +domain=localhost +json-api=0 +json-api-username=fucking-change-this-value +json-api-password=seriously-fucking-change-this-value +api-secret=RTZP5JZ2XGYCY=== +```` + +##`[nntp]` + +All NNTP server-related settings. + +####instance_name + +This is the name for your NNTP server. I don't really know what the point of it is, but hey its there (FIXME). + +####bind + +This is where you put the address and port that you would like the NNTP server to run on where `x` is the address and `y` is the port in `bind=x:y`. + +####sync_on_start + +* When this is set to `1` your NNTP server will sync articles with its peers on startup. +* When this is set to `0` then no syncing will take place on statup. + +####allow_anon + +* When this is set to `1` bluh. +* When this is set to `0` bluh. + +####allow_anon_attachments + +* When this is set to `1` bluh. +* When this is set to `0` bluh. + +####allow_attachments + +* When this is set to `1` posters my attach images to their posts. +* When this is set to `0` posters may not attach images to their posts. + +####require_tls + +* When this is set to `1` then any NNTP connection to this server will need to use TLS. +* When this is set to `0` then any NNTP connection to this server will not need to use TLS (but it could? - FIXME) + +####anon_nntp + +* When this is set to `1` bluh. +* When this is set to `0` bluh. + +##`[pprof]` + +All pprof-related settings. + +####enable + +* When this is set to `1` bluh. +* When this is set to `0` bluh. + +####bind + +FIXME diff --git a/srnd.go b/srnd.go index f65f455..83cdbc8 100644 --- a/srnd.go +++ b/srnd.go @@ -17,7 +17,7 @@ func main() { if action == "setup" { log.Println("Setting up SRNd base...") daemon.Setup() - log.Println("Setup Done") + log.Println("\nSetup Completed.") } else if action == "run" { log.Printf("Starting up %s...", srnd.Version()) daemon.Setup()