Federating the Onions

April 23, 2018 - Reading time: 3 minutes

Within Freedombone it has long been possible to view fediverse instances via an onion address. That has applied to GNU Social, postActiv and more recently Pleroma. But this is really just the client to server part of the communications pipeline and federation between instances (server to server) remained exclusively via the clearnet.

A couple of years ago I did do some investigation of whether I could get GNU Social to federate via onion addresses, which would have the advantage of being independent of the DNS and certificate authority systems. There are a few php Tor proxying examples out there on Github, but none of my experients with federating GNU Social via onion addresses worked out the way I had hoped and I expect that fixing this would require a more involved level of php hacking than I'm currently familiar with.

Recently it has become possible to proxy Pleroma through Tor so that the servers can federate using Tor's DNS resolver, so I've added this as the default behavior both for the ordinary version of Freedombone and also the "onion only" version which, as the name implies, only uses onion addresses to access apps. If you're using Freedombone then this is all automatic, but if you're not the changes needed are quite simple.

If you're using Debian 9.x (the current stable) then you may want to install the tor daemon from backports. This will give you access to the shiny new version 3 onion addresses which have better performance and security properties.

apt-get -yq -t stretch-backports install tor

Create an onion address for your Pleroma instance. Within /etc/tor/torrc:

HiddenServiceDir /var/lib/tor/hidden_service_pleroma/
HiddenServiceVersion 3
HiddenServicePort 80 127.0.0.1:8011

And restart tor to generate the address:

systemctl restart tor

To find out what the onion address is:

cat /var/lib/tor/hidden_service_pleroma/hostname

Create an nginx configuration for your site. Something like:

proxy_cache_path /tmp/pleroma-media-cache levels=1:2 keys_zone=pleroma_media_cache:10m max_size=100m inactive=80m use_temp_path=off;

server {
    listen 127.0.0.1:8011 default_server;
    server_name yoursiteonionaddress;

    add_header X-XSS-Protection "1; mode=block";
    add_header X-Robots-Tag none;
    add_header X-Download-Options noopen;
    add_header X-Permitted-Cross-Domain-Policies none;
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

   access_log /dev/null;
   error_log /dev/null;

   root /etc/pleroma;
   index index.html;

   gzip_vary on;
   gzip_proxied any;
   gzip_comp_level 6;
   gzip_buffers 16 8k;
   gzip_http_version 1.1;
   gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript application/activity+json application/atom+xml;

   location / {
       client_max_body_size 15m;
       client_body_buffer_size 15m;

       limit_conn conn_limit_per_ip 50;
       limit_req zone=req_limit_per_ip burst=50 nodelay;

       add_header 'Access-Control-Allow-Origin' '*' always;
       add_header 'Access-Control-Allow-Methods' 'POST, GET, OPTIONS' always;
       add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type' always;
       if ($request_method = OPTIONS) {
           return 204;
       }

       proxy_http_version 1.1;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
       proxy_set_header Host $http_host;
       #proxy_set_header X-Forwarded-Proto $scheme;
       proxy_pass http://localhost:4000;
  }

  location /proxy {
      client_max_body_size 15m;
      client_body_buffer_size 128k;

      limit_conn conn_limit_per_ip 10;
      limit_req zone=req_limit_per_ip burst=10 nodelay;

      proxy_cache pleroma_media_cache;
          proxy_cache_lock on;
          proxy_pass http://localhost:4000;
  }
}

Where in the above case the Pleroma daemon is running on port 4000.

Now edit your secret.exs Pleroma configuration file and add the following line:

config :pleroma, :http, proxy_url: {:socks5, :localhost, 9050}

You will then need to recompile Pleroma.

cd where_you_installed_pleroma
sudo -u pleroma mix clean
sudo -u pleroma mix deps.compile
sudo -u pleroma mix compile

And restart the pleroma daemon.

systemctl restart pleroma

You should now be able to access Pleroma from the onion address and also federate with other instances which also support server to server onion addresses via a tor proxy.


Improving read/write performance

April 22, 2018 - Reading time: 2 minutes

I've been running small computers on EMMC or microSD cards since 2010, so I know that when it comes to reading or writing things to disk performance can be quite sluggish. This is one of the reasons why I usually turn logging off, because anything which might be often writing to disk is going to slow things down.

Recently I was wondering whether there were any software or kernel tweaks which I could do to try to improve microSD performance. I found a few things which may make a small difference, but the overall answer is that there isn't much you can do other than buy the best quality Sandisk or Samsung microSD you can find. Running some benchmarks I found that microSD read speed can vary wildly from about 22MB/s all the way down to 5MB/s, with the variations happening for no obvious reason but probably due to the underlying microcontroller logic on the memory card itself as it does things like wear leveling.

I bought a Cubieboard 2 in 2014 and it has been used (and abused) quite extensively for testing Freedombone image builds. It has the option to connect an external drive via a SATA cable, but I had never tried doing that because I assumed that a secondary power supply would also be needed. It turns out though that isn't the case, so I was wondering whether I could actually boot the file system from SATA. According to the Cubieboard wiki the answer appeared to be no, or at least not directly.

Normally single board computers only boot from EMMC or microSD, but there's a way to work around this limitation. What you can do is have a boot partition on the microSD card which then points to the main rootfs on a SATA connected SSD or hard drive. This is fairly easy to do by modifying the root path within a file called boot.cmd. I modified the freedombone-image command with an extra --sata option to do this, and so long as you copy the resulting image both to the microSD and the SSD then it works.

This is one of those things which is so ridiculously simple that I wonder why I didn't do it four years ago, but I was mostly focussed upon the Beaglebone Black as a platform, Debian support for Cubieboard was extremely basic and there were a ton of much higher priority problems to solve at the time.

The end result is that running on an SSD the disk performance on Cubieboard 2 is at least an order of magnitude faster, and this is quite pleasing since it means that using these A20 based boards for things like NextCloud or Syncthing is a much more attractive proposition.


Tidying up Tor

April 20, 2018 - Reading time: 1 minutes

Something which I noticed recently is that on the rare occasions where the Debian tor package gets updated it's potentially possible to lose the hidden service settings within your torrc configuration file if you make the wrong choice and say yes to use the maintainer's version of that file. Especially if you were running the "onion only" version of Freedombone this could be pretty bad. You wouldn't lose the keys, but it would still be very inconvenient and stop you from accessing apps.

Fortunately, there's a solution to this. The torrc file can define an include path where secondary configurations can be loaded. So this is now what happens in the current version of Freedombone. The configuration automatically gets migrated to use an include path to the settings. So no matter what happens when the tor package gets updated you shouldn't lose your settings or access to your apps via onion addresses.


Freedombone 3.1: Prospects for a better internet

April 16, 2018 - Reading time: 3 minutes

It has been quite a while since the last official release, so it's about time for another. Freedombone 3.1 continues on a Debain 9.x (stretch) base and there have been a few new applications added since last year, the most notable of which are Pleroma and PeerTube. Both of those apps possibly might have a big future if current trends play out the way I think they will.

This release also includes significant improvements to the mesh version, allowing you to change protocols on the fly. Presently there doesn't seem to be any clear winner in the battle of mesh routing protocols, so it makes sense to include the most common ones and have the user decide. The mesh system is now also pure IPv6 and I like to think that this system is a kind of proof of concept for what the internet could become if supporting legacy software and the client/server paradigm wasn't an obstacle.

There has also been a change of logo. The graffiti style logo was used from the beginning and although I still like this logo I wanted something which was more consistent with the ASCII header of source files and the message of the day within the software itself. So the new logo is really just a colored in version of the ASCII logo. An early criticism was that perhaps the logo should be just an icon of some kind, because it's possible that the system will end up being used in non-English speaking areas. I think that's a reasonable concern and although it hasn't been a problem so far it might be worth investing in some new logo artwork in future.

A question I always ask myself when putting out new software is "is this still relevant?". The world of software moves quickly and things which were once important become no longer so as the technological landscape changes. Freedombone is one of those curious cases where it's not me that's aligning with the world but the world that's coming to meet where I am instead. The issues which motivated the creation of this system are becoming more relevant over time, rather than less. Things like net neutrality under threat, censorship, W3C approving DRM, infrastructure centralization and fragility, growing realization of how out of touch Silicon Valley companies are with most people's lives, aggressive demonetization and the end of the idea that advertising can be a "win-win situation" for creators of web content.

Change is obviously needed, but what kind of change? Just "writing to your MP", as Open Rights Group frequently recommends might be necessary sometimes but isn't sufficient. I think the public have to take matters into their own hands and reclaim the internet as a platform for everyday life rather than just as a vehicle to be used cynically to increase the size of Zuckerberg's bank account. Hosting web systems at an individual or community level can be part of that, and although it's not yet consumer grade easy it is becoming more feasible for more people.


Email via Onions

April 6, 2018 - Reading time: 2 minutes

I use org-agenda, the Emacs task manager, as a TODO list and the problem of getting email to work from an onion address has been a remaining very low priority task for the last couple of years. Not many people need this sort of functionality, but as time passes the problems with conventional email get more acute, especially if you are running your own server.

The problems with existing email can be summarized as:

  • You need a domain name, which costs money.
  • You need a TLS certificate. This isn't as much of a problem now as it was a couple of years ago, but LetsEncrypt is becoming a single point of failure.
  • The protocols were devised during the "profdoctor" stage of the internet, when most users were academics and everyone trusted everyone. Security was an afterthought, and the consequence was a massive spam problem.
  • Port forwarding has to be done for NAT traversal. What if you don't control the internet router?
  • Indiscriminate blocking based upon IP address ranges is increasingly a problem.
  • Some ISPs block email ports.
  • Some ISPs force users to proxy outgoing email via their own server, making censorship or MiTM a possibility.
  • PGP/GPG is needed for content security. A lot of people whinge about the unusability of email encryption.

Using onion addresses gets around the above issues. With onion addresses the public key crypto comes for free, so PGP isn't strictly required and the nay-sayers can stop whining. If you're paranoid enough then you can still use it as an extra encryption layer. Using onion addresses also ensures end-to-end security between email servers.

So long as you're willing to put up with a random-looking email address, and your friends are sufficiently geeky, then onion addresses solve a lot of niggly problems.

Recently I've put some effort into making this work on Freedombone and managed to arrive at a solution where you can send email between onion addresses or between an onion address and a clearnet address. Configuring Exim to do this is mind-bendingly tricky, but possible. This is also a sufficiently niche thing that there is not much information available out there, and what information exists is usually either far out of date or just wrong.

Support for onion email addresses will work "out of the box" with a new Freedombone install. There is also an app called bdsmail, which does something similar but using I2P as the transport mechanism. So you can take your pick, whether you're a Tor fan or an I2P fan.


Another Blogging System

March 31, 2018 - Reading time: 1 minutes

The popular Ghost blogging system has been in Freedombone for a while. Recently I was trying to update it using the current Node LTS version (8.9) but not getting very far. The command line app had its option to specify the user account deprecated, and that seemed to be an important feature without which the installation process became a lot more complex.

I was struggling to get the ghost command line to work without a lot of errors and was also thinking that it's 2018 and surely blogging software doesn't need to be this complex to administer. The essence of blogging software is pretty simple, and probably it doesn't require these thousands of javascript dependencies.

So I've decided to remove Ghost from Freedombone for now. Instead I've replaced it with Bludit. Bludit is much simpler and easier to install. It has no database, so moving it from one domain to another or making backups is just copying a directory. The amount of RAM needed is negligible, so it should run even on the most minimal single board computer. It also of course supports RSS via a plugin.

Perhaps Ghost will return in future, but for now I think Bludit is a better option for self-hosting. When you're self-hosting web systems it's not just the bling which matters, but also the practicality of maintaining the system over time and on low cost hardware.

This means there are now two blogging options on the server version of Freedombone - Bludit and HTMLy. Both are databaseless and written in PHP.


About

The blog of Bob Mottram, a Free Software hacker and maintainer of the Freedombone project.

Web site

Email/XMPP: bob@freedombone.net

Matrix: @bob:matrix.freedombone.net