Category Archives: Software

Postfix with relayhost over stunnel on macOS 10.12 Sierra

I like to have a working mail setup on all machines as this allows to be notified about cronjobs that failed and also to be able to send other notifications that would otherwise be lost. It is also especially useful for things like git send-email or automatically sending GPG signatures with caff to others.

However, mails cannot just be sent from any device and mail servers on the internet usually reject mails from dial-up IPs or public WiFi networks. To fight spam, techniques like SPF have been developed that restrict the mail servers that are allowed to send mails for the domain name used in the From: field. Therefore the best way is to relay all outgoing mail through the mail server that is responsible for your domains.

While most tools also allow you to configure an external SMTP server, it is on one hand tedious to configure it everywhere and on the other hand also insecure if you have to write the username and password for authentication to many user-readable configuration files on your system. Therefore I am running a local MTA on all the computers I administrate to relay mails to a central mail server.

Continue reading

Goodbye Sailfish OS and Jolla

Already in September 2017 I migrated away from Sailfish OS and I am now using an Android phone. For me, this is the first time I left the niche market of alternative phone operating systems to go mainstream. Originally I used a Nokia dumb phone and then in 2009 got the Nokia N900 as my first smartphone with Maemo as OS, which was so much joy, easily extensible due to the open platform, and had a sliding keyboard. After a few years I continued with the Nokia N9 with the now abandoned Meego. The N9 was released shortly before the Nokia phone department was switched to produce Windows phones for Microsoft. However, a group of former Nokia developers formed Jolla, a company producing an alternative phone operating system called SailfishOS.

Continue reading

Upgrading a VM from macOS 10.12 Sierra to macOS 10.13 High Sierra in VirtualBox

For testing purposes, I have a VM in VirtualBox currently runnning macOS 10.12 Sierra. Now that macOS 10.13 High Sierra is in Beta, I wanted to upgrade my VM to this new release. However, this proved to be difficult with the usual ways. This blog post will describe how to upgrade a Sierra VM to High Sierra.

Continue reading

How to run rsync on remote host with sudo

Sometimes I want to transfer files including ownership. This is not possible as normal user as the chown(2) system call requires special privileges, that is: uid == 0. However, I do not want to open ssh access for root, but go with the usual way to elevate my privileges: sudo.

I will go through common solutions presented on the web and explain why these do not work at all without significant modifications on the remote host and then present a working solution using X11-Forwarding that is less invasive.

Continue reading

Backup with duply to Amazon S3: BackendException: No connection to backend

I stumbled across this problem during setting up duplicity backups to a S3 bucket. As it took me quite a while to resolve this, I wanted to document this problem and its solution here. I just hope someone else with the same problem may find this blog post.

I tried to set up duply, a frontend for the backup tool duplicity, to back up to Amazon S3 storage.

The challenge appeared to be that I wanted to do this with the version available in Debian wheezy. The problem described here is probably already fixed in duplicity >= 0.7.0. These are the versions I used:

i duplicity	wheezy-backports	0.6.24-1~bpo70
i duply         stable			1.5.5.5-1
i python-boto   wheezy-backports	2.25.0-1~bpo7

Problem

I added S3 as a target to the duply configuration as documented on various places on the web. However, I always ran into this error message:

$ duply donkey-s3-test status
Start duply v1.5.5.5, time is 2015-03-12 00:03:40.
Using profile '/etc/duply/donkey-s3-test'.
Using installed duplicity version 0.6.24, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. Not GPG_KEY entries in config.
Test - Encryption with passphrase (OK)
Test - Decryption with passphrase (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.10622.1426115020_*'(OK)

--- Start running command STATUS at 00:03:40.984 ---
BackendException: No connection to backend
00:03:41.301 Task 'STATUS' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 00:03:41.301 - Runtime 00:00:00.316 ---

Similar occurrences of this bug are also tracked here: https://bugs.launchpad.net/duplicity/+bug/1278529

Solution

The exception above is highly unspecific and returning such a generic error message is bad style in my opinion. It took me quite a while to find the solution. To make it short, with this snippet from my /etc/duply/donkey-s3-test/conf file I got this to work:

TARGET='s3://s3-eu-central-1.amazonaws.com/.../'
TARGET_USER='...'
TARGET_PASS='...'
DUPL_PARAMS="$DUPL_PARAMS --s3-use-rrs"
# XXX: workaround for S3 with boto to s3-eu-central-1
export S3_USE_SIGV4="True"

Using a shell export in the configuration file is clearly a hack, but it works. In fact, you can also export it to the environment before running duply or set it in the configuration file of the boto library. However, with the former, you do not have to change anything on the duply invocation.

Why does this solve the problem?

I found out that the problem was not reproducible for some people because it only appears in specific regions. I use Frankfurt, EU (eu-central-1) as my Amazon S3 region. According to the documentation, only the newest API V4 is supported in this region:

Any new regions after January 30, 2014 will support only Signature Version 4 and therefore all requests to those regions must be made with Signature Version 4.

The region Frankfurt, EU was introduced after this date. This means this new region only accepts requests with “Signature Version 4” and not any prior version. Meanwhile other regions continue to accept the old API requests.

This kind of setup is complete madness for me. Especially for open source projects with developers all around the globe, this just means that some developers could not reproduce the problem. Who would assume your endpoint region matters?

In fact, the duplicity manual page has a whole section on how European endpoints are different from other locations. Unfortunately, the recommended --s3-use-new-style --s3-european-buckets does not solve this problem. I could not even observe any difference in behavior with these flags.

Apparently, the boto library used by duplicity for access to Amazon S3 supports the new “Signature Version 4” for API requests, but it is not enabled by default. By exporting this environment variable S3_USE_SIGV4=True the library is forced to use “Signature Version 4”.

The specification of the target protocol for duplicity is another peculiarity. Make sure you use s3:// and specify an explicit endpoint region in the URL, as I could not get it work with s3+http:// and also always with the hostname for your region.

Further Investigations

Unfortunately, the duplicity option --s3-use-rrs which is supposed to put the files into the cheaper Reduced Redundancy Storage (RRS) does not seem to do anything and all uploaded files get the standard storage class. Probably I have to maintain my own installation of the latest versions of duplicity and boto to get all the features to work.

Depending on where you are in the world, YMMV.


Edit 2018-05-23: Fixed a typo in DUPL_PARAMS. Thanks to Zedino.