Category Archives: English

A custom clock face for my kitchen clock made with TikZ in LaTeX

Should we distrust Comodo after issuing a rogue SSL certificate for Windows Live?

About a year ago, I wrote an article why I no longer trust StartSSL. Back then, I said I switched to a paid certificate issued by Comodo under the PositiveSSL brand instead. A reader now brought a recent issue with a Comodo certificate erroneously issued for Microsoft’s Windows Live to my attention and asked whether I would still prefer them over StartSSL.

Arno wrote this comment (link):

Do you still trust Commodo to be more trustworthy than StartCom just because they asked for money to handle revocations? Think twice – a guy from Finland managed to get a valid certificate from Commodo for “live.fi”, (Microsoft Live in Finland), just because he was able to register “hostmaster@live.fi” as his e-mail-address:

http://arstechnica.com/security/2015/03/bogus-ssl-certificate-for-windows-live-could-allow-man-in-the-middle-hacks/

I started to type my answer as a comment as well, but soon I realized my explanation just became too long to be a comment, so I turned it into an article on its own.
Continue reading →

Backup with duply to Amazon S3: BackendException: No connection to backend

8 Replies

I stumbled across this problem during setting up duplicity backups to a S3 bucket. As it took me quite a while to resolve this, I wanted to document this problem and its solution here. I just hope someone else with the same problem may find this blog post.

I tried to set up duply, a frontend for the backup tool duplicity, to back up to Amazon S3 storage.

The challenge appeared to be that I wanted to do this with the version available in Debian wheezy. The problem described here is probably already fixed in duplicity >= 0.7.0. These are the versions I used:

i duplicity	wheezy-backports	0.6.24-1~bpo70
i duply         stable			1.5.5.5-1
i python-boto   wheezy-backports	2.25.0-1~bpo7

Problem

I added S3 as a target to the duply configuration as documented on various places on the web. However, I always ran into this error message:

$ duply donkey-s3-test status
Start duply v1.5.5.5, time is 2015-03-12 00:03:40.
Using profile '/etc/duply/donkey-s3-test'.
Using installed duplicity version 0.6.24, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. Not GPG_KEY entries in config.
Test - Encryption with passphrase (OK)
Test - Decryption with passphrase (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.10622.1426115020_*'(OK)

--- Start running command STATUS at 00:03:40.984 ---
BackendException: No connection to backend
00:03:41.301 Task 'STATUS' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 00:03:41.301 - Runtime 00:00:00.316 ---

Similar occurrences of this bug are also tracked here: https://bugs.launchpad.net/duplicity/+bug/1278529

Solution

The exception above is highly unspecific and returning such a generic error message is bad style in my opinion. It took me quite a while to find the solution. To make it short, with this snippet from my /etc/duply/donkey-s3-test/conf file I got this to work:

TARGET='s3://s3-eu-central-1.amazonaws.com/.../'
TARGET_USER='...'
TARGET_PASS='...'
DUPL_PARAMS="$DUPL_PARAMS --s3-use-rrs"
# XXX: workaround for S3 with boto to s3-eu-central-1
export S3_USE_SIGV4="True"

Using a shell export in the configuration file is clearly a hack, but it works. In fact, you can also export it to the environment before running duply or set it in the configuration file of the boto library. However, with the former, you do not have to change anything on the duply invocation.

Why does this solve the problem?

I found out that the problem was not reproducible for some people because it only appears in specific regions. I use Frankfurt, EU (eu-central-1) as my Amazon S3 region. According to the documentation, only the newest API V4 is supported in this region:

Any new regions after January 30, 2014 will support only Signature Version 4 and therefore all requests to those regions must be made with Signature Version 4.

The region Frankfurt, EU was introduced after this date. This means this new region only accepts requests with “Signature Version 4” and not any prior version. Meanwhile other regions continue to accept the old API requests.

This kind of setup is complete madness for me. Especially for open source projects with developers all around the globe, this just means that some developers could not reproduce the problem. Who would assume your endpoint region matters?

In fact, the duplicity manual page has a whole section on how European endpoints are different from other locations. Unfortunately, the recommended --s3-use-new-style --s3-european-buckets does not solve this problem. I could not even observe any difference in behavior with these flags.

Apparently, the boto library used by duplicity for access to Amazon S3 supports the new “Signature Version 4” for API requests, but it is not enabled by default. By exporting this environment variable S3_USE_SIGV4=True the library is forced to use “Signature Version 4”.

The specification of the target protocol for duplicity is another peculiarity. Make sure you use s3:// and specify an explicit endpoint region in the URL, as I could not get it work with s3+http:// and also always with the hostname for your region.

Further Investigations

Unfortunately, the duplicity option --s3-use-rrs which is supposed to put the files into the cheaper Reduced Redundancy Storage (RRS) does not seem to do anything and all uploaded files get the standard storage class. Probably I have to maintain my own installation of the latest versions of duplicity and boto to get all the features to work.

Depending on where you are in the world, YMMV.

Edit 2018-05-23: Fixed a typo in DUPL_PARAMS. Thanks to Zedino.

App Updates with Hidden Features

2 Replies

There seems to be a recent trend for smartphone apps: hidden features are shipped in app updates without mentioning the new functionality in the changelog. Only later the feature suddenly becomes active without further notice.

Recently Threema added the ability to conduct polls in a group chat. Apparently, parts of the feature were already shipped before the feature was released to everyone. I was able to answer polls on my phone running the Android app, but I was not able to create them. One could argue that maybe their app UI is so extensible that it can display anything as provided by their API. However, once I installed the next app update—mentioning the new poll feature in the changelog—the UI for polls also changed significantly. This is a clear indication that the feature has a UI part which was already included in the previous update, but was hidden from the user until announced publicly to everyone.

Similarly WhatsApp recently added their second screen solution to the popular messaging service. The menu item to scan the bar code on the WhatsApp web site suddenly appeared in my app, without any update. The next update now mentions the feature in the changelog, but apparently it was already shipped with the previous version.

I totally understand the intentions behind these hidden updates. The companies behind the apps want to enable their users to adopt new services immediately without disruption. If Threema is not able to understand a message that starts a new poll, the user will see garbage, or an error message, or a request to update the app. For WhatsApp, users would be asked to scan the bar code, but cannot find the menu item in their app. Both examples demonstrate undesirably user experience they want to avoid.

However, this means any app update might always contain a hidden feature. Something you do not see in the UI yet, but which is included in the code. No changelog mentions this new functionality and unless someone starts to decompile and inspect every app update, nobody will notice it at all. Maybe there was already a planned feature shipped with an app update once, but it was never activated? Nobody will ever know.

For me, this was on Android, where Google basically only reacts to complaints from users to remove misbehaving apps. Did they also get these new features past the app review process for iOS by Apple? They are well-known to be very harsh in their app reviews and easily reject updates not satisfying all criteria. Did these companies tell Apple about the new features before telling everyone else? Would a small company be able to pull off the same thing and ship hidden features?

For a suspicious user this is a scary situation. Maybe they don’t want to continue to use the app with the new feature? The examples above were mostly harmless, but what if the new hidden feature is something that affects your privacy? The blue ticks for read receipts in WhatsApp already caused a lot of uproar from users a few months ago. (Fun fact: as I tried to link some reference for this, I noticed this new feature was not even worth an announcement on their blog. Well, it got enough publicity, anyway).

What if an app suddenly decides to share your current location all the time? Sure, it asked for the permission to get your current location, but it was only using it for another purpose before. You agreed to allow it for the purpose you saw in the UI. Users have to be aware that everything they allow an app to access might suddenly be used for a purpose they have not seen in the UI before—and no review of the app mentioned it.

By the way, the same has already been true for web services for a long time. Updates are inherently provided by the company running the service and you cannot use the old version anymore. You would even only know about the update beforehand, if they gave you a heads-up. Are we also heading to a continuous update model for apps now? Are we just fine with unanticipated changes being applied to the apps installed locally on our devices?

This kind of problem is tied closely to closed source software, because nobody can easily verify what it actually does behind the UI. These recent examples prove one thing to me: if you want full control over what software does on your device, you have to use open source software only.

Yes, the pessimistic haters will say that I had to expect nothing else from closed source apps, anyway. But I will not give up on closed source software now. Frankly, I cannot do that without also giving up a lot of convenience and ease of use at the moment. Nevertheless, I will watch out more closely which permission I hand out to which app. Likewise you should be aware every app on your phone might suddenly use your data and information about you for something completely different, even before the app update changelog announces any new features.

John Murphy – Anonymous Rejected Filmscore