File synchronization

One of the obvious choices is Nextcloud. It provides a relatively easy way to put files to a centralized location and fetch them from there. Simple and working. Among many provided services, it has some automatic synchronization means and it’s supported as a storage backend by many mobile applications.

The other option is Syncthing. Its purpose is exactly synchronization. And it looks like a lot of care was put into making it to work transparently under all the possible circumstances. It’s packaged for common GNU/Linux distributions as well as in F-Droid. One can simply install it, run it, attach devices to share data with, define what to share with what and possibly how and that’s it. No central repository or further manual actions are needed, Syncthing does its job in the background.

And finally, there is git-annex. A smart and flexible git-based solution for distributing large files that can be used in many setups, whether these are a bunch of disks and/or Nextcloud and/or whatever else. Probably very valuable and unique in such scenarios.

I have already been using Nextcloud, but only occasionally. In order to use it regularly, I would have to move it to a Debian server. But Nextcloud is not packaged for Debian. The suggested way of using it is to run a full Nextcloud image with all the services, in a container or a virtual machine. Which looks like a maintenance nightmare for the given purpose. Then there is the option of using Nextcloud as a service, but it would be overkill for me and I wouldn’t like sharing my data, whether plain or encrypted, with a third party.

git-annex is very interesting, but not necessarily very good for general synchronization. For instance, synchronization is not that automatic, git-annex assistant is often extremely slow on my PC, git-annex runs on Android only in Termux (which is probably not a real problem but it’s still not that nice as Nextcloud or Syncthing). It’s focused on somewhat different use cases and I’m afraid using it for general purpose synchronization would be far from set up & forget.

Syncthing looked most promising of the three for my needs.

Syncthing is designed to be easy to set up and then doing its work no matter what. But I still applied some tweaks.

I set up my own Syncthing discovery service to not rely on arbitrary servers. I’m not sure whether it’s actually better or worse than using public services (it depends) but I like having things under my own control.

I also set up additional synchronization of some folders with a device that runs all the time, in order to reduce the chance that two devices get out of sync because they are connected to the net at different times. With Nextcloud, there is a central server supposed to be always reachable so it’s not a problem there. Unlike Nextcloud, Syncthing works without a central server. But adding some redundancy doesn’t harm.

I disabled relaying to be sure my setup is working all right and my data don’t travel unnecessarily around the world without noticing it. I can always enable relaying when needed.

Both the web UI and the Android application look nice, user friendly and easy to use. I synchronize four devices with Syncthing. It seems to be working mostly smoothly and as expected so far. The synchronization is working whether I’m connected to a local WiFi or to a mobile network, even although none of the devices has a public IP address. Cool.

Not everything is perfect though. I experienced some problems, which all seem to be related to Android in some way. I’m not sure whether to blame Android or Syncthing. Anyway, none of the problems was serious enough to prevent Syncthing use.

One problem I experienced was that in exactly one of the six pairs of my devices the devices couldn’t see each other. Quite weird and I couldn’t find out the reason. I’ve eventually resolved it by pointing one of the devices in the given pair to a static address of the other device rather than using dynamic discovery. Not ideal, but it fixed the problem and is a good enough workaround at the moment.

I hit some other problems when setting up a one-way synchronization of a large directory to an Android device. The directory was already present on the target device but with a bit different contents. Syncthing could handle it well but Android apparently doesn’t like directory names ending with dots and file names containing certain characters such as question marks or colons. Syncthing reports “permission denied” and “file does not exist” in such cases and it took me quite a while to figure out what the problem actually is.

During the process, I tried deleting some subdirectories on the target device. Then they got mostly re-synchronized but in one case it happened that a subdirectory was missing on the target device and I couldn’t get it synchronized. The source device could see the subdirectory was missing on the target but the target thought everything was in sync. I couldn’t find a way to re-synchronize and I had to copy the given subdirectory manually. Then it was in sync and reported as such on both the ends.

I couldn’t find a way how to use Syncthing for sharing in Android. Syncthing is shown among applications for sharing and it can be invoked from there. Then the target synchronization folder or its subdirectory and the name of the file to store the data to can be selected. But I couldn’t find a way to actually store the data to the selected file.

Setting up synchronization of various folders on my devices using Syncthing helped me get rid of a significant amount of manual copying and/or the need of using a USB cable to copy things to or from my Android devices. My experience with Syncthing has been very positive so far and it opens some new possibilities to utilize my portable devices.