Using Rsync on Android: Getting Syncopoli talking to the rsync daemon

Syncthing has largely taken the FOSS throne of continuous file synchronization on both servers and desktops. When it comes to my phone, however, I can still see a use for discrete, on-demand file synchronization – at least if changes only appear on one end, be it on the phone or the server. And for that purpose, there is really nothing else to turn to than good ol’ rsync.

Syncopoli seems to be the most up-to-date rsync client on Android, seeing as it’s code has at least been updated within the past year. Even though I believe I know how the basics of rsync on the command line, Syncopoli is not the most intuitive of mobile apps and presents some special problems. In the following I will detail both setting up the rsync daemon on the server and getting a Syncopoli setup that matches the server.

This initial setup will use an unencrypted rsync protocol because there were enough things that could go wrong and few enough debugging tools without adding encryption and keys into the mix. Hopefully, I will get on to switching to an SSH-based setup later but seeing as I’m only using it to sync publicly available mp3 files, I’m not too concerned. It does however call for the following…

Disclaimer: This is not a privacy focused setup. The files I’m syncing are not considered private in any way. I’m happy for the world to see that I’m downloading a new episode of Newshour and even – should the worst come to the worst – to share said episode with the world. It is, however, concerned with security. Nobody should be able to upload anything to my server. Nobody should be able to take over my server using rsyncd as a wedge.

October 2021: I wrote a follow-up where I configured Syncopoli to communicate with an OpenSSH server, rather than the rsync daemon. It builds on the understanding from the setup detailed in this post so even if you want to switch to SSH it might still be worth at least reading this first.

Rsync daemon

First, I had grown accustomed to rsync just working as a client without any setup. This is due to the fact that the machines I had been sending files to always have an SSH daemon running which was what was responding to my client. In order for Syncopoli to use the rsync protocol, I had to set up an rsync daemon listening instead.

Fortunately, the rsync daemon is fairly easy to configure, especially if you’ve ever set up network filesharing, like NFS of Samba, before. The configuration file defaults to /etc/rsyncd.conf and follows an ini-like format. First come the global settings, then a section for each file share (or module in rsync parlance). Sections are indicated by the “[Module Name]” header. The bare minimum looks like this

pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log

[MODULE_NAME]
path = /path/to/folder
comment = My Files
read only = true
timeout = 300

The daemon can most easily be started by running the systemd service:

systemctl start rsync.service

and tested from a client linux machine

rsync -r rsync://host.ip.on.lan/MODULE_NAME /local/folder

First and foremost: Testing the daemon is in my view a lot easier from a linux command line than from Syncopoli because of the feedback, reusing commands, faster transfer speeds etc. Only once I’m sure that the server works as I intend, do I start mucking about with Syncopoli.

Secondly, MODULE_NAME in the command obviously refer to the MODULE_NAME in the configuration above, and not the path in the module. Rsync-over-SSH uses absolute paths, rsync-over-rsync uses module names (although you can specify paths under the module using relative path notation).

In Syncopoli (v. 0.5.3) I set my global settings as follows:

• Protocol: Rsync
• Server address: The server’s address on my local network
• Port: 873 (the default for the rsync protocol)
• User: Anything (I haven’t yet restricted the users allowed to sync)
• rsync options: the defaults (-vHrlt), i.e.
  • -v for verbose
  • -H for preserve hardlinks (not that I use them)
  • -r for recursive, i.e. include subfolders
  • -l for copying symlinks as symlinks
  • -t for preserve modification times
  • and at some later point probably --delete so that files that disappear from the server also disappear from the phone

Syncopoli

The I add a Syncopoli profile by clicking the big plus-sign on the main page/profile list. A profile in Syncopoli can match an Rsync module one-for-one – or it can target some subfolder or sync with special options. If I just want a copy of the entire module, I do something like

• Remote to local (rather than Local to remote)
• Profile name: Test (this does not refer to anything on the server side, it’s just for Syncopoli’s list so it can be whatever I want)
• Origins: MODULE_NAME
• Destination: /storage/emulated/0/RsyncFolder (in order to minimise the things that can go wrong I pick a folder on the phone’s storage rather than an external SD card due to permissions)
• Additional rsync options: (left empty)

If I want a subfolder rather than the entire module, I would add the relative path (from the absolute path in the rsyncd module specification) to the Origins setting.

Starting the sync job is done by pressing the profile’s play button. Viewing the daemon’s response and progress of the sync job is done by just clicking on the profile name.

Debugging

When Syncopoli works, it just works. When it doesn’t, well, I may know about or just as likely, I won’t. Here’s a few tips to help figure out what’s wrong.

Verify connection

The Syncopoli settings have an option to “Verify connection”. It seems I’m not the only one to have been confused by this as the following note has been added to the apps README file:

Starting v0.5, you must press “Verify Connection” in settings and verify that your host’s fingerprint matches before you can sync.

https://gitlab.com/fengshaun/syncopoli/blob/master/README.md

The reference to fingerprints make me think that this about matching a private SSH key file to the host. The somewhat misleading error message “Failed to verify host” for a long time made me think that Syncopoli coul not see my rsync daemon. That is not the case. It just means that the non-existant SSH private key doesn’t match the SSH server running on the host IP.

Empty logs

After any operation check the output in Syncopoli and on the server side (the value of the log file setting, usually /var/log/rsync.log. If there are no (new) lines in the server log, there was no connection made. This will usually correspond to empty output on the client, just the date or lines like “Log file not found”. A solution of sorts: Kill Syncopoli, i.e. go to Android’s Settings | Apps | Syncopoli and choose Force Stop. Or just reboot to be on the safe side. In fact, Force Stop should be used after any changes are made server-side or client-side. Once I have gone over every line in my config and found no good reason for the error, it has inevitably been a matter of killing the app and starting over.

Name lookups

The server side logs are filled with lines such as these:

2020/01/12 18:38:26 [27080] name lookup failed for 192.168.1.40: Name or service not known
2020/01/12 18:38:26 [27080] connect from UNKNOWN (192.168.1.40)

This looks pretty bad but is not – necessarily – a show stopper. I believe a name lookup failure did at one point cause the daemon to refuse the connection but the only way to replicate the issue seems to be to use the module specific setting hosts allow to a host name that doesn’t match the client machine. E.g. using any mDNS-like name such as hostname.local results in lines like the following:

2020/01/12 19:29:29 [27842] name lookup failed for 192.168.1.21: Name or service not known
2020/01/12 19:29:29 [27842] connect from UNKNOWN (192.168.1.21)
2020/01/12 19:29:29 [27842] rsync denied on module MODULE_NAME from UNKNOWN (192.168.1.21)

To skip the reverse DNS lookups (PTR) check – only really useful is one is sync’ing named server to named server – I add the following to my global settings:

reverse lookup = no

The result of this is that the PTR attempt is skipped and the connecting machine is referred to as UNDETERMINED rather than UNKNOWN. hosts allow should default to allow any and all machine, regardless of name or IP address, so I don’t know how I would have got denials.

Success

A successful connection (and transfer if any changes were discovered can easily be gleaned from the server logs by the presence of the line

2020/01/12 18:09:00 [27586] building file list

Authorization

Syncopoli supports authentication using the rsync protocol. To tell the truth I’m not sure if there’s much point, seeing as the transfer is not encrypted and so the password should be visible to anybody who is able to sniff the traffic, i.e. a network administrator on your local network and uhm a lot of people on the internets?

For what it’s worth the setup on the daemon side is adding a line to rsyncd.conf:

auth users = my_user
secrets file = /etc/rsyncd.secrets

my_user does not have to correspond to a system user. The user name is looked up in a table in the secrets file, which should have 600 permissions (rw-) and owned by root. The format of the file is as simple as can be

my_user:cleartext_password

Needless to say, the daemon needs to be restarted after any such additions. For Syncopoli this is a global setting, found in Settings (similar to how you can only have one server). Set User to my_user and Rsync Password to cleartext_password. Leave settings, Force Stop and start the app again. It may be worth checking the new setup from the linux command line first, like so:

rsync -r rsync://my_user@192.168.1.30/MODULE_NAME/ /local/folder/

To root or not to root

Rsyncd…

… must run with root privileges if you wish to use chroot, to bind to a port numbered under 1024 (as is the default 873), or to set file ownership.

rsyncd.conf(5)

Which is better for security: A chrooted process run by root or a non-chrooted process run by a non-root user with no other privileges apart from access to the relevant files? I’m guessing it depends on which is more likely to happen: Breaking out of chroot jail or a privileges escalation? All while taking control of the rsync daemon?

My decision to go with a non-privileged user rests on two simple foundations: 1) It’s a less complex setup that I understand (and therefore more secure than something I don’t really know what is) and 2) as a result it’s a lot easier to set up.

The rsync daemon does have some settings (“uid = 1234”) to allow for running file operations as a specific user, rather than root. I figured that would mean that the daemon drops privileges, say after reading the secrets file. Not so. The daemon stays running as root which means that this is more about access (to the files) than security. In hindsight I realize that this is obvious, given that the setting is module specific and not global. So instead I opted to run the entire thing as a dedicated user.

Here’s what I did to make it work. First, I created a custom systemd service file, using systemctl’s inbuilt feature:

 systemctl edit --full rsync.service

This will start me up with a copy of the /lib/systemd/system service file in /etc/system/system. --full means that I’m not writing an addendum to the main file, I’m writing a replacement (using the system one as a starting point). On the minus side, I’m missing out on improvements from updates. On the plus side, I’m a bit safer against surprises from said improvements. My only change is to add two lines to the [Service] section:

User=MyRsyncUser
Group=MyRsyncUserGroup

With this I can run the rsync daemon entirely as a non-privileged user. Which will definitely fail badly unless I alter the config file too, so here goes.

My rsync user needs to have read and write access to the pid, lock and log files. To avoid having to recreate them with each reboot or alter permissions on system directories, I created a var/run and var/log directory in the rsync user’s home directory and set the config file to use those instead.

pid file = /home/MyRsyncUser/var/run/rsyncd.pid
lock file = /home/MyRsyncUser/var/run/rsync.lock
log file = /home/MyRsyncUser/var/log/rsync.log

Next I define the port to use, as port 873 is now inaccessible:

port = 8873

Finally, I need to give the user read access to the /etc/rsyncd.secrets file. It should obviously still be owned by root but I set it’s group to be the rsync users unique group and give read access on the file to that group, i.e.

chmod 640 /etc/rsyncd.secrets

Depending on distribution an rsync daemon may default to try running chroot’ed. It will not check whether this makes sense (i.e. am I even root?) and a non-root user trying to chroot fails badly. So it’s best to make explicit that chroot’ing should not be attempted by adding the following line to each module in /etc/rsyncd.conf:

use chroot = false

Halfway conclusion

I know it’s not the ideal setup but seeing as Syncopoli is not that geared towards helping the user set up a sync job or inform them of what goes wrong when it goes wrong, I figured I wanted to take it one thing at a time.

Hopefully this is useful to others who want a simple, no-nonsense approach to file syncing on an increasingly nonsense platform. And equally hopefully, I will soon switch over to, and report on, using rsync and Syncopoli over SSH.

Using Rsync on Android 2: Switching Syncopoli to SSH from the Rsync protocol

The [Syncopoli] launcher icon is designed by Armin Moradi and distributed under Creative Commons Attribution-ShareAlike 4.0 International Public License

https://gitlab.com/fengshaun/syncopoli/blob/master/LICENSE.icons