Create a Dropbox-like service with your Raspberry Pi and Sparkleshare

I used to spend a lot of time organizing my personal data, especially when data was distributed between more than one computer or external disks. Moving data between computers, checking the latest version of file duplicates and making backup copies are tedius tasks if you have to do them manually.

The Dropbox approach is really useful for dealing with these things. You only have to place your files in the Dropbox folder and they will automatically be stored on a server and synchronized in all computers.

Sparkleshare is an application that works in the same way, but unlike Dropbox, it lets you configure your own server. This article gives you instructions on how to configure your own “Dropbox-like” service using a Raspberry Pi with an external USB drive.

Preparing your RaspBerry Pi Server

The easiest way to install the software needed in your Raspberry Pi is installing Raspbian using NOOBS. Raspbian has already installed GIT and support for NTFS filesystem which we will use for our external hard drive. If you don’t have them installed you can do it easily with the following commands:

pi@raspberrypi ~ $ sudo apt-get install git ntfs-3g

Follow these steps to setup your server:

1. Create a new user for GIT

We are going to create a new user account for git repository in order to separate the synchronized folder server data from the normal pi user. That will also create a new home directory which will be used as the mount point of our external drive.

pi@raspberrypi ~ $ sudo adduser git

You will need to enter a password for ‘git’ user. You can leave the other fields as default.

2. Setup the external USB drive

As we have created the ‘git’ user we have its home path in ‘/home/git’. Now we are going to setup our external hard drive to mount it on that directory at boot time.

  • Set proper permission to the git home directory:
pi@raspberrypi ~ $ sudo chmod 770 /home/git
  • Connect your external drive to the Raspberry Pi and find the UUID of the external drive. In this case, the external drive (/dev/sda1) UUID is ‘108A2BB4303E9765’.
pi@raspberrypi ~ $ ls -l /dev/disk/by-uuid/

total 0
lrwxrwxrwx 1 root root 15 Oct 4 01:17 04CE-C9AC -> ../../mmcblk0p5
lrwxrwxrwx 1 root root 10 Oct 4 01:17 108A2BB4303E9765 -> ../../sda1
lrwxrwxrwx 1 root root 15 Oct 4 01:17 27c717a6-452e-4e1c-94e7-198aec02097f -> ../../mmcblk0p6
lrwxrwxrwx 1 root root 15 Oct 4 01:17 B4F6-928C -> ../../mmcblk0p1
lrwxrwxrwx 1 root root 15 Oct 4 01:17 f8a1a356-425c-4df3-817d-0fb086997105 -> ../../mmcblk0p3
  • Get the UID and GID for the recently created ‘git’ with ‘id’ command (usually 1001)
pi@raspberrypi ~ $ id git
 uid=1001(git) gid=1001(git) groups=1001(git)
  • Let’s check if the external drive can be mounted with no problems at /home/git (note: if the drive is already mounted you have to umount it before)
pi@raspberrypi ~ $ sudo umount /dev/sda1
pi@raspberrypi ~ $ sudo mount -t ntfs-3g -o uid=1001,gid=1001,umask=007 /dev/sda1 /home/git
If the drive was mounted successfully we are ready to configure Raspberry Pi to do it after every reboot. We only have to edit the file /etc/fstab to add the mount information of the external drive (note: take a backup of fstab file before)
pi@raspberrypi ~ $ sudo cp /etc/fstab /etc/fstab.backup
pi@raspberrypi ~ $ sudo nano /etc/fstab
At the end of the file, add the following line (replace UUID with your own):
UUID=108A2BB4303E9765 /home/git ntfs-3g uid=1001,gid=1001,umask=007 0 0

Now you can reboot your Raspberry Pi and the external drive should be mounted on ‘/home/git’ directory after that.

pi@raspberrypi ~ $ sudo reboot

3. Create a new repository for your synchronized folder

We have to create a git repository for every synchronized folder. You only have to log in as ‘git’ user and use git command to create the new repository under a custom folder:

pi@raspberrypi ~ $ su git
git@raspberrypi:~$ mkdir /home/git/personal_data
git@raspberrypi:~$ cd /home/git/personal_data
git@raspberrypi:~/personal_data$ git --bare init
Initialized empty Git repository in /home/git/personal_data/

Now, our server is ready!

Only one small step remains: adding the authorized ssh key to the server. We will do that after installing Sparkleshare in our computer.

Setup a Windows Client

At this point, we are ready to configure the client software to have a synchronized folder using our self-hosted git server installed in our Raspberry Pi.

1. Download and Install required software

2. Configure cryptographic keys

During Sparkleshare installation, an unique Client ID was created with a private/public cryptographic key pair which will be used to secure the connections between the client and the server, even if they are in our private home network. We need to setup the client with the private key and the server with the public one.

  • Locate the private/public keys in your system. They should be placed in the directory ‘C:\Users\<my_user>\AppData\Roaming\sparkleshare’ and their file names should end with *.key and * for the private and public keys respectively.
  • Convert the private key to the PuTTY format using ‘PuTTYgen’ application.
    • Run PuTTYgen
    • Load the private key (*.key file) by clicking ‘Load’ button.
    • Save private key as private.ppk (note: leave the passphrase blank to avoid being asked every time it is loaded).

PuTTYgen – PuTTY Key Generator

  • Load the saved private key with pageant.exe application on Windows startup:
    • Right-click on pageant.exe application and create shortcut on the desktop.
    • Edit shotcut properties and append the name of the private key file in order to run the “pageant” with the proper key.

Pageant shortcut with the private key appended

  • Run the shortcut and check if the password was properly set.

Pageant Key List

  • Place the shortcut into the startup folder so it will be automatically loaded on Windows startup.(Note: In Windows 8/8.1 you can access the startup folder by clicking with the right button on Windows taskbar icon and running command: “shell:startup”)
  • Finally, on the Raspberry Pi side, we have to create the file ‘/home/git/.ssh/authorized_keys’ (located in the external drive) with the content of the public key file (* When we have more than one computer we will have to place every public key appended in that file. Here we have an example of ‘/home/git/.ssh/authorized_keys’:
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3tW0W93nTa69585VbzEYc5yCwxJmyh0nOZDZnXLIQyhVHcdLl0a3iUBaHQSN5+XkkzeYsgzc50+yZHehd8liWt65wkXKDxptTmQK6xhAe1xWkct55AU8jWlkSaFtW+gk1OYvOY9HWi3cQ7wFE4QXleapXSPvy3fU5x0jNvH4cIuMFLJX32qchlxVdF42GDZTOvNRo/XyRFa7BYV9xVUouTeWzkQGnDFiFegz0cRnSJCMBLKFQbsUAHd9o42C6p15LClSndMvnnimWr7oNsb/5u55o+vEfOMZHpM/4p/x2PTzti7mxIBQHMnJzJdhErWDUw/68GhZ/w92TW4I6IVYtQ== myDesktopPC
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1IGdoSGgGSYDhNNJTB2Ez0FgquNFYnkTT7iE7edF6QP+fM/bSJk7gUWRs+Vvz2jxI2i+9BDLZ/O1oA9aZ4r2XUqlOBKCCBBBIhvERdeV9kJXqusbPUFcJQ9+FZ05uFF8Nf3Tu7tv2FhKAeUm/UvcJ2QRO7OONcUfRYoTY+ikuu6Z0uIggLm77E5NB8zO2F+/O/EDWQV3q9PKGvYb2V2lG2mykCdsqArt4XKnfk/UA4SgvJzwKghORws/vXJQbnspWzrNjN0iEe5AhDHkECm/A9fRdXFYgRDkl3cEqaIB/Ux8dfjne7WP3A9i4ho3Uc/8UcTHOW/KBkFqqJKk0dpZl myLaptop

3. Add the synchronized folder to Sparkleshare

Once we have taken all the previous steps the all that remains is to add the synchronized folder to Sparkleshare.

  • Right-click on Sparkleshare tray icon and select the option ‘Add hosted project…’.

Sparkleshare – Adding a new host

  • Configure the host with the option ‘On my own server’ and set the address and the remote path options:
    • Address: ssh://git@<raspberry_pi_ip>
    • Path: /home/git/personal_data

Sparkleshare – Configuring GIT server

And now you have the synchronized folder in Sparkleshare directory:

C:\Users\<my_user>\Sparkleshare\personal data

Every file you place in that folder will be automatically stored in the Raspberry Pi server, so you are safe from a disk failure. Additionally, if you have another computer, you can also configure the same synchronized folder and keep all your data synchronized.

Obviously, synchronization will take place as long as you are connected to your home network. When there is no connection, all changes are stored locally and updated as soon as you are connected again.

Leave a Reply

Your email address will not be published. Required fields are marked *