Saturday, 12 January 2013

DataMule - My first Android app. :)

I've been toying with various backup solutions for months now. I've tried tahoe-lafs. I tried my own hacky scripts and before that I even had the audacious idea of writing my own P2P backup software from scratch, starting at the low-level RPC layer.. My current approach is to use duplicity and throw the files on Google's cloud storage. The biggest problem with this latest approach is that full backups of my 100GB of data take about 12.5 days of maxing out my ADSL uplink. QoS eases some of the frustrations trying to use the internet while this is happening but doesn't completely remove them.

So DataMule is going to be a simple Android app that includes an SFTP/SSH client and configuration for pairs of WIFI SSIDs.

When you come into range of your "source" SSID, the SD card partition on your phone gets filled up with data copied via SFTP from your source server (until full). When you come into range of your "destination" SSID, the process is reversed and data is copied to the destination machine via SFTP. To complete the data migration, rsync is kicked off on the destination host with flag to delete after copying. The rsync will connect directly back to the source and copy (from the source) to the new file. This will just kick off a file hash verification and either delete the file (if its OK) or fix it if its not (with hopefully minimal bandwidth).

A HDD at work, a HDD at my parents place and an Android app and I'm all set. So far I've got a skeleton app working but its very very rough.

More updates to come!