Automatic backup in Linux

Some time ago I’ve started backing up my laptop hard drive in any case of failure. Until today I was simply connecting external back-up-drive and manually launching mount && rsync command from zsh history. But this requires that always after connecting this special device I need to go to terminal, log in as root then find this back up command and run it.

Recently I was wondering how this process could be improved or even done automatically. I had few requirements:

  • it should be fully automated
  • system should inform me that back up process was started and that it ends
  • this notification should be done as some system pop up in X windows

So I’ve come with a solution that connects udev, DBus notification and awesome wm naughty library … and it is really simple 😉

How it works ? When you connects back-up-device udev will pick up this event, create a symbolic link to /dev/backup, then launch backup script. This script will send a notification through DBus, this event will be received by naughty and displayed as a simple notification in Awesome WM. After sending notification signal the backup script will mount given device and launch rsync command. After successful synchronization backup device will be unmonted and another notification signal will be send to inform you that you can now disconnect it.

How to achieve this ?

First of all we need to figure out UUID of our back up device (I assume that /dev/sdb2 is yours back-up-partition) :

# blkid /dev/sdb2
/dev/sdb2: UUID="97377fed-edaf-407b-9e02-5c6cecd6dceb" SEC_TYPE="ext2" TYPE="ext3"

Then we need to create new udev rule in /etc/udev/rules.d we can call it 99-backup.rules:

ENV{ID_FS_UUID}=="97377fed-edaf-407b-9e02-5c6cecd6dceb", SYMLINK+="backup"
ACTION=="add", ENV{ID_FS_UUID}=="97377fed-edaf-407b-9e02-5c6cecd6dceb", RUN+="backup"

Of course you need to replace my UUID with one you get from blkid command. Those two lines are responsible for creating symbolic link called backup in /dev directory to device with given UUID; second line will launch backup script when device with given UUID is added.

Finally we need to create our backup script and put it into /lib/udev directory. Here is content of this script:

function show_notification() {
	su $USER -c "export DISPLAY=':0.0'; \
		export XAUTHORITY='/home/$USER/.Xauthority'; \
		/usr/bin/dbus-send --type=method_call --dest=org.freedesktop.Notifications \
			/org/freedesktop/Notifications  org.freedesktop.Notifications.Notify string:'' \
			uint32:0 string:'' string:'$1' string:'' array:string:'' array:string:'' int32:$MSG_TIMEOUT"
function backup() {
	show_notification "Backup process started! Do NOT disconnect backup device."
	mount /dev/backup /mnt/backup
	ionice -c 3 rsync -az --delete --exclude=/dev --exclude=/mnt --exclude=/media --exclude=/run --exclude=/proc --exclude=/tmp --exclude=/sys --exclude=/var/log --exclude=/var/run / /mnt/backup
	umount /mnt/backup
	show_notification "Backup process successed! You can disconnect backup device."
backup &

In this file you only need to change USER variable to yours UNIX user name, also you can adjust MSG_TIMEOUT to your needs (-1 means that default value will be taken). Most difficult part in this script was to figure out how to send DBus notification from root to regular user, since all window managers create and listen on theirs own session scoped buses. As you can notice we can force dbus-send command to send signal to proper bus instance by executing this command on behalf of  given user and with exported DISPLAY and XAUTHORITY variables.

Full source code is available on github. Have a nice and pleasant back up’s 😉

EGit Iteractive Adding

So what is “interactive adding” in git? Imagine situation when you change two or three lines in file (this could be also two or three blocks/methods), then you realize that you will want to commit those changes in two (three or more) separate commits.

Here is Al Blue’s description how interactive adding works in native git: Git Tip of the Week: Interactive Adding

What about EGit? Honestly this is currently supported … but it isn’t really straight forward to access. You need to select a file then choose from context menu Team > Compare With > Git Index and then you can easily move changes around, finally when you are happy with both versions (left is working tree and right is one that will be staged/added to index) just save compare editor contents. Version from right hand side will be added to index as it is in this editor.

OK, so now you can easily stage and unstage partial changes … maybe you would also like to partial replace staged changes by version in HEAD? Currently in version 1.1 you can compare version from git index with HEAD version using context menu Team > Compare With > Git Index with HEAD, but you cannot overwrite anything … yet, because (hopefully) in 1.2 such option will be available. Here is patch that adds this functionality (to be honest this is a side effect…).

And what about (my favorite) Synchronize View? Unfortunately EGit 1.1 don’t allow to move any changes in compare editors launched from Synchronize View … but with change mentioned above and with this one both features would be available from Git Commits model (before it was named as Git Change Set model)

Here is a short screen cast that shows how this work:


End of Google Summer of Code 2011

Officially coding period in Google Summer of Code was closed last Monday (August 22nd) , so this is good time to sum up last three months of my work on EGit project.

During that time I’ve manage to contribute 32 commits (29 of them are already merged into master, rest is pending for review). I have also two not finished changes in my local git repository. First is about supporting non-workspace files in Workspace presentation model in sync-view, second is refactoring of current Git Change Set implementation.

Here is detailed list of things that was done during GSoC and are available in current nightly build:

  • improved synchronize wizard
  • Workspace presentation model will refresh after repository change
  • pushing from sync view to multiple repositories is now possible
  • Git Change Set presentation model will be refreshed after workspace change
  • context menu in synchronize view was cleaned up and git actions are shown for multiple selections
  • synchronization on folder level is now possible and it will narrow results just for selected folder (when synchronization is launched for project then results will show changes in whole repository)
  • local changes can be drag and dropped between <working tree> and <staged changes> nodes in Git Change Set presentation model

Performance improvements for Workspace presentation model are still awaiting for review in Gerrit. Some part of performance and memory usage improvements for Git Change Set are also pending in Gerri, I’m currently working on rest of required refactoring in Git Change Set presentation model.

I want to thank my mentor Matthias Sohn for his commitment in GSoC, for reviewing my patches and his feedback. It was a great pleasure working  with him! 😉

Progress on non-workspace files in Workspace presentation model

For last three weeks I was working on including non-workspace files in Workspace presentation model in Synchronize View. This idea can sounds weird since the Workspace presentation model was designed to show only changed files that are in Eclipse workspace … but there are some use cases in which such behavior seams to be reasonable (eg. EGit and JGit projects doesn’t import repository root into workspace only projects are imported and files like master pom.xml, README, LICENCE and so on are left outside workspace).

If you don’t import all data into Eclipse workspace you won’t see any incoming nor outgoing changes. Even more, after merge/rebase actions changes in non-workspace files will magically appear in those resources and can even brake something (eg. massed up in master pom.xml file).

So, what is current status ? First of all I want to point that I’m describing here current situation in my working copy, described here changes currently are not available in public. After hours of debugging, code analysis and rewriting my code I’ve manage to show some indication in synchornize view that there are some non-workspace chagnes. The next step will be to show what particular file was changed and then enable all operations like overwrite, mark as merged, etc.

Here is a screenshot showing current status:

Non-workspace files in Workspace presenation model
As you can see there is home folder in sync results. It even have expand indicator (but after expanding it there is nothing inside). I suppose that on windows this folder will correspond to drive letter, but I’m not sure here.  Now I only need to tell Team Framework that there are some files inside home node …

This seams to be quite easy task, since parent folder is already included in view … but this isn’t that easy. First of all Workspace model is based on IResource‘s with are manageable by Eclipse platform. Of course I can create such resource using IWorkspaceRoot.getFolder() and IWorkspaceRoot.getFile(). Unfortunately non-workspace resources created by those two methods will return theirs first segment (on unix system it will be usually /home) as theirs project. Of course such project doesn’t exist in workspace therefore this node is empty.

Honestly speaking I currently don’t have any good ides how to move things forward … as usually I’ll dive into debugging and code analysis …

Any hints and ideas from the community are kindly welcome!