Gerrit London Hackathon May 2013

At the beginning of May 2013 first European Gerrit Hackathon took place in London. It was quite some time ago (more then a month), but in my opinion it is always good to have summary afterwards.

So, as I mentioned it was first Gerrit Hackathon in Europe, organized by Luca Milanessio in London and kindly hosted by ITHR Consulting. Twelve participants come from variety of  countries and industry areas to work together on Gerrit. We had three really productive days full with many interesting discussions, about project future and new improvements, and code sessions with interactive feedback loop (no delays or time zone differences) … just focus on task at hand and proper solution for it 😉

I think that main topic and killer feature of this hackathon was inline editing driven by Marting Fick (Qualcomm), Edwin Kempin (SAP) and Dave Borowitz (Google). As far as I know this is already available in current master branch (2.8-SNAPSHOT) and allow user edit its commit in the browser. By ‘edit’ I really mean editing files in browser and ‘commiting’ them back (of course this will create new patch set). With this functionality you can easily and quickly fix typos/white spaces/comments in code and commit message without fetching given change locally, amending and pushing back. This could save tons of time… but of course it could hit you very hard if you are not careful enough.

Another interesting topic, which actually is not often addressed during such events, was … documentation. Lets be honest, Gerrit documentation is good when you are a contributor/commiter, but for the new comers or end users it is simply unhelpful. Huge thanks to Fredric Luthander (Ericsson) who bring this topic up and did awesome work in this area! … I’m not really good in documentation, and still need to update Gerrit docs about JavaScript and GWT based plugin development hopefully will do it … in few months 😉

Next topic was statistics and some ground work around generating reports from Gerrit. AFAIR Edwin Kempin, David Pursehouse (Sony Mobile), Gustaf Lundh (Sony Mobile) and Emanuele Zattin (Switch Gears) had some discussions how stats can be collected and accessed. AFAIR there was also a (POC (?)) patch send for review that was adding REST service with some basic statistics.

I think that most of Gerrit administrators and contributors doesn’t know what term ‘capability’ means in Gerrit environment. So, ‘capability’ is ‘type of permission’ eg. ‘forge autor’ or ‘label verified’ are core Gerrit capabilities. Unfortunately plugins cannot contribute their own specific capability, this is really painful in case of replication plugin. Which uses ‘start-replicate’ capability defined in core (but not used there) to grant users permission to execute replication. Looks like this awkward situation was somehow painful for David Ostrovsky (independent) since he started working on this topic (together with Dave Borowitz). I know that there was some patches send for review and I hope that in 2.8-SNAPSHOT this problem is sorted out. Also this means that other plugins can contribute theirs own capability and extend this way Gerrit access rights.

There was also continuation of (never ending story) Gerrit multi-master configuration. As usually this topic was brought by Luca Milanesio and Deniz Türkoglu (Spotify) 😉

Deniz Türkoglu during hackathon was also working on ‘blame plugin’ for Gerrit. The idea is to send mails to code-line-authors when somebody changes specific line or code section. AFAIR there was serious problems with Gerrit API that disallow accessing DB out of RequestSope. Hope that this problem will be solved in nearest future and we could enjoy this plugin in community 😉

And finally, last but not least, my main focus area in Gerrit… the WebUI extendability. Together with Luca Milanesion, Emanuele Zattin and David Ostrovsky we tried to make Gerrit more extendable. David was pushing to server side UI extensions and already did some ground work for this, so I’ve picked up this idea and implemented server side extension port for contributing links to Gerrit top menu (code example), then Luca comes and integrated it with GitBlit plugin. But my main goal is to have native UI plugins in Gerrit either in JavaSript, GWT or ClojureScript (everything that compiles down to JavaScript) so few hours later I’ve proposed event based JavaScript API. Right now this is only a concept and I’m looking for feedback about it, currently it only allows to add rows into patch info table (code example).

Last few hours of hackathon I’ve spent on investigating GWT replacements for Gerrit WebUI. After investigating some possibilities I’ve chosen AngularJS and did initial hacking. There is not much to share right now, I can just say that implementing project list page in Angular was really fun and straightforward. But playing with new JavaScript ecosystem was quite pain for me … maybe I’m to Java-ish ;). Currently I have replacement for current GWT based project list page in Angular, but this was the easy part (I think) more difficult would be to integrate this with current Gerrit GWT UI and build system.

As we are in ‘build system’ topic… during hackathon decision was made that Gerrit will give a try to Buck (ant-like, developed in Facebook, similar to Google’s) build system. In Gerrit 2.8 you will not find pom.xml but BUCK file, this transition should make Gerrit development and releasing easier. Gerrit’s Buck srripts can generate Eclipse project configuration files, also use maven repositories for fetching dependencies. I can confirm that with without tons of Maven projects in Eclipse, IDE is more responsible, GWT development is faster and easier… also build time is shorter … but … there are the down  sides as well. Buck is only supporting unix like systems and it is not (yet :)) an industry standard.

OK, I think thats it… I had a great time during this event also during my morning runs in Kensington Gardens. Hope to visit London again. See you all on next Gerrit Hackathon/User Summit 😉

Gerrit User Summit and Hackathon, Eclipse DemoCamp Poznan

Last two weeks (8-23 November 2012) was really exhausting, first Gerrit User Summit and Hackathon in Google HQ, then on Thursday (22nd November) Eclipse DemoCamp in Poznan.

First things first: Gerrit User Summit.

This was two-day event in one of Google’s training center. I was really impressed how vibrant Gerrit community is; to be honest I didn’t expect so may people attending to this event and be so active during presentations. There was people not only from Android-oriented companies but also organizations like WikiMedia and hardware companies like Qualcomm or Garmin. First User Summit day includes presentations about server side plugins and improvements in Gerrit 2.5. Second day was mostly occupied by Prolog presentations, I’m really impressed how powerful Prolog rules are and now they can customize Gerrit workflow (eg. you can force at least two +1 votes before change can be submitted, or prevent author from submitting his own change or choose another submit type base on branch name). This day also includes my presentation about Web UI plugins. I’ve presented how simple it is to generate stub GWT and JavaScript based plugin projects, then how easily they can be build and deployed in Gerrit. Currently UI plugis are pretty limited, they can be loaded into UI and say ‘hello’, nothing more but IMO this is good starting point to further development. My slides can be found here.

Then Gerrit Hackathon begins.

Up to 15 persons was hacking Gerrit starting from 12th November. Most of them was focused on improving existing UI and extending its functionality. Useful features were added like:

  • editing topic branch name and commit message in place,
  • configurable dashboards,
  • customizable top level menus,
  • file level comments

were added. Gerrit get also new white look and feel and most of GWT RPC services was rewritten to be REST. My personal focus was on auth-plugins and together with Colby Ranger (Google) and Luca Milanesio (GerritForge) was working on this topic. During first two days we manage to come with draft API and first prototype implementation for handling authentication during git clone commands (using HTTP and SSH protocols). Then move to porting LDAP based authentication and switching Web UI interface to new implementation, unfortunate this appear to be not so easy. Couple hours before hackathon ends we presented hackish implementation that works (in our simple use case) now we need to polish it and make workable for all different kind of use cases 😉

I want to thank here Shawn Pearce for organizing those two awesome events and Google for hosting us in theirs headquarters. Also many thanks to all great developers that I had opportunity talk and code with during User Summit and Haskaton. Hope to see you all at next Gerrit Hackatahon and User Summit.

Eclipse DemoCamp Poznań

Next on my road map was Eclipse DemoCamp in Poznan, this is almost like a tradition that I’m participating in this event, some times as a attendee, some times as speaker. This time I was presenting short introduction to Gerrit. It went well, I got some questions during presentation and afterwards. It appears that Gerrit is already used in some companies in Poznań also some persons from audience was interested in adopting it in theirs organization.

As usually, DemoCamp in Poznań was a great opportunity to meet and chat with my old Eclipse friends: Krzysztof Daniel, Jacek Laskowski, Łukasz Stachowiak, Jacek Pospychała, Natalia Klimasz and others.

Many thanks for organizers (Łukasz Stachowiak and Natalia Klimasz) for organizing yet another great DemoCamp in Poznań. See you all next time! ;D

I’m joining CollabNet

After more then five years working at NCDC in Szczecin I decided to move on and start something new.

During that five years I learn a lot and worked with extraordinary people! Thanks to NCDC I meet great people at conferences (like GeeCon, 33rd Degree, Confitura) and Java User Group’s meetings. I know this for sure that I won’t be in this place with this knowledge and skills if I didn’t join NCDC back then. Thank you all!

But it’s time to move forward, meet new people and learn new things … since first of September 2012 I’m employed in CollabNet and will be working on Gerrit and TeamForge integration. Also part of my duties in CollabNet would be improving Gerrit, JGit and EGit projects so stay tuned for ours contributions 😉

Oh, and one last thing I’m moving to Berlin …

Huge performance boost for EGit sync-view

For last few days I was working on EGit Synchronize View performance, especially on Workspace presentation model (the Git Change Set is next on my list ;)). My starting point was 1m 40s to compare two linux kernel versions, v2.6.36 versus v2.6.38-rc2. Such result seams to be very good, but you need to know that it was achived on SSD hard drive and comparing to regular HDD it would be much worst (maybe more then 3 or 4 minutes).

What can be improved ? Fist of all, current implementation whenever it is asked for members of particular folder it opens repository* and read data directly from it. Of course there is a cashing mechanism, but it is only on single resource level. Therefore when you are launching synchronization on repository that have about 300 folders, current implementation will create and configure 300 connections* to repository to read data and then will cache it.

So, my idea was to create only one connection*, read all data at once and put them into global cache. This cache will be used whenever any list of members for given folder will be required. This approach gives about 2.5x performance boost to synchronization (from 1m 40s down to 40s). This result looks much better and maybe on HDD this action will take less then 2 minutes … but this isn’t over 😉

Reading members of folder is one thing, but getting information about particular file (it is changed, added or removed and does this change is incoming, outgoing or conflicting) is another. Currently we are reusing default implementation of SyncInfo class from Team Framework. This is really good implementation … when you cannot obtain such information from version control system. In Git  we have SHA-1 for each file and folder version and we didn’t have to compare file contents to check they are similar or not, comparing SHA-1 is sufficient. This should save lots of CPU time, disk IO’s and developer time waiting for synchronization to finish ;).

Now when I already have cache that contains list of all changed resources it was natural thing to add information about change type to it. Then whenever Team Framework need to know change type it can be easily obtained from this cache … no IO’s are needed, no comparison just read from in-memory-cache and return proper value.

I’m sure that you are wondering how fast synchronization can be now … I can only that it is REALLY fast … as you can remember my stating point was 1m 40s, now same comparison will finish in less than 7s!! This means that now synchronization will be 14 times faster then before! What this means for a regular user? Well, it meas that you will get results of ‘Synchronize Workspace‘ action almost instantly.

Unfortunately, mentioned above changes are sill awaiting for review in gerrit, you can grab them from change #3891 and build it locally. I hope this will be included in 1.1 release …

* jgit uses concept of walks (with filters) through repository, but I’ve used more commonly recognized terminology here