Posts Tagged → hackathon

Gerrit Hackathon at Google HQ… next one is coming

Gerrit Hackathon 2015Gerrit Hackathon 2015

As always after the Gerrit User Summit, a Gerrit hackathon took place.
This time it was a five days event (9-11 November 2015), where members of the Gerrit community could work together, fully focused on making Gerrit a better software.
2015 edition gathered 15 participants from various companies like Google, SAP, Sony Mobile, Qualcomm, OpenStack, Axis Communications, Gerritforge and of course CollabNet.
Hackathons are really intensive periods of time for Gerrit project: over 400 patches were merged, three releases (2.11.5 and two release candidates of 2.12) were performed, countless number of open changes and patchsets were pushed for review.
This blog post summarizes work done during that period of time, showcasing new features upcomming in 2.13 and 3.0 release.

Gerrit metrics

Gerrit Metrics in Grafana2Gerrit Metrics in Grafana2

If you are responsible for running mission critical software for your organization, you must know how important monitoring and metrics are. How important it is to get fine grained information about the application performance. It is simply not enough to know whether it is up and running but also what is the overall shape of it.
This kind of information is especially critical when users start complaining “Gerrit is slow”.
From time to time such complaints arrive also to our team in Potsdam, then we use Splunk to analyze the load based on Gerrit logs and give our recommendations how to tweak Gerrit. Based on such cases our Gerrit Performance Cheatsheet was composed meuh7iu.
Starting from Gerrit 2.13 we will have a new tool in our toolbox! Internal Gerrit metrics!
DropWizard Metrics library is used as internal engine. Gerrit exposes over 1300 metrics about crucial internals e.g. http server response time, git receive pack, git counting objects, cache sizes, etc….
What is even more awesome, plugins can report their own metrics using the core API. This way replication plugin for 2.13 will report time taken to replicate repository data to various locations.
One thing is to collect metrics, the other is to store them. For this purpose three new plugins were created: metrics-reporter-elasticsearch, metrics-reporter-graphite and metrics-reporter-jmx. This gives possibility to plug Gerrit in into already existing infrastructure.

Hooks as plugins for core events

There are two ways in Gerrit how one can be notified about git operation related events. One is via event mechanism and another via Gerrit hooks. Both provide almost identical functionality making deciding on implementing one of them harder.

During our hackathon work was started to extract hooks mechanism into plugins that would listen to core Gerrit events.

This work is still ongoing, but once it gets finished one that want to run server side hooks must install the Gerrit hook plugin.

gwtorm can be used from plugins

You may be wondering what is the gwtorm. This is a library written for Gerrit project to access relational databases. It is a lightweight method of connecting your Java application to multiple different DB backends. Initially it was meant to be used only by GWT based applications (hence gwt prefix in its name), but currently it can be used by any Java application.

Why to use gwtorm in plugin? Well if you don’t want to modify Gerrit schema (which is highly discouraged) to store your plugin data and want to support many SQL dialects out of the box, gwtorm is the way to go.

The first plugin that will use this library is gerrit-ci-plugin.

Gerrit 2.11.5 and 2.12-rc

Gerrit releases don’t happen too often. Some time we had to wait long months (and over 1000 commits) to get new stable version of Gerrit. Usually just before the hackathon a release candidate of new stable version is cut from the master branch.

During this year’s hackathon we got three releases! One was a service release for 2.11 (updated release notes) branch containing fixes for javascript clipboard, styling, commit validation error handling.

Apart from service release two release candidates were published for Gerrit 2.12.

Submit whole topic dialog

Gerrit 2.12 changes how patches are submitted to the repository after code review. In all previous versions there was so called “merge queue” which was responsible for submitting patches in the right order. If particular change was submitted but its ancestors were still under review it ended up in a special state “submitted, merge pending”.

In 2.12 changes arranged in a branch (one change depending on the other) can be submitted at once by single click on the submit button on the topmost change.

Additionally new feature called ‘submit whole topic’ was added. It enables submission of changes that share the same topic. This can be done across multiple projects and branches.

One thing that struck us when this feature was presented during Gerrit User Summit was change in the semantic of Gerrit ‘topic’. Before 2.12 topics were only metadata that could have been freely added and removed. Plus there was possibility to search for changes that share the same topic. Starting from  2.12 setting a topic on changes will change how they are submitted. In some rare cases one can submit changes of others or block them because of a change not visible to all is still waiting for being reviewed.

To make submitting more verbose during the transition period, a submit dialog was proposed. It pops up after clicking on the submit button only when changes from the same topic would be submitted. It presents the list of changes submitted in a topic and without it, so that the submitter can choose whether to just merge the change in question or all changes of the same topic.

CI verification

Some time ago a Diffy build bot was introduced to verify changes pushed to gerrit-review.googlesource.com, but after some time it become unreliable, often was simply not verifying because it was not running.
Now there is new verifier in the picture. Based on proven gerrit-trigger-plugin custom REST API pooling strategy and Jenkins tandem. It is kindly hosted by the GerritForge. Long life to new GerritForge CI bot!

NoteDB

NoteDB is the defining feature of Gerrit 3.0. It will replace “conventional” database system and store everything inside git repositories. All the data that is currently stored in SQL DB will be moved to git repositories. Review related information will be stored in the particular repository using git-notes and special refs. User data will be moved to dedicated repository.
During the hackathon further steps into achieving the goal of removing the dependency from SQL DB were performed, some integration tests were fixed plus NoteDB tests were enabled  as a part of verification job.

New Gerrit UI with Polymer

Last but not least, the new Polymer based WEB UI for Gerrit was initially announced and integrated into Gerrit’s build process.

During the Gerrit User Summit, Google has presented the draft of a new Gerrit WEB UI. This time it is written using Polymer framework, which is a new JavaScript UI design framework from Google.

The new WEB UI will be fully written in JavaScript, making it easier for UI/UX designers to modify and faster to develop and compile.

As I mentioned before, during hackathon PolyGerrit project was integrated into standard Gerrit build system. It requires teaching Buck how to deal with javascript and its dependencies.

What is next?

Next is the Berlin Gerrit Hackathon in 2016. We’ve open a poll to gather input from the community about preferable date between 22nd of February and 25th of March 2016. Please participate if you would like to join us and hack Gerrit in Berlin 🙂

Gerrit London Hackathon May 2013

At the beginning of May 2013 first European Gerrit Hackathon took place in London. It was quite some time ago (more then a month), but in my opinion it is always good to have summary afterwards.

So, as I mentioned it was first Gerrit Hackathon in Europe, organized by Luca Milanessio in London and kindly hosted by ITHR Consulting. Twelve participants come from variety of  countries and industry areas to work together on Gerrit. We had three really productive days full with many interesting discussions, about project future and new improvements, and code sessions with interactive feedback loop (no delays or time zone differences) … just focus on task at hand and proper solution for it 😉

I think that main topic and killer feature of this hackathon was inline editing driven by Marting Fick (Qualcomm), Edwin Kempin (SAP) and Dave Borowitz (Google). As far as I know this is already available in current master branch (2.8-SNAPSHOT) and allow user edit its commit in the browser. By ‘edit’ I really mean editing files in browser and ‘commiting’ them back (of course this will create new patch set). With this functionality you can easily and quickly fix typos/white spaces/comments in code and commit message without fetching given change locally, amending and pushing back. This could save tons of time… but of course it could hit you very hard if you are not careful enough.

Another interesting topic, which actually is not often addressed during such events, was … documentation. Lets be honest, Gerrit documentation is good when you are a contributor/commiter, but for the new comers or end users it is simply unhelpful. Huge thanks to Fredric Luthander (Ericsson) who bring this topic up and did awesome work in this area! … I’m not really good in documentation, and still need to update Gerrit docs about JavaScript and GWT based plugin development hopefully will do it … in few months 😉

Next topic was statistics and some ground work around generating reports from Gerrit. AFAIR Edwin Kempin, David Pursehouse (Sony Mobile), Gustaf Lundh (Sony Mobile) and Emanuele Zattin (Switch Gears) had some discussions how stats can be collected and accessed. AFAIR there was also a (POC (?)) patch send for review that was adding REST service with some basic statistics.

I think that most of Gerrit administrators and contributors doesn’t know what term ‘capability’ means in Gerrit environment. So, ‘capability’ is ‘type of permission’ eg. ‘forge autor’ or ‘label verified’ are core Gerrit capabilities. Unfortunately plugins cannot contribute their own specific capability, this is really painful in case of replication plugin. Which uses ‘start-replicate’ capability defined in core (but not used there) to grant users permission to execute replication. Looks like this awkward situation was somehow painful for David Ostrovsky (independent) since he started working on this topic (together with Dave Borowitz). I know that there was some patches send for review and I hope that in 2.8-SNAPSHOT this problem is sorted out. Also this means that other plugins can contribute theirs own capability and extend this way Gerrit access rights.

There was also continuation of (never ending story) Gerrit multi-master configuration. As usually this topic was brought by Luca Milanesio and Deniz Türkoglu (Spotify) 😉

Deniz Türkoglu during hackathon was also working on ‘blame plugin’ for Gerrit. The idea is to send mails to code-line-authors when somebody changes specific line or code section. AFAIR there was serious problems with Gerrit API that disallow accessing DB out of RequestSope. Hope that this problem will be solved in nearest future and we could enjoy this plugin in community 😉

And finally, last but not least, my main focus area in Gerrit… the WebUI extendability. Together with Luca Milanesion, Emanuele Zattin and David Ostrovsky we tried to make Gerrit more extendable. David was pushing to server side UI extensions and already did some ground work for this, so I’ve picked up this idea and implemented server side extension port for contributing links to Gerrit top menu (code example), then Luca comes and integrated it with GitBlit plugin. But my main goal is to have native UI plugins in Gerrit either in JavaSript, GWT or ClojureScript (everything that compiles down to JavaScript) so few hours later I’ve proposed event based JavaScript API. Right now this is only a concept and I’m looking for feedback about it, currently it only allows to add rows into patch info table (code example).

Last few hours of hackathon I’ve spent on investigating GWT replacements for Gerrit WebUI. After investigating some possibilities I’ve chosen AngularJS and did initial hacking. There is not much to share right now, I can just say that implementing project list page in Angular was really fun and straightforward. But playing with new JavaScript ecosystem was quite pain for me … maybe I’m to Java-ish ;). Currently I have replacement for current GWT based project list page in Angular, but this was the easy part (I think) more difficult would be to integrate this with current Gerrit GWT UI and build system.

As we are in ‘build system’ topic… during hackathon decision was made that Gerrit will give a try to Buck (ant-like, developed in Facebook, similar to Google’s) build system. In Gerrit 2.8 you will not find pom.xml but BUCK file, this transition should make Gerrit development and releasing easier. Gerrit’s Buck srripts can generate Eclipse project configuration files, also use maven repositories for fetching dependencies. I can confirm that with without tons of Maven projects in Eclipse, IDE is more responsible, GWT development is faster and easier… also build time is shorter … but … there are the down  sides as well. Buck is only supporting unix like systems and it is not (yet :)) an industry standard.

OK, I think thats it… I had a great time during this event also during my morning runs in Kensington Gardens. Hope to visit London again. See you all on next Gerrit Hackathon/User Summit 😉

Gerrit User Summit and Hackathon, Eclipse DemoCamp Poznan

Last two weeks (8-23 November 2012) was really exhausting, first Gerrit User Summit and Hackathon in Google HQ, then on Thursday (22nd November) Eclipse DemoCamp in Poznan.

First things first: Gerrit User Summit.

This was two-day event in one of Google’s training center. I was really impressed how vibrant Gerrit community is; to be honest I didn’t expect so may people attending to this event and be so active during presentations. There was people not only from Android-oriented companies but also organizations like WikiMedia and hardware companies like Qualcomm or Garmin. First User Summit day includes presentations about server side plugins and improvements in Gerrit 2.5. Second day was mostly occupied by Prolog presentations, I’m really impressed how powerful Prolog rules are and now they can customize Gerrit workflow (eg. you can force at least two +1 votes before change can be submitted, or prevent author from submitting his own change or choose another submit type base on branch name). This day also includes my presentation about Web UI plugins. I’ve presented how simple it is to generate stub GWT and JavaScript based plugin projects, then how easily they can be build and deployed in Gerrit. Currently UI plugis are pretty limited, they can be loaded into UI and say ‘hello’, nothing more but IMO this is good starting point to further development. My slides can be found here.

Then Gerrit Hackathon begins.

Up to 15 persons was hacking Gerrit starting from 12th November. Most of them was focused on improving existing UI and extending its functionality. Useful features were added like:

  • editing topic branch name and commit message in place,
  • configurable dashboards,
  • customizable top level menus,
  • file level comments

were added. Gerrit get also new white look and feel and most of GWT RPC services was rewritten to be REST. My personal focus was on auth-plugins and together with Colby Ranger (Google) and Luca Milanesio (GerritForge) was working on this topic. During first two days we manage to come with draft API and first prototype implementation for handling authentication during git clone commands (using HTTP and SSH protocols). Then move to porting LDAP based authentication and switching Web UI interface to new implementation, unfortunate this appear to be not so easy. Couple hours before hackathon ends we presented hackish implementation that works (in our simple use case) now we need to polish it and make workable for all different kind of use cases 😉

I want to thank here Shawn Pearce for organizing those two awesome events and Google for hosting us in theirs headquarters. Also many thanks to all great developers that I had opportunity talk and code with during User Summit and Haskaton. Hope to see you all at next Gerrit Hackatahon and User Summit.

Eclipse DemoCamp Poznań

Next on my road map was Eclipse DemoCamp in Poznan, this is almost like a tradition that I’m participating in this event, some times as a attendee, some times as speaker. This time I was presenting short introduction to Gerrit. It went well, I got some questions during presentation and afterwards. It appears that Gerrit is already used in some companies in Poznań also some persons from audience was interested in adopting it in theirs organization.

As usually, DemoCamp in Poznań was a great opportunity to meet and chat with my old Eclipse friends: Krzysztof Daniel, Jacek Laskowski, Łukasz Stachowiak, Jacek Pospychała, Natalia Klimasz and others.

Many thanks for organizers (Łukasz Stachowiak and Natalia Klimasz) for organizing yet another great DemoCamp in Poznań. See you all next time! ;D