hoshposh's posterous

I have recently been part of a group that has released, SpiffyUI, a new GWT framework as an open source project. One of the things that we have taken for granted in our full-time positions is the infrastructure we have available for any projects we start in our walled garden. As well as version control we are able to associate team collaboration tools like wikis, continuous integration (CI) services, continuous deployment instances and continuous analysis like sonar. We can also publish artifacts to our maven repositories for other projects to consume.

In this article I want to describe how we managed to use various exceptional resources to be able to check-in source code and automatically build and publish artifacts from our project to a public maven repository. Since we are an open source project the resource providers allow us to do this without having to pay money for their services.

Code Hosting

 There are a plethora of version control and code hosting services that we could choose from, though we elected for Google code. It was a close thing between that and github, but the majority of the team were more familiar with subversion rather than git. Google code provides us with a core set of services to allow our project team to work together. Importantly, we get a wiki to start documenting different aspects of our project. We also get an integrated issues area to use for bugs and new feature requests. Finally, there is a way to enable a code review service that people can use to submit patches that the team can review, and it integrates into both the source control and the issue system.

Continuous Integration

 CI was solved by CloudBees. They provide a Jenkins instance for projects hosted on their servers.  As an open source project we are restricted to a 300 build minutes from their m1.small profile, for our build per month. This forces us to be mindful of the frequency of our automated build jobs. We only have access to their essential Jenkins plugins to use in our builds. With all of that in mind, we still get everything we need to build our project, publish the checkstyle results and deploy artifacts to a maven snapshot repository.

We poll the svn repository every 4 hours. As a side note this also serves as the first opportunity for the project team to get informed about any build failures after recent svn commits.

One thing we make use of, to be able to isolate the publishing credentials away from the build files, is a private filesystem area that CloudBees makes available to all of their hosted projects. This allows us to place a custom maven settings.xml file in the private folder, and reference it from the Jenkins build job. Through maven profiles we can use a special set of credentials that identify the cloudbees jenkins build as the source for snapshot deployments.

Maven Repository

 For the maven repository we elected to sign up our project on the Sonatype servers. They allow open source projects to host their release and snapshot maven artifacts on the sonatype servers. They can also synchronize artifacts over to maven central. Their documentation is awesome, and through their JIRA system you can register your project and get team members added as users who can deploy to your project repositories. With this in place projects that are using our snapshot builds will have access to binaries built with the latest code.

Continuous Deployment

The final part of our infrastructure which is not automated at the moment, is our use of Google app engine to deploy a running instance of our sample application. 

Over time this could be built upon to generate a continuous deployment server that could use the latest snapshot artifact to host an integration test application.

As I was reading a book on my kindle and came across a website with supporting source code samples, I thought "wouldn't it be great to be able to send that link to my laptop" :)

I think it could be done using some kind of amazon service that could push received links out to subscribed devices. Maybe the easier way would be to read the book on the laptop using an alternative reader ;)

Hudson has been around for quite some time now, and while it is receiving a lot of attention on some sites, I was recently reminded how useful it can be for local developer sandbox builds.  For the past few years I have replaced the need of invoking the large project builds from the command-line, preferring to use Hudson as a wrapper.  I should note and reiterate that while I use Hudson for its CI capabilities, in this situation I am using Hudson for its ability to invoke and wrapper any build invocation.

Some of the main benefits are

• You get full access to the entire build log (no more scrolling off the terminal window).
• You get historic access to past build logs.
• No longer need to manage project specific environment variables in your terminal; all contained in the Hudson job.
• Display charts for unit tests, checkstyle, findbugs and other analysis tools being employed in your projects right within your Hudson dashboard.
• Everything managed from your browser.

There is one hidden Hudson Job attribute which allows you to setup a custom workspace, rather than go for the default location.  This allows me to point to the working directory for my project sandbox and use it as is, without customizing it for Hudson or place it in a special place.  The warning here is that I would resist the temptation of using Hudson's version control capabilities since Hudson will want to start from scratch in some circumstances and erase the workspace and retrieve everything from version control.

My only other advice is that while you might also be tempted to tweak the build so that it utilizes more Hudson capabilities, unless you are able to roll those up into your projects official build resist that temptation.  As soon as you run into a situation where you are not running the same build as everyone else, then you risk being out of step with the latest build changes.

Good luck!

I just had a thought, that I will investigate and provide further details on. One of the challenges with a Maven infrastructure is ensuring that everyone in your team is pointing to the same Nexus repository, or has entries for various profiles.  Essentially this means managing entries in the settings.xml file.

With Puppet from Puppet Labs, they provide a way to perform configuration management and data center automation using focused DSL.

Idea (ding!)

Establish the Nexus server as a Puppet master, that has configuration settings to manage settings.xml for maven clients.  The types of things to propagate to any puppet clients with a class of developer-machine might be to at least have an entry to use the Nexus server to retrieve all dependency artifacts :)

There is an awesome maven plugin, gwt-maven-plugin, that provides GWT (Google Web Toolkit) project actions.  While it is still in development, with the current version being a SNAPSHOT release, it still appears to be fairly usable.  However if you follow the instructions that are posted for generating a project using their supplied archetype, you will run into problems.

Incorrect directions (at the time of this writing)

mvn archetype:generate \
   -DarchetypeGroupId=org.codehaus.mojo \
   -DarchetypeArtifactId=gwt-simple-archetype \
   -DarchetypeVersion=1.3 \
   -DgroupId=myGroupId \
   -DartifactId=myArtifactId

Correct directions:

mvn archetype:generate \
   -DarchetypeGroupId=org.codehaus.mojo \
   -DarchetypeArtifactId=gwt-maven-plugin \
   -DarchetypeVersion=1.2 \
   -DgroupId=myGroupId \
   -DartifactId=myArtifactId