Using Groovy with Hudson to send rich text email

Recently I migrated from CruiseControl to Hudson as our CI build server. One thing I found lacking in Hudson was to send rich text email containing information regarding changelist details, test result etc. The email extender plugin do provide few variables to be inserted into the mail but that is very limiting.So I modified the plugin to send HTML email which were generated using Groovy‘s SimpleTemplate.

In the template the AbstractBuild object was exposed as a binding. Using that one can compose email containing some changelist info, test stats etc. Further the Groovy template can be used to write email content in any form, look and feel as one wants. This post would explain how to write a simple groovy script to display build related information

A patch has been submitted to Hudson as REF2175. In this post I would try to explain how this feature can be put to use.

Plugin Configuration

Email-Ext configuration

The updated plugin now provides new fields for adding the mail template. In the plugin template you can add any groovy based script which would then be passed to the groovy template engine. Also you can enable weather the email is of type text/html or text/plain.

All the variables provided by the email-ext plugin earlier are still available for use as bindings

Mail Content

For the mail content we would use the Groovy’s SimpleTemplate. The complete template can be accessed from here. The email generated has following main sections.

• General Information
• Committed changelist
• Test Result summary
• Build logs

Lets see few of these in details.

Collecting Changelist

In the template we first collect the changelist information from the build object. For more details refer to the properties exposed by the AbstractBuild. So here we collect all the changelists from lastSuccesfulBuild. Later we would use this to display the changelist contents in details

<%
def lastSuccesfulBuild = build.previousNotFailedBuild
def failed = build.result != hudson.model.Result.SUCCESS

def currResult = build.result
def prevResult = build.previousBuild?.result ?: null

def consecutiveSuccess = currResult == hudson.model.Result.SUCCESS && prevResult == hudson.model.Result.SUCCESS

def builds = []
def changes = []
def count = 0

if(consecutiveSuccess){
   builds << build
   def changeItems = build.changeSet.items
   count += changeItems.length
   changes += changeItems as List
}else{
    while(lastSuccesfulBuild){
	builds << lastSuccesfulBuild
	def changeItems = lastSuccesfulBuild.changeSet.items
	count += changeItems.length
	changes += changeItems as List
    	lastSuccesfulBuild = lastSuccesfulBuild.nextBuild
    }
}
%>

Displaying the Changelist details

Now we would iterate over the changes to display the details. The groovy code is highlighted below. In my case the SCM repository is Perforce. So the changelist object is PerforceChangeLogSet and hence the properties accessed are exposed by it

 <tr>
    <td><b>Modifications since last  successful build:&nbsp; (${count}) </b><hr size="2" width="100%" align="center"/></td>
   </tr>
   <tr>
   	<td>
    	<table width="100%" border="0">
        <% changes.each { item ->
	    def cl = item.change
	%>
          <tr>
            <td style="font-size:10pt; font-family:Arial, Helvetica, sans-serif">
              <a href="http://perforceserver/cgi-bin/p4describe.pl?@${cl.changeNumber}">${cl.changeNumber}</a>
                by ${cl.user}@${cl.workspace} on ${cl.date}</td>
          </tr>
          <tr>
            <td><pre style="font-size:8pt; font-family:Arial, Helvetica, sans-serif">${cl.description}</pre></td>
          </tr>
          <tr>
            <td style="font-size:10pt; font-family:Arial, Helvetica, sans-serif">
             <span style="font-size:10pt; font-family:Arial, Helvetica, sans-serif; "> Affected files:</span>
            	<ul style="font-size:8pt; font-family:Arial, Helvetica, sans-serif">
                <% cl.files.each{ fileEntry -> %>
                  <li>[<a href="http://perforceserver/cgi-bin/p4browse.pl?@diff+${fileEntry.filename}+${fileEntry.revision}+{@action}">Diff</a>]
                  [<a href="http://perforceserver/cgi-bin/p4browse.pl?@filelog+${fileEntry.filename}">History</a>]
                  ${fileEntry.action} ${fileEntry.filename} (${fileEntry.revision})
                  </li>
             	<% } %>
      		   </ul>
               <hr size="1" width="80%" align="left" />
            </td>
          </tr>
         <% } %>
	    </table>
    </td>
   </tr>

This results in following output

Test Result display

I had a requirement to depict the test results in the mail. The result should display the testcases which have started failing seperately from the previous failing test.(I know all test should pass but in reality few test continue failing due to some reasons or other). So here we iterate over current test results and previous test results to determine the delta. Also we collect some data to create a link for the failing test

Segregating the test results in such a way brings the required attention to the test cases which had started failing

  <%
	if(build.testResultAction) {
	    def rootUrl = hudson.model.Hudson.instance.rootUrl
		def testResult = build.testResultAction
		def jobName = build.parent.name

		def lastBuildSuccessRate = String.format("%.2f",(testResult.totalCount-testResult.result.failCount)*100f/(testResult.totalCount))
		def lastBuildDuration = String.format("%.2f",(testResult.result.duration ))

		def startedPassing = []
		def startedFailing = []
		def failing = []

		def previousFailedTestCases = new HashSet()
		def currentFailedTestCase = new HashSet()

		if(build.previousBuild?.testResultAction){
			build.previousBuild.testResultAction.failedTests.each {
				previousFailedTestCases << it.simpleName +"." + it.safeName
			}
		}

		testResult.failedTests.each{tr ->
			  def packageName = tr.packageName
			  def className = tr.simpleName
			  def testName = tr.safeName
			  def displayName = className+"."+testName

			  currentFailedTestCase << displayName
			  def url = "${rootUrl}job/$jobName/lastBuild/testReport/$packageName/$className/$testName"
			  if(tr.age == 1){
				startedFailing << [displayName:displayName,url:url,age:1]
			  } else{
				failing << [displayName:displayName,url:url,age:tr.age]
			  }
		}

		startedPassing = previousFailedTestCases - currentFailedTestCase

		startedFailing = startedFailing.sort {it.displayName}
		failing = failing.sort {it.displayName}
		startedPassing = startedPassing.sort()
   %>

This data is then later displayed in list form as shown below

Conclusion

Using Groovy for generating the email content provides considerable flexibility. It exposes the Hudon data model (which is quite neatly designed) and makes it easy to extract information from it. Further it reduces the requirements to modify the plugin code considerably.

I find it more convenient over the xsl based approach used by CruiseControl. Below is a snapshot of the complete mail send to the users.

Update

I have uploaded the compile hpi file here. Just download it and then add it to your Hudson setup at Hudson -> Manage Hudson -> Manage Plugins -> Advanced

Learning from the source

Learning is an important part in life of Software developer. With fast changing technology one has to keep oneself updated of current happenings. When I took this field as my carrier around two years ago I did not had any formal education in computer science. For me programming was a hobby since college days. Luckily the company I joined had a great belief in open source technology and I got a first hand experience with them. With that started my learning.

The open source project are a treasure mine for any body to learn how to do things and write code. How to follow best practices and avoid errors. I have gone through code of many open source project and each time I got to learn something from them. Here I would mention few of the projects that I learned from which may help new commers in Java to start with

1. Spring – This is a fantastic piece of code. It was the second open source project (after Struts) that I used and explored (and still exploring !!). I would recommend any newbie to go through Spring code base. What it taught me was invaluable…
   • Writing comments – Its code is highly organized and documented. The javadocs are complete and you get to know how to document the facts. And really it improves readability. Remember that code is written once and read much more.
   • Organizing packages – How to structure your code. Here is excellent talk on this aspect by Juergen Hoeller.
   • Template Pattern – Spring uses Template Pattern heavily for all Transaction,JDBC handling. Wait what is a pattern !!. I saw the first reference to that in Spring doc and then I realized there usefulness and later I studied many more
   • Proxy pattern and AOP – Initially Txn management looked a magic to me. I always wondered how it gets done transparently. Again the code answered my query and I came across two great concepts AOP and Proxy pattern. It further taught me what a good framework is.
   • Dependency Injection – Frankly speaking this was strange for me for the first time. I had no experience of working with EJB’s till now so had no clue of what service locator pattern is. And why the code cannot be tested easily without it. It was a bit later I realized its importance when I worked with some older code and found how difficult it is to test your code without getting the whole application up
   • Test cases – Spring test cases taught me how to write true test cases. Those test which have real assert statements and not just to increase the coverage. How to take advantage of DI to its fullest.
   • Thread Locals – How the Spring know about transaction status when you call a method!! Where the state is maintained. Thread local can provide a very neat solution for such kind of problems where state has to be maintained across method calls
2. Hibernate– This one was tough one but given the complexity of orm you cannot blame it also. Though code wise its not of that good quality but solution wise its cool. By clearity I mean comments and documentation.I always wondered how it generates those queries and performs on demand loading of collections. From it I learned about
   • CGLIB and ASM libraries – The magic behind the power of Hibernate lies in runtime code genration. Going through its code I came across this aspect of Java. And that forced me to understand about what a bytecode is. I always knew that Java is a bytecode based language but never understood what a bytecode means. Now these libraries can generate class bytecodes at runtime. So they are a powerful tool to solve few classes of problem
   • ANTLR – The HQL language that is used with Hibernate has a parser for it. Again I realized that parser can be used for such thing also. Till now I thought they were only meant to do something with the code only.
   • Hibernate in Action – If someone has to understand Hibernate this is a must. It would teach you much more beyond Hibernate and explains what ORM tries to solve. What are transactions and what are domain objects.
   • Cache – Hibernate can cache the domain objects transparently. And thus I understood the importance of cache and there use
3. ACEGI– I came across this project while working on a security framework. It uses Spring configuration heavily and followed same practices a s Spring. But what i learned from it was how to design your code. Whats the use of Interfaces and how your design should be open to extension and not to modification. It reminds me of building blocks where you assemble your structure by assembling small pieces. Similarly it provides you all the ingredients and its upto you to assemble them. If you do not like a particular part replace it WITHOUT modifying the framework code.Initially it may seem overwhelming but once you get past the original learning curve you would get the big picture. Security is a complex domain and with a solution like this you can easily tackle any of your product requirement
4. JDK – This is a real big mine and has lot more to offer.And With the code available out there you cannot ask for more. I have not seen the code for most but the one I got hooked into was the Concurrent package. For me threads and concurrency were like advanced concepts referred in last section of any Java book. And I never mustered enough courage to understand them. I read about the Concuurent package in some blog and then explored this alien area for the first time. Luckily I got to know of a book called Java Concurrency in Practice . It cleared many of my doubts in this area but still few remained. But the code for concurrent package cleared most of them Locks,Buffers ,Non blocking data structures etc. You might not use them but understanding how they work by seeing the code really helps. Further the documentation for this package is excellent and explains clearly what to expect and how to use.

The list can go on and on as there is no end to the learning. I tried to list few of the code bases I explored. For any person with few years of experience these things might look obvious but for a new person and especially from non cs background they are the initial steps. Java land is a big one and its churning out things at a fast rate. Some of the open source stuff would amaze you with what they can do. For any new person starting may seem a difficult thing at first. But slowly going through such projects he can understand how things work. With frameworks abstracting out most such things it becomes more important to know such things. As till they work fine you do not have a problem but when they went wrong it would be you who has to solve the issue.

There are many projects and technologies which are on my todo list and I am not getting time to invest. I would list few as a reminder for me

• Terracotta – It can share a normal POJO across multiple jvm and that to without code change. Power of AOP and a innovative way to manage the delta in object changes. It remain on top of my list
• Grizzly – Use NIO for non blocking connections. It can significantly increase the scalibilty of any multi threaded application involved in network IO. NIO is still a enigma to me. My concepts are still not clear on non blocking and blocking operations

So from where do people learn in Java land??

Maven2 Hibernate Plugin and Spring

Continuing my explorations on Maven I had a requirement to generate the database schema using Hibernate tool feature. And maven had a plugin readymade for this. The maven plugin expects the hibernate configuration through the config file but we were using Spring’s LocalSessionFactoryBean to maintain the Hibernate configuration.

Stuck with that I downloaded the source which is scattered over multiple sub modules. Browsing through that I saw few annotations like
@component role=”org.codehaus.mojo.hibernate3.configuration.ComponentConfiguration” over some attributes. Now that was something new and I had no idea what it was meant for. Searching about it I found ( should have known) that Maven internally uses Plexus which is an IOC container similar to Spring.

That gave me the only hope of using the plugin without modifying it and not wasting much time. The plexus container works on the concept of Role. Let me explain that a bit with maven plugin. The plugin uses an interface ComponentConfiguration for retrieval of the Hibernate configuration from various sources like config file,annotation etc. Thus in theory I can plugin my own implementation which gets the config from Spring container. Now the question was how to inject it.

The Plexus container works on the concept of Role. Have a look at this 5 min tutorial at there site. Here is the interface for the ComponentConfiguration

public interface ComponentConfiguration {
    String ROLE = ComponentConfiguration.class.getName();

    Configuration getConfiguration( ExporterMojo exporterMojo )
        throws MojoExecutionException;

    String getName();
}

The ROLE field acts as a key for the container.The mapping of implementation to the role is done through an xml located at META-INF/plexus/components.xml inside the jar. So here was my mapping

<component-set>
  <components>
    <component>
      <role>org.codehaus.mojo.hibernate3.configuration.ComponentConfiguration</role>
      <role-hint>springconfiguration</role-hint>
      <implementation>example.hibernatetool.SpringComponentConfiguration</implementation>
    </component>
  </components>
</component-set>

Next comes the questions how the plugin would pick a particular implementation. Here the role-hint comes. The HibernateExporterMojo which all the mojo’s extend has a field

    /**
     * @component role="org.codehaus.mojo.hibernate3.configuration.ComponentConfiguration"
     * @noinspection MismatchedQueryAndUpdateOfCollection
     */
    private List componentConfigurations = new ArrayList();

So when Maven creates the plugin class then Plexus populates componentConfigurations with the implementations of ComponentConfiguration. And thus our implementation also gets injected.Now only task left is the final wiring of all these things through pom.xml

     <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>hibernate3-maven-plugin</artifactId>
        <version>2.0-SNAPSHOT</version>
        <configuration>
            <componentProperties>
                  <implementation>springconfiguration</implementation>
                  <console>false</console>
              <outputfilename>schema.sql</outputfilename>
              <appcontextlocation>
                  classpath:spring-context.xml
                  </appcontextlocation>
            </componentProperties>
        </configuration>
        <dependencies>
           <dependency>
            <groupId>example.plugin</groupId>
              <artifactId>hibernate-support</artifactId>
              <version>1.0-SNAPSHOT</version>
            </dependency>
           <dependency>
               <groupId>org.hibernate</groupId>
               <artifactId>hibernate</artifactId>
               <version>3.2.1.ga</version>
            </dependency>
            <dependency>
                <groupId>org.hibernate</groupId>
                <artifactId>hibernate-annotations</artifactId>
                <version>3.2.1.ga</version>
            </dependency>
            <dependency>
                <groupId>mysql</groupId>
                <artifactId>mysql-connector-java</artifactId>
                <version>5.0.0-beta</version>
            </dependency>
        </dependencies>
      </plugin>

Here though implementation property we tell the plugin to use my implementation for getting Configuration. Then through dependencies I add my class’s jar to plugin classpath. The hibernate jar was required as plugin was picking some wrong jar and causing problems.Thats all that was required.

Execute the plugin through mvn hibernate3:hbm2ddl and get the ddl script generated for the schema.

Hello world!

I have shifted to WordPress now. My previous blog was at jroller http://jroller.com/page/techkriti