2007-08-31

Pimping with Groovy

One of the great advantage to the Groovy programming language is that you can take Java libraries and extend them. The formal technical definition of this process when done in Groovy is called "Pimping".


http://groovy.codehaus.org/Pimp+my+Library+Pattern


So why do we care about Groovy?
  • Groovy allows you to access all the power of Java from shell scripts
  • Groovy compiles to Java binary format allowing you to use Groovy from Java
...and now...
  • Groovy lets you pimp

Groovy, the programming language for pimping.

Oh yeah.

2007-08-30

Where do we go now?

Last year Google dropped their SOAP API and today they announced that You Tube launched a new API. That got me thinking.


This Google-hate-on-SOAP was covered at my NFJS conference, the reason cited was the strong typing issue in SOAP.

As I've written SOAP services I've found that you get tied to the POJO or DTO object that you write. This is okay if you are a small shop and you control both end points (read "me") but it becomes a problem when you have 3rd parties working with your services. The problem gets pretty bad after a half dozen versions...

What to do? Well, apparently if you ar a multi-billion dollar organisation you do this:

Google Data API

Which is the google answer to the problems in SOAP. Now if you aren't Google you could do a few different things... you could just use the GData libraries... reverse engineering your own back end for them since they don't release the server side code for this stuff... or you could do what was suggested by the ThoughtWorks consultant at NFJS... you _could_ do loosely typed SOAP services.

Huh?

A loosely typed soap service is a soap service that takes an XML document as its only parameter. That means your method names change from

@WebMethod
public User getUserByComplex(ComplexObject complex)

to

@WebMethod
public XmlDocument getUser(XmlDocument xml)


which doesn't seem all that profound from the method signature but it does introduce a bit of pain. The pain introduced is in the necessity of implementing a strategy pattern...

Strategy Pattern


... in this case you would have a collection of parser or processing engines that would return a result document. The particulars on which result and how we get it are determined by the concrete implementation.

But even this seems a little hackish. But, take a look at this:

stone mind on rest, groovy, and other things

And, I think this sums up where my mind is headed. After spending a lot of time hacking around in the J2EE playing grounds where there are all kinds of horribly stodgy things, conflicting ideas, and just plain crazy ideas... I've come away with a couple of golden nuggets I'll share with you and where I think the technology stack will be in three years.

I don't have an affinity for Python. I don't have a particular love of plain old Java either, but I do have a yen for many tools that Java lovers have. I love their XML handling tools. I love entity manager based persistence engines.

I think Groovy is going to be a major player for shops like mine where we have a need to work with Java and other tool chains too. We need to get things done quickly but we also need to be a bit stodgy at times. Groovy means we can bring in the Java "enterprise" stack without choking on the "enterprise" part.

Grails helps a lot too. It is rapid application development, MVC, Hibernate3, and Spring in a box and you can crank out applications in it once you know what you're doing. The trouble is getting to know what you are doing if you are a PHP programmer. (which we can find... but not JSP programmers)

I think there needs to be a few more Grails books. But I'm adept at figuring things out and I can get by without a book. I'm impressed that Grails has a DWR plugin so that means the best of all possible worlds.

I am looking forward to getting things done with Java, Groovy and Grails in the next couple of months. I think Groovy will take off in the next few years and legitimize the dynamic language on the Java stack. Groovy is just variant enough off of Java to improve development cycle but close enough to Java to keep the barrier to entry low.

2007-08-29

Groovy Grails: Little joys

I'm working on a Groovy Grails project and I've just written a controller that takes a post from a form with only a partial representation of the object. Full representation of the object in the form is easy just do this:


myObj.properties = params
myObj.save()


The properties save to the Database without much fuss and if the data is invalid we'll toss back to the edit view with a bucket of error messages. But, what if I only wanted to update one field in my object from a form... or sub form... or Ajax call?

...take a look at this little slice of groovy:


public class testObj {
Long a
Long b
String c
}

def t = new testObj(a:0,b:1,c:"foo");

t.properties.each({
println it
});

def params = ['a':1,'b':2]
println "-----------"

params.keySet().each({
println it + "->" + params[it]
t[it] = params[it]
})
println "-----------"
t.properties.each({
println it
});



The output looks like this:

metaClass=groovy.lang.MetaClassImpl@157c2bd[class testObj]
class=class testObj
b=1
c=foo
a=0
-----------
a->1
b->2
-----------
metaClass=groovy.lang.MetaClassImpl@157c2bd[class testObj]
class=class testObj
b=2
c=foo
a=1


What just happened? We mapped the values from a Map object onto the fields of an object with out knowing what the fields in the Java object were. Without introspection.

I presume this works with a POJO and if it does I'm going to go have a party tonight because I think web form processing, ETL, and a whole mess of other ORM type activities just got a helluva lot easier in Java land.

Aren't you happy?

2007-08-24

Getting Fit and Groovy

Fitnesse + Groovy + Hudson

Today I'm going to talk about my Continuous Integration environment and Acceptance Testing Framework. We are using Fitnesse to write acceptance tests that run in Fit, Hudson (running ant) to create our builds, and we're writing our Fitnesse Fixtures in Groovy.

The big picture

The Fitnesse Acceptance testing suite paired with the right tools becomes an absolute authoritative documentation of all business processes implemented in the software under test. It becomes not only a tool for validating software but also a tool for quantifying formal and informal business processes. For some companies this can provide a light weight tool for business analysts to understand their business where no formal tool provided any such insight before.

References:
  • Hudson - a continuous integration server that is extremely easy to set up and manage
  • Fitnesse - a stand alone wiki that is a server and wiki all in one which makes life easy for neophyte administrators but harder on experienced admins and developers... yes I know they spelled fitness wrong.
  • Groovy - a revolutionary new scripting language that compiles to Java byte code.
Why should I care?

Hudson: Because continuous integration facilitates testing early and often and prevents software projects from getting into a broken state. You want to know where your problems will be early since fixing unit level bugs is easier earlier in the development cycle rather than later.

Fitnesse: Because acceptance testing should test function of the code independent of presentation of code. We sing the praise of Model View Controller separation in code... why not in test? Business users can understand spreadsheets and fitnesse tests are just wiki spreadsheets. Fixtures are just preambles to calling your re-usable API. So all business features can be encoded into the fitnesse framework leading to automated functional regression testing. That means easy business visibility to programmer's progress and an easy way to update changes in requirements.

Groovy: Because traditional Java Bean code doesn't "facilitate expressing what a program is really trying to accomplish" and that is not something you want to get bogged down by in your fixtures... fixtures are not "real" or "permanent" code! Groovy lets you crank out fixtures quickly so you can focus on the important code that going to get called by that fixture.

Some business process details

Typically what happens in software projects that age is the documentation that expresses what the business thinks the project should do becomes less and less accurate of what the software does do. This happens either because the business expresses a requirement change outside the documentation or because the regression testing system does not look at business level acceptance tests for the system as a whole.

Using continuous integration and an automated acceptance testing solution that expresses requirements in business level language means that all business cases are tested each build. The system does not "drift" from requirements. If the business changes the requirements that is expressed in a way that demonstrates how much "breakage" the change represents. This affords the business an objective way to measure the cost of the change in an unambiguous way.

The Fitnesse wiki becomes the repository for all knowledge about the business rules expressed in the software system. This documentation of a business rule is automatically validated against the system and the code either operates this way or not. There is no disparity between what the business thinks the business rule the software supports is and what the code implements as a business rule.

Making it happen

As of this writing, Hudson and Fitnesse should share the same file system to make writing your ant build scripts easier. Hudson will call the ant build script for your project, I've specified a "fitnesse" target in the script that deploys all the classes for the project into the fitnesse files directory allowing you to run fitnesse tests from the wiki on the code base.

Tasks overview
  • deploy hudson.war into a servlet container... that could be Apache+Tomcat, Jetty, JBoss + Tomcat, or whatever your heart desires as long as it can deploy war files. The servlet container for hudson should not use the same port as fitnesse. Typically users want fitnesse to run on port 80.
  • "deploy" fitnesse to an established and stable directory that ant and hudson can reliably access. You will set up fitnesse to either grab port 80 or catch port 80 forwarded traffic so that fitnesse is easy for your business users to get to.
  • set up your ant build script to compile groovy to class files and to deploy classes into the fitnesse file system for execution by fitnesse wiki pages.
  • tweaking the classpath of the fitnesse wiki pages to include the groovy embeddable jar file.
On deployment

I've posted before on deploying software to Linux boxes so I won't spend any time on that issue here. I should think you can find good tutorials on how to configure Fitnesse and Hudson for your environment. Instead I'll talk about the unique features of my configuration.

The secret sauce

The ant build script has the following
 <property name="fitnesse-dir" location="/usr/local/fitnesse/FitNesseRoot/files/OLA"/>
<taskdef name="groovyc"
classname="org.codehaus.groovy.ant.Groovyc"
classpathref="classpath"/>

You can add a line to a FitNesse wiki page that adds classes to the class path for the test runner. For example on our fitness server we have groovy installed (via symbolic link) under /usr/local/groovy and fitnesse installed under /usr/local/fitnesse.

Hudson runs an ant build script that has a fitnesse-dir property, a groovyc task definition, and a fitnesse target after build.

We set up Groovy tasks like this:

<taskdef name="groovy"
classname="org.codehaus.groovy.ant.Groovy"
classpathref="classpath"/>
<taskdef name="groovyc"
classname="org.codehaus.groovy.ant.Groovyc"
classpathref="classpath"/>



And build the groovy files to classes by doing this:

<groovyc destdir="${build}" srcdir="${src}" classpath="${build}">
<include name="**/*.groovy"/>
<include name="**/*.java"/>
</groovyc>

... instead of calling javac, we are using groovyc so we get java and groovy compiled side-by-side able to work with each other seamlessly.

Next, we set up the fitnesse dir property as:
<property name="fitnesse-dir" location="/usr/local/fitnesse/FitNesseRoot/files/MyProj"/>


So that later we can do this:
<target name="fitnesse" description="Set up for FitNesse tests" depends="build">
<mkdir dir="${fitnesse-dir}">
<delete file="${fitnesse-dir}/proj-fitnesse.jar">
<jar jarfile="${dist}/proj.jar" basedir="${build}">
<jar jarfile="${fitnesse-dir}/proj-fitnesse.jar">
<fileset dir="${build}">
<fileset dir="${test-dir}">
</jar>
</target>


Now on the wiki pages we set the path to include our project jar and the groovy jar...
!path /usr/local/fitnesse/FitNesseRoot/files/MyProj/myproj-fitnesse.jar:/usr/local/groovy/embeddable/groovy-all-1.1-beta-2.jar

There isn't really anything special to get Fitnesse to use Groovy fixtures. Groovy compiles to class files and Fitnesse will look on its class path for class names that match the fixture name. For example if we create a fitnesse table that looks like this:


|MyFixture |
| param1 | param2 | param3 |
| joe | | blogs |
| mary | joesephine | blogs |


Now when you run the test the page will look for MyFixture.class in your jar file. It won't care whether the class is Java or Groovy. If your fixture is a groovy class, you have provided the groovy jar to the classpath and it will just run.

2007-08-21

Sick today & Software Engineering Ignorance

... felt it coming on yesterday. Definitely sick today. For light reading:
Software Engineering Ignorance.

What is the world coming to? I'm really surprised that people are willing to ignore thirty years of accumulated wisdom.

2007-08-20

The future...

I've spent a lot of time the last few months thinking about the future. Which technology should I invest my time in? Which framework will pay off the most? Where will I be in ten years?

Obviously, these are unanswerable questions. Everyone in the IT industry is asking these questions right now. With the second internet wave now fully upon us we are seeing a return of the technical uncertainty that IT outsiders didn't see in the first internet wave.

The fragmentation of the market is only natural and it is only natural to assume that the bigger players in the tool space will likely survive. But, which one will thrive? Who is getting the critical mind-share to prosper?

I think the next year or so will tell us which framework is really going to win. This time we know all the relevant business models for selling software. We know that free software is a vital on-ramp to small companies whether they are for profit or not. We know that the amateur programmer plays a vital role in the software ecosystem building the rough 1.0 for companies that don't have the backing to hire professionals.

Yet companies that stake their new businesses on amateurs only pay a huge technical debt later when they have to deal with success. I personally have seen two businesses fail due to huge technical debts that launched the company rapidly, but later swamped the same company when the software they had developed so cheaply could not scale to handle unbridled success. Their success bankrupted them as they could not handle the increased demand and earned bad reputations that caused their business to plummet.

In both cases I witnessed competitors saw that the new business idea prospered but failed due to technical debt. In both cases a branch company was formed and grew more slowly on a much more solid financial, process oriented, and technical base. The result was the second company does much better and destroys the original.

There needs to be a balance point in the software ecosystem. Businesses need to launch cheaply and quickly and still be able to refactor themselves for great successes. I think groovy has a good chance at meeting this need. It will be rapid to learn and should scale well... and when things get really tight you are ready to fall back to Java and the Enterprise stack.

Is there another programming language that offers an easy on-ramp from neophyte to journeyman out there? Is a tool like Ruby really going to make in roads into banks and cell phones? Is there another path?

I don't know for certain what will happen. But, I've made my choice.

2007-08-15

hear the sound of Groovy

This venerable article at IBM: "Practically Groovy: Reduce code noise with Groovy", explains how software development with Groovy stays compatible with Java and reduces "code noise" also known as non-intrinsic complexity. A measure of code in any language is dedicated to solving problems caused by using that particular combination of language and technology. In short, these problems have nothing to do with anything your users may care about... the goal of Groovy is to reduce the amount of attention spent on those things in a J2EE environment.

2007-08-13

Hibernate3 evil bug

The version of hibernate that I'm using has this bug. I had one query in my code base with " foo =: bar " instead of " foo=:bar " and all my JUnit tests started failing. Remarkable that something so small and silly could destroy an entire code base. There is no warning in the trace to point you toward this syntax... a single space blows up your whole project... evil... just pure evil.

2007-08-11

The rising star: SUSE

Deals like this one with AMD just highlight the business savvy that Novell has brought to the Linux playing field. This isn't nearly as big a deal as the Novell and Microsoft deal that made such a fuss in the community. But, it does play in nicely with my previous post about the SUSE distribution's future dominance.

Installing, administrating, and generally just living with SUSE is much easier than any other distribution. SUSE reproduces their custom GUI interface for system administration faithfully in an nCurses environment. That means that if you have neophyte system administrators you brought over from a Windows shop you can teach them the GUI administration techniques and everything stays the same even when the system is running headless. The same YAST administration tools are present on SLES, SLED, and OpenSUSE.

If redhat would learn one thing from SUSE it should be consistency of administration. Consistent easy administration for neophyte system administrators. Consistent easy administration through all their distributions. SUSE does this well, redhat does not.

I live thirty minutes from redhat's headquarters but I use SUSE.

2007-08-10

Ant: the subant tag

In SVN I have this hierarchy:

trunk/ --+
|
|
|
+ subProjectA/ -+
| |
| +- build.xml
|
|
+ subProjectB/ -+
|
+- build.xml


under trunk I've added a build.xml file and using the Ant 1.6 tutorial I've added this build.xml file:


<?xml version="1.0" encoding="UTF-8"?>
<project default="dist" name="My Project">
<macrodef name="iterate">
<attribute name="target"/>
<sequential>
<subant target="@{target}">
<fileset dir="." includes="*/build.xml"/>
</subant>
</sequential>
</macrodef>

<target name="dist">
<iterate target="dist"/>
</target>

<target name="compile">
<iterate target="compile"/>
</target>

<target name="clean">
<iterate target="clean"/>
</target>
</project>


This presumes all sub projects have a compile, clean, and dist target. I may add a test target as well.

2007-08-09

SLES10 hands down

In my mind SuSE wins hands down. OpenSuSE and SLES10 are the best enterprise Linux distributions our there and I don't even think about RHEL in production.

Why? No other distribution hits this sweet spot of utility properly:
  1. make cloneconfig - beautiful, make a kernel config exactly like your running config without any worries
  2. yast2 +/- gui - easy to learn sysadmin tasks... low training and pain threshold
  3. autoYast - easy to clone installs
  4. LDAP / ActiveDirectory integration - integrate and play nice with Windows domains so easy a newbie could do it!
  5. centralized patching - keep your systems patched from one patch repository
  6. "plays nice with VMware"
  7. profile.local - put your localized custom configurations in *.local files along side the distribution files... end the Gentoo configuration file merge nightmare by keeping your localized configuration files separate.
Linux is Linux, you can do all these things on any distribution you just have to figure out how. With SuSE you don't have to figure any of this practical system administration stuff out.

2007-08-08

My Linux Desktop

In order to get to work I set up the following custom software:

  1. java 1.6
  2. groovy 1.1
  3. grails
  4. eclipse 3.2
  5. jboss 4.2.1
  6. ie4linux
  7. crossover office
  8. beryl
  9. vmware player with a Windows XP image


Everything else I take as default from the distribution. I evaluated distributions and picked one where it was easiest to set up the distribution to work well in a windows dominated environment.

2007-08-07

GenericRulesEngine: an Abstract Class

This is my take on how to implement rules engines in a project. I'm still working a few things out but you get the idea.


package com.vifprogram.tie;

import java.io.InputStream;
import java.rmi.RemoteException;

import javax.rules.InvalidRuleSessionException;
import javax.rules.RuleRuntime;
import javax.rules.RuleServiceProvider;
import javax.rules.RuleServiceProviderManager;
import javax.rules.StatelessRuleSession;
import javax.rules.admin.LocalRuleExecutionSetProvider;
import javax.rules.admin.RuleAdministrator;
import javax.rules.admin.RuleExecutionSet;
import javax.rules.admin.RuleExecutionSetDeregistrationException;

import org.drools.jsr94.rules.RuleServiceProviderImpl;

/**
* This is the base class for you to write your own rules driven things.
* @author shawn
*
*/
public abstract class GenericRulesEngine {

private final String RULE_SERVICE_PROVIDER = "http://drools.org/";
protected String RULE_URI = null;
protected StatelessRuleSession _statelessRuleSession;
protected RuleAdministrator _ruleAdministrator;
private boolean _clean = false;

public GenericRulesEngine() {
super();
RULE_URI = getRuleURI();
}

/**
* Returns the string such as MyClassName.drl
* @return
*/
protected abstract String getRuleURI();

/**
* you could implement just this...
* return AssignmentManager.class.getResourceAsStream(RULE_URI);
* or invent some fancier method of loading the resources...
* @return
*/
protected abstract InputStream getRules();

/**
* runs the cleanup
*/
protected void finalize() throws Throwable {
if(!_clean) {
cleanUp();
}
}

/**
* set up the rules engine
*
*/
protected void prepare() throws Exception {
RuleServiceProviderManager.registerRuleServiceProvider(
RULE_SERVICE_PROVIDER, RuleServiceProviderImpl.class );

RuleServiceProvider ruleServiceProvider =
RuleServiceProviderManager.getRuleServiceProvider(RULE_SERVICE_PROVIDER);

_ruleAdministrator = ruleServiceProvider.getRuleAdministrator( );

LocalRuleExecutionSetProvider ruleSetProvider =
_ruleAdministrator.getLocalRuleExecutionSetProvider(null);
/*
* This is the point at which the rules file is loaded. If you wanted
* the rules file to load from a different location you would change
* this here:
*/
InputStream rules = getRules();
RuleExecutionSet ruleExecutionSet =
ruleSetProvider.createRuleExecutionSet(rules, null);

_ruleAdministrator.registerRuleExecutionSet(RULE_URI, ruleExecutionSet, null);

RuleRuntime ruleRuntime = ruleServiceProvider.getRuleRuntime();

_statelessRuleSession =
(StatelessRuleSession) ruleRuntime.createRuleSession(
RULE_URI, null, RuleRuntime.STATELESS_SESSION_TYPE );

}


/**
* @throws RemoteException
* @throws InvalidRuleSessionException
* @throws RuleExecutionSetDeregistrationException
*/
private void cleanUp() throws InvalidRuleSessionException, RemoteException, RuleExecutionSetDeregistrationException {
_clean = true;
_statelessRuleSession.release();
_ruleAdministrator.deregisterRuleExecutionSet(RULE_URI, null);
}



BTW: I ripped off most of this I'm sure. I just can't remember who from.

2007-08-04

BarCampRDU

I attended my first "unconference" today, BarCampRDU very enjoyable. It is mainly about making connections with other cool people doing cool things. Mainly, I'm not cool... and rather boring... so I was horribly out of place. Anyone can present if they want to and conversations can devolve into a morass rather quickly.

Still, I enjoyed talking about terribly geeky things with other people who aren't from marketing.

2007-08-03

Importing Public Static Methods

Consider the way you get "assertNotNull" into a JUnit4 test. You do it like this:


import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;



The public static methods of the class Assert are pulled into your lexigraphical context and now you can write code like this in your methods:


@Test
public void testStringyGoodness() throws Exception {
String next = null;
assertNull(next);
next = "foo";
assertNotNull(next);
assertTrue(next.equals(foo));
}


And, all of this is relatively simple... but is it good? I guess that the Object Oriented Paradigm is so well established now that we can do things like this in the language and not expect "old skool" programmers to immediately break things with it.

2007-08-02

JUnit and JPA

So, we have a need to test EntityManager driven classes from inside our JUnit tests. How do you set this up? Well, I for one added a testDb persistence unit to my persistence.xml file and then set up this abstract class...


package com.vifprogram.tie.test;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;

import org.junit.After;
import org.junit.Before;

/**
* This testCase object works in conjunction with a test persistence unit you
* define in your persistence.xml. It sets up an entity manager that is aware
* of all the classes in your build path.
*
* Inherit from this class to build unit tests which need an entity manager.
*
* @author shawn
*
*/
public abstract class EMTestCase {

private static final String TEST_PERSISTENCE_UNIT = "testDb";
private EntityManagerFactory emf;
protected EntityManager em;

public EMTestCase() {
super();
}

@Before
public void initEmfAndEm() {
emf = Persistence.createEntityManagerFactory(TEST_PERSISTENCE_UNIT);
em = emf.createEntityManager();
}

@After
public void closeEmfAndEm() {
em.close();
emf.close();
}

}