Groovy related random links

Here's some links in no particular order and for no particular reason...

Groovy + Ant = GANT - Guillec treats us to a Venkat inspired tour of Gant.

Practically Groovy: Of MOPs and mini-languages - an Andrew Glover classic on MOP and a good read. I dusted it off in recently to refresh my memory.

Getting Groovy with with - Jeff Brown drops some Groovy moves on us. Always one for the slick trick that Jeff Brown.

How to write a Spelling Corrector - Rael Cunha shows us a reimplementation of the Norvig spelling corrector. Rael's post is really on Java but still shows some Groovy. Is there a Groovier version?

Some JavaFX, Java, Groovy Examples - Andres Almiray blows minds with his meta-hot JavaFX/Groovy cross over code. Along with his follow up it's a fantastic example of JVM polyglot programming.

... and now for something completely different.

Top Ten Groovy Programmer Exclimations...

10. It doesn't hurt my mind anymore!
9. That's what a Mock Object should be!
8. Why didn't I try this sooner?
7. I get it now!
6. I could use this everywhere!
5. Now *that* is Groovy!
4. Fascinating.
3. Ah ha!
2. *maniacal laughter*
1. Woah.

Go read GroovyMag there's a new edition out now!

Why Debugging is hard...

Groovy Grails and your URL

Getting the URL you are at in Grails is pretty easy. Just use the dependency injected properties inside your controller and some of the nifty Grails built-in helpers.

For example, want to know the application you are running in? Each grails application controller will be able to grab this info from the grailsApplication object magically available to it via the power of dependency injection. That grailsApplication object will hold metadata about the project which is the info stored in the file application.properties...

def warName = grailsApplication.metadata.'app.name'

Want to know your controller? Each controller has the variable "controllerName" already for you to use.

So you could build a back link to your current controller using the request object and metadata like so:

def urlString =  request.scheme + "://" + request.serverName
     + ":" + request.serverPort
     + "/" + grailsApplication.metadata.'app.name' 
     + "/" + controllerName + "/" 

21st Century Network management

Oblong Industries

About once a year I see something like this: http://oblong.com/ and it renews the passion I have for technology changing the way we interact with our machine creations. I only dream that someday I can create a work that is as visceral in its impact. I hope systems like this will one day be everywhere and also not require any UI gadgetry such as VR gloves like the ones shown in the video.

SearchGroovy.org

Just in case you weren't aware. There is a search engine for Groovy Programmers. The GroovySearch.org is a Google custom search engine that is maintained by Jeremy Rayner, Guillaume Laforge and some other guy. (no really... anonymous no idea who it is)

I've been using this search for a while but I've noticed not all Groovy programmers know about it yet. Spread the word!

Fighting Annotation buildup

It was called XML hell. More precisely XML configuration hell. You would spend hours and days fighting with XML configurations that grew in size and complexity until they became a nightmare.

They call it Annotation hell. It is what happens when you get a the same problems as you had in XML hell manifesting in Annotations. The earliest mention of Annotation hell I can find is from 2004 in James Strachan's old weblog where he uses the term "annotation overkill" but is wrongly quoted in other blogs as saying "annotation hell".

There is an Annotation hell. And I've seen it. A good example is here in Eric Redmond's blog Annotation Hell where he posts a method with 10 annotations on one method operating in at least four contexts.

I've seen this before... although admittedly not nearly as bad... and I have called the same thing "Annotation Buildup" it's the cruft you get on a class that you are annotating for too many simultaneous execution contexts. It is a symptom of a design philosophy flaw.

The annotation centric design directly reverses the problem of XML hell by merging everything back into the class. This is good. It centralizes information related to the class into one place. The flaw is that all the configuration piles up in the same places and it isn't clear how the annotations make sense.

Grails solves this problem using Convention over Configuration (CoC) and a GORM Mapping DSL the effect is a much cleaner looking class. GORM can use the JPA if configured properly. And this isn't a bad combination to work in but the real advantage is being able to both centralize configurations for a class and also group them by context.

Convention keeps configuration hell at bay whether it is created by XML or annotations. Allowing a configuration DSL allows you to keep the flexibility of configurations without the XML and by placing the DSL inside the class definition you can keep everything related to a class together. The DSL also have the advantage of keeping related configurations grouped together so that you don't have to stare at a pile of annotations that don't relate to each other.

CoC in combination with in class DSL for configuration can keep annotation buildup at bay and keep away XML and annotation hell.

Groovy is to Java like C++ is to C?

A nearly heretical thought entered my mind. C++ was a proper super set of C. I used to work in C from a C++ compiler for example and the advantage was that I had C with a little bit of syntax freedom. Like the ability to declare variables anywhere in a function. In fact I had forgotten that you couldn't declare variables just anywhere in C during an interview where the interviewer felt like he needed to really grill me on my knowledge of C.

It isn't hard for me to imagine then that Groovy is nearly a proper super-set of Java and is therefore similar in relationship to Java as C++ was to C.

With a little tweaking your tool chain could be set up to see things the same way. It could see Groovy as Java++. The change would aid adoption and aid in developer migration. In fact you can see here (in my own post) that you can write Groovy/Java and not really Groovy/Groovy. Props to Andres pointing out how I'm not really thinking in Groovy yet.

I'm sure I'm not the first person to have this thought but I thought I'd go ahead and voice it here since I've not read anyone else taking this point of view.

Top Ten reasons *not* to buy GroovyMag

Here are some reasons NOT to read GroovyMag...

10. You don't care about the future...
9. Programming is a fad...
8. You fear change...
7. You are a rock, you are an island...
6. The internet is for losers...
5. Nothing will ever top assembly for writing clear and concise code.
4. Bah. Technology.
3. You don't have an email... what is this email thing?
2. The internet is just a series of tubes...
1. You can't read.

Go read GroovyMag!

More fun with Groovy and Reflection API

This time in a TagLib I need to see all the properties of a domain class but I don't want to look at anything that isn't sent to hibernate.

import java.lang.reflect.Modifier
static getFields(obj) {
      def names = []
      def fields = obj.getClass().declaredFields
      fields.each({ field ->
       if(!field.synthetic)
        if(!Modifier.isStatic(field.modifiers))
         if(!Modifier.isTransient(field.modifiers)) {
          names.add(field.name.toString())
                                 }
      })
      return names
    }

... in another method I'll filter out the Closures by checking against Closure.class and the property's class.

Top Ten ways to spot a Groovy Programmer

10. Still addicted to Java but doesn't complain about getters and setters
9. Sunglasses. Really. Sunglasses.
8. More productive and lest angsty
7. Not worried about Java 7
6. Breezes through XML and JSON work
5. not afraid of FP or currying
4. pals around with GinA and likes DSL
3. asks did you mean the BGG, DGG or DGG2?
2. still has lots of boats because the boats are not burned... (ask Scott Davis)
1. reads GroovyMag!

It's Programmable... It's configurable...

How much difference is there between a program and a configuration?

Consider that a program is a set of instructions for a universal Turing machine. If a configuration determines how a program does something... if the configuration becomes sophisticated enough so that it even includes a DSL when does it cross the line from merely a configuration into a programming language.

When does a program become configuration? When does a configuration become a program? What is the difference?

GroovyMag

I'm very happy to see the first edition of GroovyMag published. Please go out and support the Groovy Magazine for Groovy and Grails developers. I'd love to see this magazine get some good circulation! I'm thankful to Michael Kimsal the GroovyMag crew for helping me launch my career in writing!

Get the word out and start getting Groovy today.

It is all UI even API

It is all UI and UI leads us down certain paths.

This is an idea I've had for a long time but it hasn't been posted here before so let me lay down for you the why behind the tag line of this blog: "The power of software is the power of thoughts and ideas."

These are important concepts: The purpose of the UI is to aid the user in thinking about what they are doing inside their machine. The UI should match what they are thinking about as closely as possible.

This is all well and good for users. It gives them pretty looking sites that are easy to use and understand. It leads to ideas about making user customization of content a certain flavor of easy that involves allowing for graphical interaction with the system.

But, the programming language is UI too.

I have recently graduated into a tier of programmers that write programs for other programmers to use. But, I have always known that unless you are building the system from nothing then you are the user of another person's UI. It may be called an API but it is still UI.

Programming languages are UI too, they are UI with a limited audience, developers. Just as EE folks design CPU with a "programmer's model" in mind so too should library developers develop software with application developers in mind.

The best way to do this is through "dog-fooding" which is to say that a developer should be forced to eat their own dog-food. When you develop something you should have to use it. If you can't use it you can't really make well informed decisions about how that product should work.

As a developer I try to get inside the minds of the people who use my products. I try to anticipate what they will do, how they will do it, and what ways they may want to do things differently. I can operate this way because after nearly two decades of working with computers in one fashion or another I am convinced there is no right or wrong intrinsic in the machines themselves.

Any meaning, purpose, or sense of working or brokenness is determined by the user. To make a system that is bug free you have to know the user. The technology therefore must be designed from the user back to the machine.

The problem is compounded by the fact that there are multiple types of users. You can build classifications for each of them profiling what their expectations are and building their view of the system around that. Working for one user is broken for another. To design a truly beautiful system you need to account for as many of your user classes as possible.

Often what system designers and developers leave out of their design consideration is themselves. Developers are users too. The must live with their own design choices over time and are prone to the same flaws that other user classifications are prone to as well.

The reasons technologies succeed and fail are tied up in the intersections of actual functionality and perceived function and value. These are emotional decisions as much as logical ones. Very technically speaking any Turing complete language is capable of any task that any other Turing complete language can do.

So the power we find in expressive software is actually the ability to cleanly express some thought or idea. We are attempting to communicate both to the machine and the user some expression of action, computation, art, or logic that will achieve the goal of the user. The software is most effective when the developer and the user understand each other through the lens of the software.

This is true for your web application, mobile application, programming language, or your developer API.

The web framework developer is working for their user which is the web application developer. The persistence layer developer is working for their user which is both the framework developer and the application developer. The application developer is working for the end users of their system.

The most effective developers will be the ones that realize this and deliberately cross as many disciplines as appropriate for their application space. This kind of renaissance developer will be most effective at knowing how what they will write will be received. This kind of developer will know implicitly that it is all about user interface and developers are users too.

Just as architects have known for generations that a building must be beautiful both on the outside and the inside yet still respect the laws of nature so too must computing grow up. So in my phrase "the power of software is the power of thoughts and ideas" is nestled the concept that not only power but beauty and value of software is the beauty and value of thoughts and ideas.

The metrics of our beauty in software is the cognitive dissonance we create in attempting to enable our users to create their art with our tools. When you create software you are implicitly directing the thoughts of another person or machine. Just as an Orwellian language could forbade certain thoughts your API could forbid a design or direct users down a path. The challenge is to match your design to the users' thoughts and direct them toward good solutions without imposing a mental fascism.

The converse is side of this is that you want to direct your users toward maintainable and workable designs. The goal should be to guide not to force. So working out an good design is a balancing act.

Just some thoughts that I've had while looking at possible improvements to an existing development framework.

Edit: Oh yes, I saw this shiny "follow" gadget and added it to the upper right hand corner. Won't you do me a favor and click it a few dozen times so I don't feel too bad about having "0" followers. Thanks.

Already voted. It's too late.

I usually use this space to make technical posts but I just want to say this: I've already voted. You can all stop robo-calling me, spamming me, feeding me your opinions... threatening me if I don't vote with you... whatever.

You can just give up on me. My vote is cast. No matter how angry, upset, hopeful, patriotic, or guilty you make me feel it's too late. I've already voted.

Now please stop it.

Programming Language as Human Computer Interface

Programming Languages are in fact Human Computer Interfaces and should be treated as such. The design of a DSL should match the thought patterns of the domain experts in a way that creates as little cognitive dissonance as possible. The result should be that a DSL can become a systems interface point between domain experts, programmers, and systems of systems themselves. We are already seeing this. We will continue to see this. The at the very tip of this idea are the DSL which Java and Groovy make so simple to build.

Programming is a human activity that occurs on multiple levels of thought. Thought tools assist in thinking. We call these metaphor. Why shouldn't programming language tools also help assist programming. The framework level language should therefore be used to create as thoughtful and targeted a set of metaphors for the application level programmer... who in turn is creating tools for the problem-domain experts to use.

Still alive...

Still alive, working very hard.

Convenience To Security Scale

I was thinking about a few things... we're doing a kind of security system audit at work and I've come up with a few koans about it:



Convenience is inversely proportional to Security.


The more secure something is the more inconvenient it is.


The easiest security system to use is one that doesn't exist.



In one measure the goal of security is to make things hard for people. Specifically, it should make things extremely hard for Bad People (tm) and not too hard for Good People (tm). These are both user groups that you are attempting to distinguish between. Both the good and bad people are people attempting to use your system and you are making a call as to which group a person belongs in.


So to that effect I've created the Convenience to Security Scale (patent pending).


On a scale from 1 to 10 a system that rates a 1 has no security whatsoever and is (presumably) very easy to use. And, a system that rates a 10 is perfectly secure because it is completely inaccessable. The goal is to judge where on the scale your system needs to be.


This model works even for insecure and frequently crashing programs since the frequent crashes are viewed (in this model) as a security feature preventing attackers from using the service. So making the program more stable and thereby more useable makes it more attackable... lowering its rating on the scale. A program that can't execute is perfectly secure since it can never be attacked.


The "secure" in this model is also referring to the protection of information... no "secret" information is divulged by the program. So a program that can't produce output is perfectly secure... it is also perfectly unusable.

Comparing Stats on this blog Feb & July 2008

I've been keeping stats on this blog since Febuary 1st of 2008. Here's some numbers. Don't know if they are good, bad, or whatever.

February hits: 1,503
July hits:     2,681

So I've not yet doubled my traffic in the last six months. What do those numbers over the last six months look like in total?

13,961 Visits
10,954 Absolute Unique Visitors
17,323 Pageviews
1.24 Average Pageviews
00:01:00 Time on Site
85.76% Bounce Rate
78.40% New Visits

... so most folks just bop through here and on their merry way. only about 15% are getting anything from their visit. I wonder what the average visitor's browser looks like? Top five browsers are a small surprise...

1. Firefox / Windows            49.86%  
2. Internet Explorer / Windows  20.06%  
3. Firefox / Linux              13.37%  
4. Firefox / Macintosh          6.88%  
5. Safari / Macintosh           4.61%

Firefox trumps explorer. But then consider the audience... programmers... yeah, I guess that makes sense. Interestingly 70% are Windows users, 13% are on Linux, and about 10% are on Macintosh. That's a bit surprising to see Linux rank higher than Mac.

I won't try and break down the search engine stats here but what I gather is that most visitors to this page are getting here through google or yahoo. Surprisingly, the most used key words are "grails portlet" I'm very happy to see GRAILS-254 coming along. Check out Lee Butts' documentation of his work.

Ironically, I never did follow through on a Grails Portlet because we dropped the use of Portlets in our organization. It would not have paid off for us to do portlet development until we had a much larger constellation of projects. I may have to revisit portlet work in Grails in the coming months judging from this demand.

I'm not a bit shocked that my more technical Grails articles enjoy the most readership so I plan on ending my indulgence of more abstract topics and getting down some serious Grails related business. I've just shipped a major Grails project that represents six months of work so I should be able to dig deep into that experience for a few posts.

What I'd love to hear either formally or on the side is how these numbers stack up to other blogs. Are these good numbers? Are they bad? I'm hanging them out there for everyone to see (or laugh at) because I have no delusions about being a well read blog and no reason to hide them. So I've offered first. You go next.

Why do you read this blog? Apparently about 3,000 of you come back here from time to time. How can I focus my topics to help you better? What would you like to ask me about my last year working in Grails full time? Any pointers from "professional" bloggers will be welcome.

UK Gov't and Lost Personal Data

Slashdot is running a story on UK Gov't Lost Personal Data On 4M People. I follow these stories (as previously posted here) because the management of Personally Identifiable Information is probably one of the hardest and most vital issues we have in front of the internet in the next decade.

As I have observed previously in this blog, computer programmers are beginning to build systems that substitute for subtle key social infrastructure that we (as mere programmers) have no idea that we are replacing. As computers take over the duties of identifying social standing, trust-worthiness, and other faculties that used to be purely human domains the protection of key information for each person either becomes deathly important or our ideas about how to verify identity have to be re-designed on a global scale.

Daunting, serious issues that will probably not be addressed until there are truly severe consequences to either the financial system or civil liberties. These are the core issues I set out to explore in this blog originally. In the simplest terms, the software engineering and the software design of the world's information systems need to grow up. And fast.

Unfortunately, the concept of "data guardian" feels like a hack and not a good one. I think we need to design information systems that obviate the need to keep PII secret at all... or systems that eliminate the need for the transmission of PII. Such systems would probably have to operate in radically different ways on the human level.

A fundamental property of information is that it is easily replicated the idea that we must prevent the spread of a specific class of information is like fighting the tide. Instead redefine identity and authority. If you must have a "secret ID key" (aka Social Security Number in the US) allow it to be easily changed.

Until society adapts to these ideas we all must carefully protect our personal information and those of us that write software to deal with credit, visas, or medical data must be vigilant beyond the immediate call of duty. Your web application might lead to someone's identity being stolen. Please think about that.

Anti-Management

Bruce Webster is fast becoming one of my favorite IT-management authors right now. If you haven't already,1 check out his article The Dead Sea Effect which summarizes why some companies have high turn-over and only retain the worst employees. It's a good read and the main ideas in that article should at least inform your decision about what job offers to accept.

I also listen a lot to Jared Richardson and his advice which is usually about positive examples of how to lead teams and set up software shops. Jared's writings tend to be the exact opposite of Bruce's. Bruce focuses on what people do wrong and Jared tends to focus on what people do right.

I've taken Bruce Webster's writings along with things I've learned from Jared's talks and thought up these anti-examples of management. I call them anti-management tips... some are from Bruce's observations, others are Jared's flipped in reverse.

anti-advice for managers, CEOs, and leaders everywhere:

1. Don't fire anyone, ever. The best way to demoralize troops is to avoid ever taking action against the incompetent, destructive, or otherwise evil employees you have. In fact, you should reward people with negative attitudes, poor work ethic, or destructive personalities by giving them the same or better salaries than the productive and positive people in the company. This way you show people that it doesn't matter if they do well or work hard and you will not remove negative influences or stand up for them. Not firing or ever taking action to remove harmful employees at all shows that you don't care to protect your company, the jobs of your employees, or create a non-hostile work environment.
   


2. Fire people randomly. The less relevant to performance you can make the firing or layoff of employees the better you will destroy their work ethic. Just up and fire people for reasons that nobody can fathom. It will help people feel that your workplace is unfair and that their loyalty is misplaced. If you do this enough you can start your very own "Dead Sea" as competent people leave for new jobs to avoid the chopping block. If you have a fair process that tells an employee if they are doing well or headed for a pink-slip then you might accidentally give people the idea that your work place is fair. Don't do that. Be unfair.
   


3. Never promote from within. If you never promote your people in-house you are telling them that they will never go anywhere and you see no potential in them. In other words, if you work here you must be a chump. If you want to advance in your career... this company is just a stepping-stone!
   


4. Only promote from within. Promote incompetent people, hostile people, people who stuck around long enough. Don't promote for skill. That would show fairness. Never hire outside the good-old-boy network showing that it only matters if you've been with the company a long time. That helps people to either seek new opportunities or hunker-down and wait for the rising tide of high turn-over to eventually lift their ship up the corporate ladder. Whatever you do don't make promotions based on skills, ability, training, or any other metric that might seem fair.
   


5. Never train your people. If your people have no real potential why bother training them. They might get better professionally and leave your company! Don't do that. You need to keep people from becoming talented enough to get other job opportunities. Only promote people who show a dis-interest in self-improvement to help underline this idea.
   


6. Switch skill requirements and retrain constantly. Make people feel like the future is uncertain. Make them feel like any skill they learn will immediately become worthless.
   


7. Never adjust plans. Don't change plans just because reality changed. This way you can further validate that you don't really care to protect your company, profits, and by extension the jobs of your employees. You further promote the image that you are detached, disinterested, and only drawing a paycheck yourself.
   


8. Change plans constantly. Paradoxically changing plans too frequently gives the impression that you have no idea what the hell you are doing, have no long range vision, and are not competent to protect the employment of your staff. Finding the balance requires skill and thoughtfulness and you never want to display those otherwise people might start having hope for the future.
   


9. Never sweat the details. Showing total disinterest in the details helps to foster that sense of detachment and disinterest in protecting the employment of your people. A well functioning company that is profitable and rewards its best employees is essentially ensuring continued profits for share holders and continued employment for the majority of its employees. Sometimes the details mean the difference between profit and loss. Never showing interest in the details shows not only a disinterest in the daily work lives of your employees as people but also a negligent disinterest in the affairs of your company. Smart employees will spot this and be effectively demoralized by it. So, go for two rounds of golf this afternoon.
   


10. Last minute Micromanage. Sweat all the details, preferably only about three weeks before the due date of any project. Swoop in at the last minute and add more managers to a project and start micromanaging details and changing things right up to the deadline. Doing this right necessitates that you create project plans that are multi-year or multi-month and then ignore all status updates for as long as possible. Micromanaging this way helps people who work under you not only feel you don't trust them but you are also incompetent and may try and hide that by blaming them. Micromanaging is bad but most people either learn how to live with it or find a balance with their manager if it is consistent... don't give people the opportunity to think they understand what is expected of them. Inconsistency is how you keep people guessing and uncertain about their futures.
    


11. Solicit feedback, then ignore it. Let people know you are just toying with them and don't take them seriously. Give them a party afterwards to rub it in.


12. Celebrate mediocrity. Celebrate and reward meaningless accomplishments, never acknowledge super-human effort. This will create the impression that you either don't pay attention or that you find above and beyond performance meaningless. Conversely, we celebrate a baby's ability to walk but we don't celebrate a teenager's ability to do the same. Loudly congratulating a baby for taking a step gets a giggle from the baby, loudly congratulating a teenager for taking a step gets nothing but disdain from the teenager. Your employees at different levels of ability operate the same way. Of course, you could also throw a party for the person who didn't screw up this time consider comments in front of the whole group like: Hurray! You didn't totally screw up! We're so proud of you. Who's acting like a competent developer? You are! Yes, you are! Wooshee wooshee woo!
    


13. Only listen to consultants. You need to let people know you think they are incompetent.


14. Never hire expert consultants to help with projects. You need to let people know you will never get them the help they need. Let them know you'll never train them, get them what they need to succeed, or actually care about what they are burning their precious life's flame out on.


15. Never cancel consultant's contract. Let people know you can spend lots of money, just not on them.


16. Never get to know your employees. Keep them at arms length, don't get to know them, tell jokes to other managers in front of them without acknowledging their existence. This keeps the employee feeling distant and detached. You want to let them know you think they are a cog.
    


17. Never tell your employees how the company is doing. Keep them in the dark and avoid telling them what's really going on in the company. That way they learn to use the grape-vine as the ultimate source of information. Your company will run on rumor and innuendo.
    


18. Broadcast everything negative. Anytime something the slightest bit wrong happens run around like chicken-little screaming that the sky is falling. That'll help morale.


19. Contradict subject matter experts. You should also contradict and constantly undermine your subject matter experts to show them that their expertise is worthless. If you contradict the best minds in the industry all the time you'll help people feel you are clueless. This will help heat up your little Dead Sea project.
    


20. Abuse your employees. If you can't physically beat up your employees make sure you set up situations where you can emotionally beat them up. Set up processes with lots of negative interactions with people telling each other "no" a lot. Don't use technology to alleviate the need for negative interactions between people. Make sure everyone is forced to confront their coworkers frequently for the smallest infractions.
    
    Whatever you do... don't create or install any metrics, automated tests, dashboards, continuous integration servers, bug tracking systems, or anything that could impartially handle the tasks of negative feedback that help create a perception of fairness and predictability. That kind of action would positively affect morale. Instead, keep work-life nebulous, unpredictable, and as unfair as possible. Make people battle each other based on emotion, gut feeling, and innuendo forcing politics to rule decisions instead of cold hard facts.
    

Comedy aside, I think building a talented software development team is a lot like building a talented basketball team... or any sports team really. You may need talented developers that are really star players, but, you may have to play without them in which case having a good strategy is key.

Frameworks, delicious frameworks...

This blog has been on "auto pilot" for the last few weeks while I've been on vacation. As my reserve of scheduled articles has dried up I guess it's back to work. I'll be at the trijug meeting tonight. We'll be hearing from Hadrian Zbarcea about Apache Camel which is yet another framework on top of Spring.

As I've noted previously in this blog, the J2EE space is being very effectively invaded by Spring. So effectively so that even J2EE Application Server provider JBoss provides Embeddable EJB3 which is a sub-set (as of this writing) of the full EJB3 environment that can deploy on "light weight" containers such as tomcat.

If that last paragraph made no sense, try thinking of it this way... JBoss, WebSphere, and other Application Servers provide a host of services that are considered part of an "Enterprise" stack. These services sit underneath specific applications. These are things that are conceptually low level like messaging between processes/beans, facilities for looking up shared resources, data persistence and storage, and security. Things like HTTP, HTTPS, and such are considered additional services and are provided by applications. A typical J2EE application looks up the additional resources it needs as it runs. So when your bean initializes it has to do a lot of leg-work looking things up and initializing resources.

The EJB3 application standard is in part an Inversion of Control framework. In some situations using Java 5 annotations EJB3 beans specify what resources they need and the framework wires up those resources so that they are available. Most notably the Entity Manager is usually wired up using annotations and not using JNDI lookups. It's so radically different that we call the new paradigm JEE 5 in some circles... not only skipping the numbers 3 and 4 (which are ugly looking anyhow) and jumping right to (the sexy looking) number 5... but also moving the number to the end of the acronym so we can spot the noobs.

Spring works on this Inversion of Control (IoC) design principle and this is a key feature of Spring that Grails makes heavy use of. The design inverts the J2EE sensibility of having a bean init and then look up everything it needs. It puts the control of a bean's initialization in the hands of the framework. Working this way actually means that the majority of logic for how a application server works moves out of the application server itself and out of the application and into the framework and its configuration files.

That means you can run complex applications in simpler "containers" like tomcat and jetty. That makes server administration simpler for shops running smaller and simpler applications. Shops with larger and more complex applications can migrate their Spring applications to heavier more robust clustering Application Servers. So you have simple to scalable in these lighter frameworks.

But, without the conventions found in Grails, IoC and Spring applications become "XML-er-ific" and choc-full of XML-ly goodness. You end up having to specify nearly all the same lookups and wirings you were having to do under J2EE anyhow. You just move the work from compiled Java to light-weight XML. It's better but you don't really escape the work.

This configuration issue is where Grails shines. Grails removes the need for most configuration work by introducing conventions. Because Spring is still under Grails you can over-ride or change the conventions if you want or you can abide by them and get the full benefit of Grails. So once again Groovy and Grails provide you with the choice of staying as shallow or diving as deep as you need. A great benefit that is missing in most development frameworks which either force you to become an expert of force you to keep the training wheels on forever.

I'm curious to learn about Apache Camel to see how they solved these issues. I'm also curious to see if JBoss will bring along their embeddable EJB3 to compete in this space. Obviously Groovy/Grails is my personal favorite but I'm always up for learning from others.

Software waste

Consider the plight of Microsoft and their rampant success that has also sown the seeds of their current woes. Also consider the plight of the BBC and their strange web platforms. And, I think we can start to see the inside of a very real recurring problem that date back at least to Fred Brooks Jr. and probably point back to problems that have plagued all engineering projects that require much management over-head.

Computers do a great job of covering up wasted labor. That's because they can crunch through stupidity like nobody's business. You have to wonder if management could actually see the product being built if they would take such a stance (both at Microsoft and at BBC).

It turns out that being insular in your development practices means you get to be "the old man on the mountain" and your shop turns into something quite unique... most of the time in a very bad way. The ideas of DRY, open source, and code reuse are ideas that we hold onto in the great wide community of developers to keep us from wandering down dead-end trails. The results of which are utter disasters unless you are Microsoft or the BBC which are both big enough to just absorb the loss.

My message to you is to get out there and get some of your sacred cows challenged. Learn how to defend your choices and your technical abilities or learn to change them. Do some open-source work. Stop making stupid rules to limit the creative talent you employ. Get out there and learn from others.

Software is a system of ideas

Take a system of ideas, turn it into code, let people interact with it. Software that works is software where everyone's ideas match and are consistent with a reality that can be represented in the computer. If your ideas don't match what is in the computer or in the heads of others then there is a bug.

Technical Debt and Technical Credit

The Martin Fowler entry titled Technical Debt has probably been the single most profound article I've read about software design in the last five years. It honestly describes why quick and dirty is a necessary evil and why shops need to build in development cycles to essentially "fix" working software. The idea of "paying down" technical debt is completely lost on non-programmers because they can't see the result directly. Instead what they see is the result of not paying technical debt.

I've had the misfortune of having this conversation:

Client: How long will this take?

Me: We've estimated that it will take about three weeks pushing just about everything else out of this development cycle.

Client: That's crazy! At my last job they could do the same thing in three days, that task should be trivial! Why will it take you so long?

Me: The legacy system design which we have to account for will never allow those changes without a significant amount of work. I'm sorry. Your other shop must have spent time upfront in preparation for such changes so that you didn't see the cost when you asked for them.


Sadly, the client can hardly help feeling that I'm incompetent. I've been set up for failure by a history of delivering too rapidly. It's something Bruce Webster calls the Dangers of a Successful IT project and I've had the problem in spades.

Obviously, I am working in a different legacy environment than the client's other shop was. To her she only sees working with me and her browser, to me I see working with her, the browser, and the old crufty code she just asked to have changed that has SQL in-lined in JSP and PHP, no CSS, a denormalized database, and cut-and-paste based templates (no templates at all). So changing somethings turn into a repetitive batch-job of a task and require full regression testing to make sure we didn't forget a page.

I'm working in a Technical Debt scenario and I've just been asked to do something that requires I pay back a lot of the technical debt in the system. In the scenario I'm thinking of the company had incurred over eight years of technical debt it had never bothered to try and pay back. The interest payments were huge by the time my "simple change" was asked for.

But, I've observed that are situations of both Technical Debt and Technical Credit. A Technical Credit is a credit that you get for working with tools that are ready made for agility. The Grails Framework for example is perfectly positioned to give you a major Technical Credit. Ready made plugins such as the Searchable plugin and the JFreeChart plugin are examples of quick to get and use tools that will make you seem like a technical guru even if you aren't.

This all applies to clients at the same company as you or clients at different companies. Anyone who is non-technical and has a stake in the project needs to be seen as a client of your work. And you, dear developer, need to cultivate a relationship of trust with them.

I think the analogies of Technical Credit and Technical Debt together can make strong cases for doing the unglamorous work that needs to be done... or adopting things like Open Source tools in the Enterprise. The developers on a project need to remember to sell this idea and tie-in the incidents of debt making and credit making to project time-lines to management. That is a difficult but important task that will build trust between you and your clients.

Maintenance Software Architect?

If you can accept the idea of a Software Architect the idea of a maintenance architect makes sense. However, in the circles I travel in there is actually some resistance to the idea of a software architect at all.

This resistance is usually driven by a misrepresentation of what the role should be. An architect role should be a mentor and guiding hand in the business' technology related initiatives. The architect should not be about the business of "bossing around" developers.

The Maintenance Architect role makes perfect sense if you look at what the job would be involved in doing. Even in the absence of a Software Architect a Maintenance Architect could still make sense. The Maintenance Architect would basically herd all the "cats" (developers) in a direction. The M.A. would steer a Perl shop (for example) toward exposing their CGI as distinct REST services that could then be fronted with Flex for example. In time you could whittle away at the CGI with Grails applications or another technology that fit your new technical stack. Eventually after a great number of iterations you would be transitioned to the new technology. You would "cook the lobster" if you will... slowly transitioning the shop to one of the 21st century software stacks.

It would be a long sighted goal and it would take a long sighted person to pull that off. The guy you'd look for would be someone with patience, endurance, and commitment. They would have to be a stern guiding hand yet be flexible enough to wait out the many development cycles to see their plan fulfilled. Not an easy order to fill and not a common person to find.

Proposed at BarCampRDU: iBatis plugin for Grails

One of the stranger ideas that came out of BarCampRDU this year was that we might do an iBatis plugin for Grails. Such that GORM could then run on-top of iBatis or perhaps you would tweak iBatis to run a faux-GORM in Grails context.

That seems like an exceptionally hard plugin to write. I think I might need some convincing before I even took a serious look at something like that. It is at least technically possible since GORM itself is a plugin. But, that doesn't mean there isn't an iBatis guru out there that would love to learn how to do iBatis + Grails for the folks that needed that.

Anybody out there interested?

BarCamp RDU 2008

This year's BarCampRDU was fantastic. Presentations given this year covered a really diverse set of topics. Frankly, too many topics to cover in this post. Ironically, I forgot to vote...

Talks on Social Networking? Yeah, but that is so last year. Instead I saw talks on iPhone programming and the Fan Programming Language, Cloud Computing using the Amazon EC2, Big Lee himself was there getting a talk going on social controls over Spam, and there were even topics such as Hadoop and Mahout (talk led by Grant Ingersoll no less). Not to mention a crowd of serious people working on incubating technology startups in the RTP area. And yes, a group discussing Groovy and Grails was there and yours truly did more than lurk.

The interesting thing about Grails is how broad an audience it targets. At the moment the core G2One guys (I've only met Jeff Brown) are focused on Java shops. This is probably a good "head of the spear" tactic for them. (See I was paying attention in the startup talk) as the integration story between Java and Groovy and Spring and Grails just totally fantastic. There are only so many hours in the day and only so many people you can talk to. So the choice to focus was probably wise. I think there are some great advocates and allies for Grails waiting out there in frustrated LAMP stack developers who are ready to move to the next level.

The fact that you can dive as deep into Grails as you want to or stay as shallow as you want is so powerful. By leveraging Groovy which by its very construction means I can dig deep into Java if I need to means for small projects that scale out they can be incrementally and iteratively backed off down to Java applications if you need to. Or, you can stay fast an loose on top of the Grails platform cranking out features like there's no tomorrow. And there are many LAMP guys who are going to love that. I think I've met a few smart guys that might actually find a few great ways to help Grails grow into this region.

During the between presentations times I met folks who work at exciting technology companies like rPath, Lulu, Tibco, and through them I've heard about the small Google office here in RTP working on Android related software. And, since I've started to get out and about I know that Sun also has a small group of engineers here in RTP as well as Oracle and the obvious big guns like RedHat, Cisco, Nortel, and IBM.

I think that the RTP area (because of the insane growth seen in Chapel Hill, Durham, Cary, and Raleigh) is just inevitably the next great place to be for new young companies seeking to innovate, invent, and create. If there is a recession it's not here, if any thing I'm seeing a boom town in my backyard. Why aren't there more high profile startups and more technology companies here? I think it's just a perception thing.

I can't mention everyone and everything in this post ... in a scant few hours I got as much out of BarCampRDU as I have some week long conferences ... I'm sure that many of the connections I made at BarCampRDU will help to inform future posts and might even change the course and focus of my career. Yes, it was that good. If you didn't go... you missed out.

Groovy class tricks: Zero Padded Number

Take the example of a SKU number. We need the number to be zero padded. For example the number 10 should print as "0010" and not "10" in any display of this object. These numbers might be generated from some source and handed to us to store as an additional identifier for some object. So the source is a number but the destination is a zero-padded number string.

With groovy we have two ways to tackle this problem. First we could store the value as a string in our domain class. We know that the source data should be a number and the output will always be a string. This has the advantage of showing the zero padded number in our database for when folks do queries.

Now, Groovy automatically creates your getters and setters. You never write them yourself. But the trick here is that the getters and setters still exist that means you can do something like this:

class SkuItem {
    String skuNumber
    public void setSkuNumber(Long number) {
        skuNumber = String.format('%014d',number) // NOTE: zero-pad to 14 characters
    }
    public String toString() {
        return skuNumber
    }
}

Doing the zero-padded number this way means the following code will throw an exception and crash:

def sku = new SkuItem(skuNumber: "10")
println sku

The exception looks like this:

Caught: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object '10' with class 'java.lang.String' to class 'java.lang.Integer'

Which is what we want if we intend to enforce that a skuNumber is actually a number. This code will work:

def sku = new SkuItem(skuNumber: 10)
println sku
sku.skuNumber = 20
println sku

and produces this output:

00000000000010
00000000000020

But now that we know that in Groovy the getters and setters are there we can solve this problem the other way. Storing the string in the domain class means that database reports will see the zero padded numbers but that is inefficient and means that the database is working harder to build string based indexes. These are numbers and working with numbers is faster and more compact.

Knowing that the getters and setters are under the covers even in our groovy constructors we could rewrite our SKU storage class like this:

class SkuItem2 {
    Long skuNumber
    public String getSkuNumber() {
        return String.format('%014d',this.skuNumber)
    }
    public String toString() {
        return this.getSkuNumber()
    }
}

Which has the advantage of storing numbers as numbers but the moment the item is fetched it gets turned into a zero padded string... for example:

def sku2 = new SkuItem2(skuNumber: 30)
println sku2
sku2.skuNumber = 40
println sku2

which now produces the output:

00000000000030
00000000000040

... which just goes to show you get even more out of Groovy when you understand how it works underneath the covers.

Language is a Cognitive Technology

Making the rounds is a story out of the MIT news office about the Piraha tribe whose language does not include any counting numbers. They do have the concepts of more than and greater than and the concepts of few and many. Counting isn't a useful skill for them.

What is most intriguing to me is not the missing counting concepts but this odd quote from the end of the article:

One other discovery of the project is that the Piraha can perform exact matching tasks as long as there is no memory component to them, but once there is a memory component, they approximate their matches. This suggests that language is a cognitive technology that aids humans in memory tasks.

That is an interesting phrase to me: language is a cognitive technology.

Programming languages represent cognitive tools to work with a computer. That is definitely true. If you can consider language itself a technology of the mind or a cognitive tool that is an intriguing idea. Is your ability to only use one language or one programming language akin to only being able to use a screw driver?

If we can consider language itself a technology then we have an interesting new time-line for the progress of technology. We can go back to prehistory tracing the advent of the basic concepts of mathematics building up to today's computers which ostensibly hold the fruit of tens of thousands of years of cognitive technological effort.

And this begs me to question... what if we had started on the wrong path all those millenia ago. What if there are better cognitive platforms to build our systems of thought on top of but we couldn't explore them because our tools of cognition effectively created a platform of thought that prevented us exploring other methods of thinking. There could be ways of thinking and computing we can't even look directly at because they are so alien to us.

This idea cuts both ways. It tells me that if a paradigm is too far removed from mainstream thought it could die because not enough minds can (or will) adopt it. That means there are undoubtedly modern linguistic paradigms that are limiting but also unavoidably entrenched. This is good and bad. One language helps people to be mobile even as it limits them.

Consider how English is changing as speakers in many countries begin to change English to match the paradigms of the local tongues it mingles with. The resultant language may become a world tongue called Panglish in some breathless articles. It's all speculation right now.

But, we see this all the time in Programming languages. We see programmers transplanted from one language into another that bring with them the idioms of the languages they left behind. When this becomes laughable is when what is changing is far more than syntax but entire ways of thinking through problems that are being butchered by forcing the new language to work like the old one.

Here is a counter intuitive idea, but, some limitations are actually liberating. When children play in an open field they will cluster together. If you fence in an area larger than what they are actually occupying they will spread out to fill the space. The same is true about how you think about things.

Too broad a programming framework means that you can't effectively address any specific problem. It means you have too wide open a field. Instead the more a framework leads you to an answer the faster you will move toward a goal. Too efficient and limiting of a framework and you can't steer toward an appropriate design and must instead work around your framework.

Think of it as a volume of space where each point represents a system design. Your framework is like a channel or road driving you toward a region of all the possible system designs you can imagine. There is a volume inside that huge space of all possible systems that contains designs that fulfill your requirements (stated and otherwise) and your framework drives you in the direction of a handful of these. If your framework is too strict there could be a problem where the volume of answers lie in one direction and your framework drives you in another.

That would be an impedance mismatch between the problem and the framework. You would need to modify the problem or the framework to bring them into concert. Or you would have to abandon your framework entirely.

Most of us today are working in a world where our frameworks are pulling us toward solutions that are wholly web-based. That is just a fact of what is in fashion today. It isn't wrong either. The web will finally disassociate the cognitive and the physical. Just as MP3 is slowly killing the CD so too will the web slowly kill off certain legacy computing paradigms.

What do the Piraha know and can think about that we can't? Perhaps they have some insights of genius that are not evident to us who are encumbered with numbered thoughts? I wonder how many of us are like the Piraha happy in our own world but are fast colliding with a world that is driven by new thoughts and ideas utterly alien to us. As alien as numbers are to the Piraha the concepts behind cloud computing and massively parallel computing systems are headed our way. What will the programming languages that help us get there look like? How far off course have we drifted along in the channels of thought carved by our programming infrastructure?

It's just a thought.

PostgreSQL and Grails

Just a few notes on setting up Grails with PostgreSQL and PostgreSQL accounts for JDBC connections.

First you need to have postgreSQL installed. The install varies per OS. I'll let you figure that out since there's plenty of documentation on that part.

Next you need to set up your grails application to connect to the database.

Download a PostgreSQL JDBC driver and copy it into your applications lib folder, for example: myProject/lib/pg74.216.jdbc3.jar in my set up. I don't really need to talk about setting up an application from scratch or anything do I? You've all seen that enough times right?

Next, set up your data-source for using your PostgreSQL driver...

dataSource {
    pooled = false
    dbCreate = "create-drop" // one of 'create', 'create-drop','update'
    url = "jdbc:postgresql://localhost:5432/grails"
    driverClassName = "org.postgresql.Driver"
    username = "grails"
    password = "grails"

    // NOTE: both of these dialects have worked for me. But some people
    // recommend using the net.sf version and not the org.hibernate version.
    // dialect = org.hibernate.dialect.PostgreSQLDialect // honestly, not sure what
    dialect = net.sf.hibernate.dialect.PostgreSQLDialect // the difference is.

}

Right now, this configuration will promptly fail. There is no grails database and no grails user. Setting one up requires a little thinking. Obviously, we need to make the grails database first. Let's get that out of the way and move to the slightly more ... confusing ... stuff.

Setting up a database and user on a fresh install requires you to be the postgres user. This is the username created by the postgres installer that the postgres daemon runs under. I have no idea what my postgres user password is so this is how I run the postgres commands to create the grails user and database...

 $ sudo su
 # su postgres
 > createuser -P grails
 > createdb grails

... as the user postgres still, we set the grails user's password ...

 > psql grails
 grails=# ALTER ROLE grails WITH PASSWORD 'grails' \g
 grails=# \q

... and this would be ready to go except that when we try to login as grails we get an authentication error.

By default Postgres uses the authentication system of the OS for determining the identity of the user. This could be a problem if you plan on having users other than your own username or jetty/jboss/nobody use your databases. Most likely you'll want different users depending on application. So we need to modify the postgres configuration files to allow for users other than the console users to login.

The configuration files are found in /var/lib/pgsql/data or other data directory depending on your specific setup. The file for controlling authentication policies is pg_hba.conf. It is pretty well documented on its own and by default the "method" for authentication is set to ident meaning that if I am username "fred" then I can only use the database as "fred" even if I'm running the application "grails".

So for local and host users to login as a password authenticated user we need to set the "method" to password instead. I've set my pg_hba.conf to read...

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD
# "local" is for Unix domain socket connections only
local   all         all                               password sameuser
# IPv4 local connections:
host    all         all         127.0.0.1/32          password sameuser
# IPv6 local connections:
host    all         all         ::1/128               ident sameuser

... allowing a user from local host to login using a username and password pair. I restart postgres (as my own username not postgres)

 $ sudo /etc/init.d/postgresql restart

... and now from my shell I can login as the grails user to the grails database...

 $ psql -U grails grails
Password for user grails:

... and I supply the password grails and I'm logged in. That means this time when I start my grails project with grails run-app the grails user will authenticate and now my grails project can use postgres for development against the postgres database and we can work against postgres on our development machine.

EDIT:
Your whole Grails configuration file (as of Grails 1.1) should look like this

/** 
 * The top dataSource holds configuration options for ALL
 * environments... I'm presuming you want PostgreSQL in all
 * your environments but you may want to use the default
 * Hypersonic database in development and testing instead.
 */
dataSource {
 pooled = true
 driverClassName = "org.postgresql.Driver"
 // dialect = org.hibernate.dialect.PostgreSQLDialect
 dialect = net.sf.hibernate.dialect.PostgreSQLDialect
}
hibernate {
    cache.use_second_level_cache=true
    cache.use_query_cache=true
    cache.provider_class='com.opensymphony.oscache.hibernate.OSCacheProvider'
}
// environment specific settings
environments {
 development {
  dataSource {
    // one of 'create', 'create-drop','update'
    dbCreate = "create-drop" 
    url="jdbc:postgresql://localhost:5432/dev"
    username = "dev"
    password = "dev"
  }
 }
 test {
  dataSource {
    dbCreate = "update"
    url="jdbc:postgresql://localhost:5432/test"
    driverClassName = "org.postgresql.Driver"
    username = "tester"
    password = "tester"
  }
 }
 production {
  dataSource {
    dbCreate = "update"
    url="jdbc:postgresql://localhost:5432/grails"
    username = "grails"
    password = "grails"
  }
 }
}

API is API, but so is table structure...

It's sometimes hard for developers to remember that table structure is also an API. Yes, it should be considered an internal API for the application you are developing, but it is still part of a system that must be carefully constructed to reflect the reality in which it operates. It is not enough to just normalize data following the rules of database normalization... not anymore anyway... it is important to consider how you will mash-up and add onto a database schema.

Consider RESTful webservices as a unique opportunity to add data into an enterprise system. You have a URI for a resource exposed by a restful API. That URI can then be bookmarked and annotated in a system like del.icio.us for your internal users.

That kind of annotation is powerful. It means you can add tags, wiki, and annotated information to CRM pages, BOM, or other data independent of the underlying system implementation. And, not surprizingly, Grails is well situated for this. Grails projects with Content Negotation can be viewed as amazing polymorphic services that display web pages to browsers, XML to services, and JSON to ajax clients.

That flexibility means you can "bookmark" ala social bookmarking in multiple contexts ... these bookmarks can carry meta data about a linked URI. That meta-data can be anything from contact notes to wiki entries. That could mean a new era of business applications that provide semantic inter-sections of web URI based on simple RESTful principles.

Imagine a BOM bookmarked and annotated in a del.icio.us type interface by different departments. A kitting department links the BOM to a history list that represents activities of kitting. Billing annotates accounting activity by bookmarking and logging the original BOM. The accounting system could be .NET, the kitting in PHP... it wouldn't matter. And you wouldn't need SOAP or known WSDL. The core system would be simpler and concerned with just web-page functions... and (using Grails) a scant few lines for context sensitivity to what the controller renders (XML, HTML, Ajax, JSON, etc.) ...meaning you can develop supporting systems each orthogonal to the data source system.

Think of it as meta-Aspect Oriented Programming if that makes you feel better about it. But, really, it's all about using REST and the web for what they are good at and leveraging the power of the browser as a broker platform. I've heard it called Browser As A Broker (BAAB) and also Enterprise Service Browser (ESB). I think I'll call it... simple and to the point.

Hide complex behind simple...

Hide the complex behind the simple. Keep the complexity of a system behind the simplest possible interface. At every level find the simplest way to express the intention of the interface, do that, and get out of the way. In short, make things easy for your user and hard on you. It's your job.

If you are creating API (as I sometimes find myself doing) work to create the interfaces that the users of your API will find intuitive and simple to use. Hide the complexity of this work as succinctly as possible behind the inner implementation of the library.

If you are working on UI (as I also paradoxically find myself doing) work to create an interface that encourages play. I don't mean gaming. I mean, allow a user to "play with" the interface and explore it. Let them discover what it can do and try and make it intuitive to the way that they think about problems.

In the desing of both UI and API you can find yourself altering what "simple" is. You can introduce new ways of thinking about a problem by creating new implicit behaviors. This is an interesting space to work in collaboratively because you can find that users (both programmers and lay-users) have developed preconceptions and notions of how a system "must work" and so doing box themselves in.

In my experience of hiding complex things under simple interfaces I have found that there really is nothing that is truly simple in the whole. Instead things are simple in isolation but produce emergent complexity when placed in large systems. So to fight this the best weapon is to keep systems small. By keeping systems small and isolating their inputs and outputs you have a better chance of artfully controlling their interactions.

The other side of this is that our minds are horrible at containing inherent complexity. I used to think I was very good at this. I no longer think that. I now realize that I might be better at dealing with complexity than some, but my abilities have a distinct upper limit that is reached very quickly.

That is why when we build systems of any size. We developers of systems of software should acknowledge the need to test the system to validate our understanding at every level. Just as we use the scientific method to validate what we believe to be true using experimentation we should validate the way thoughts interact via experimentation.

The irony that what we are testing is in my opinion codified thought is not lost on me. Consider Kurt Gödel and his incompleteness theorems which point to the limitations of all formal systems... they are doomed to incompleteness. That is to say that you can never through proof alone (code alone) create a system to model such complex yet simple things as the counting numbers.

So there is a bug in the universe?

When you or I create software in any capacity we are indeed creating a formal system. A system that will either fail to model the real world problem we are attempting to emulate or will be fraught with inconsistency and incompleteness. So even if we were super-coders that never produced bugs, and you and I could in fact produce a perfect system, that system could be fraught with problems due to its inability to properly model or express the real world system it was based on.

The simplest answer is to devise tests to prove the correctness of our systems.

Wandering around Research Triangle Software Symposium 2008

My path through the Symposium one was eclectic. This year my needs are in the areas of project management and dealing with software development life-cycle. There comes a point when you realize you have all the right technical tools but you don't have the right soft skills in order to deal with business level problems.

I got to sit through several talks by Jared Richardson. Jared's talks this weekend gave me the perspective that the changes we are after when we say "agile" are there to deliberately limit the number of negative interpersonal interactions and increase the positive interpersonal interactions. If you do that then it doesn't matter what you call it. The technologies that make us "agile" are not panacea but tools to tweak how people deal with each other. Tightening feedback loops between people and creating automations for tasks actually translate into building trust between people and objectivity into assessments of success and failure. This was a major perspective shifting moment for me that left me a little dumb-struck. It was a "there is no spoon" moment.

By Sunday I was really worn down. I'm not a naturally gregarious person and three days of constant crowds get me beat. Besides, how many mind-blowing life-changing talks can you take in three days? So Sunday I geeked out at the technology presentations and I was glad I did.

Venkat Subramaniam covered FP on the JVM. Functional programming is something that I've brushed against repeatedly but never used in the enterprise. I think I share Venkat's predilection for learning new languages I just haven't exercised it as much. I am going to have to teach myself Erlang this year even though I know I'll never use it in production. Erlang will probably show up in this blog from time to time. Remember, this is not a Groovy/Grails blog. Groovy just happens to be what I'm working with both at work and at home right now. I strongly believe that the point of software is the codification of cognition. That means using the tool that best represents that thought and uses the best platform for delivery of that thought. Venkat has definitely raised my interest in some of the FP languages.

I'm so steeped in Groovy lately I thought I should go and get some JRuby exposure. I've never attended a talk on Ruby. I've only bashed my way through Ruby and Ruby on Rails in small toy applications that I developed during research projects over the last few years. Ruby is a very simple and powerful language (and very pretty IMHO) but I think most of what causes resistance to it is how alien it looks. The first time I heard about Ruby was from Bishop while we were in college together.

I ended up getting work using Perl, C, Ada, and sometimes Bash over the years so I never really used Ruby at work. Neil Ford does an excellent job selling JRuby as a next generation JVM language. I did detect that some of his comic jabs at Java were taken as outright insult by some of the more serious Java-heads. And this is why I say: "Never fall in love with technology." It blinds you to objectively evaluating a technology and its capabilities. That goes for all of us Ruby enthusiasts or Groovy enthusiasts alike.

Language is definitely one way to open up your mind about some of these new problems we face dealing with concurrency on multi-core systems. I think that the more important sea-change is in programming paradigm. Not necessarily language. Consider this video floating around on Slashdot (and triJug) featuring Cliff Click. It covers concurrency correctness for a data structure by the use of finite-state-automata with an extra rule: Any state can be a start state. Cliff Click's solution is to create logic in the threads that will allow each one to act correctly no matter what state it finds the data in. Now if only I could get paid to write stuff like that. That's some serious getting your geek-on.

And that stuff is written in Java. Just to prove the point that any Turing complete language can produce the same computational results as any other. It's just a matter of how you get there.

Sunday afternoon I sat through both talks by Jeff Brown the first one by accident actually. I had forgotten to move to see Venkat Subramaniam and hear his take on Guice, but I was glad I stayed to hear both of Jeff's talks. Not only does he have the fantastic style of understating things to great effect but it was an absolute thrill to hear repeated gasps from the audience as he introduced Grails to them.

And here's what's interesting about Grails is that it does get Java people excited. Much more excited than Rails does. I think Grails excites Java developers because it can run in their current environments and hook into what they already have with very little work. Grails is built-up from Spring so they can see how they can leverage what they already know about so many Java tools. And at any point of pain you can fall back to what you know.

I think Grails fills a very neglected need in the Java developer space and that need has been neglected so long because those feedback loops between tool vendors and the common programmer have been broken. As long as the Grails community works to keep those loops closed and to listen to its user population there is a very large niche for Grails in Java-rich shops. The trip to Grails is so much shorter than the trip to Rails. Even Rails on the JVM.

Jeff's second talk taught me several tricks I was missing. I got to talk to him about my Audit Logging plugin and even got a few pointers so I've made a list of things I need to do. Highest on the list is to setup PostgreSQL and get to testing with that configuration. I also got several ideas for new plugins that I could add using what I already know. I should have time for this after July until then all my free development time is double booked.

An interesting question that I got was how does Grails scale? I'm sure most of the answer to that is contained in how does Spring scale? I personally only have production grails apps under modest load some 10 to 15 transactions a second at peak (a peak which is exceedingly rare and my performance upper bound is dependent on two external systems so I had to back my application's speed off to avoid swamping the other system). But, if you want the real answer from some guys with real heavy use scenario experience go take a look at Brian Guan has to say.

The other advantage with Grails is being able to say: "If you find a part that doesn't scale, rewrite it in Java." And I'm not telling a web developer to go dust off her C book instead I'm pointing to an existing infrastructure and expertise one layer below Groovy. One that the developer already has and uses.

For reasons that I'll explain in a later post I have to use two different container managed datasources. The first one drives my GORM for access to objects and the second drives a set of custom DAO to normalize data using some very cleverly hand crafted queries. The DAO gets its data source from Config.groovy in development mode or from JNDI in both test and production mode.

Just because it's never gotten a slide... here's the resource.xml way of grabbing that for Grails...

<bean id="legacyDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiName" value="jdbc/LegacyDS" />
</bean>

... and if I wanted to grab that data source in a groovy object

  def legacyDataSource

... sadly I don't always have a JNDI defined data source in my environment so what I did is the less elegant and more bludgeon-ly:

    def openDb() {
        def conf = ApplicationHolder.application.config
        def db
        if( conf.legacy.jndi ) {
            if(!legacyDataSource) {
                def initialContext
                // Obtain initial context
                try {
                    initialContext = new javax.naming.InitialContext();
                    legacyDataSource = (DataSource) initialContext.lookup(conf.legacy.jndi);
                } finally {
                    initialContext.close();
                }
            }
            db = Sql.newInstance(legacyDataSource)
        }
        else {
            db = Sql.newInstance(
                    conf.legacy.url,
                    conf.legacy.username,
                    conf.legacy.password,
                    conf.legacy.driverClassName
                )
        }
        return db
    }

... which switches depending on whether the environment is configured for JNDI or for directly managed connections.

I met lots of folks that I hope keep the Grails enthusiasm through the next year. I think we are at the beginning of a major sea-change in the Java space and I think both Groovy and Grails will be important players in that change. Nobody can predict what's next but Groovy has a way of getting Java people excited. It's been a long time in coming and the need in the RTP area is definitely there. With Cloud Computing for Grails coming along nicely I think we have plenty to be excited about.

Beginning to understand WebFlow

I'm beginning to understand WebFlows. WebFlow is perhaps the most different part of working with Grails. You don't have to use WebFlows and in fact I think I'd advise people to avoid them until they are very comfortable in Grails. That's because the diagnosis of WebFlow is not as straight forward and intuitive as the rest of Grails. For most of Grails you just do things. With WebFlow you need to think things out.

I've developed an in-elegant technique for working with WebFlows that actually works well for me. I'll refine it a bit and write this up at some point. Hopefully some of the Grails rock-stars out there will help me get straight about it.

I'll be attending the NFJS conference in RTP this week and I plan on taking advantage of the free IntelliJ license. Maybe if I live with it for a while I'll get over myself and start getting productive. The problem is... I really am amazing with vi I've figured out how to use vi inside my current tool-chain and I'm actually quite effective at writing regex and macros in it for refactoring. I lose all that when I switch to a new tool kit so there is significant pain in adopting a new tool. Least of which is a price tag for something you don't know if you can adapt yourself to.

Now attending NFJS in RTP is what turned me on to Groovy and Grails. I happened to have been shopping around for platforms at the time so the timing couldn't have been better. Now I have Grails firmly in hand I'm interested in attending the development process improvement workshops so I can start working more effectively with the tools I currently have and potentially adding new support tools to my tool bag.

Deploying Grails 1.0.3 on JBoss 4.2.1

Grails version 1.0.3 was recently released and if you blindly try and deploy this on to an old JBoss (I'm using 4.2.1) your deployment might fail giving you this trace:

Exception sending context initialized event to listener instance of class org.codehaus.groovy.grails.web.context.GrailsContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pluginManager' defined in ServletContext resource [/WEB-INF/applicationContext.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/hibernate/annotations/common/reflection/ReflectionManager
        at java.security.AccessController.doPrivileged(Native Method)

You'll need to add the following jars (which you could take from grails-1.0.3/lib if you wanted) to your JBoss server's lib directory...

1. hibernate3.jar


2. hibernate-annotations.jar


3. hibernate-commons-annotations.jar

Why all three? Well, if you don't do all three you'll see...

Exception sending context initialized event to listener instance of class org.codehaus.groovy.grails.web.context.GrailsContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'messageSource': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Invocation of init method failed; nested exception is org.hibernate.AnnotationException: java.lang.NoSuchMethodException: org.hibernate.validator.ClassValidator.(java.lang.Class, java.util.ResourceBundle, org.hibernate.validator.MessageInterpolator, java.util.Map, org.hibernate.annotations.common.reflection.ReflectionManager)

... so drop all three in there and upgrade your JBoss server's version of Hibernate.

Drinking from the Grail

The idea behind Grail (not to be confused with Grails) has me excited. In particular the idea had come to me earlier but... I'm not Guillaume so it's not like I'm going to pull off an idea like that.

But. Guillaume is Guillaume. So that means he just might pull off this concept.

What's the big idea? It's Guillaume et al's take on CPAN/RubyGems for Java/Groovy land. The biggest strength that Perl had was CPAN. One of the best features of Ruby is the RubyGems which is ostensibly a take on CPAN for Ruby. There have been several attempts in the past at a CPAN for Java but, sadly, none have taken. The same problems that keep a CPAN for C/C++ from existing keep a CPAN for Java from existing... basically build environments get screwy.

But, Groovy is a special beastie in the world of programming languages. It is truly unique in its place because of how closely it matches Java and how it can be run as a script or as a compiled Java class. Jeff Brown gave a nice presentation at TriJug showing how a compiled groovy class' byte code looks just like a Java class' byte code... and that's a nifty trick that takes some people three or four double takes to understand. The compiler is dropping in whole methods for you. Powerful, easy to use, and a little mind-warping. It's probably the number one reason Groovy has had a shallow uptake until now. It has taken time for the community to digest this new paradigm.

It's this interesting hybrid compiled/scripted/synthesis nature of Groovy that could mean a project like Grail could work where other Java archive systems have failed. I suspect that as the project becomes reality we'll find it leaning on Groovy language capabilities more and more under the covers even if it can benefit straight Java projects it will need Groovy in some way. Probably using that MissingClassException feature and some nifty code synthesis tricks... I'm not sure...

Of course I'm just arm-chair-quarter-backing here. It's a rock star like Guillaume who's going to pull this off.

EDIT: And there we have it: Grape ... thanks for pointing that out Andres.

Lessons learned...

A short list of things I have done wrong recently...

• Working in GORM/Hibernate if you provide a mapping DSL and intend on your class to move between databases do not assume the default "hilo" algorithm will work on all databases. See id generator documentation at hibernate.org


• Do not assume that you can throw classes up and down an inheritance hierarchy in Hibernate. For example:
  

  
  class Person {
    Long id
    String name
  }
  class Parent extends Person {
   static hasMany = ['kids':Person]
  } 
  

  
  ... you need to do this to turn a "person" into a "parent"
  

  
  
      def makeParent = {
          Person.executeUpdate( "update versioned Person as p set p.class =:clazz where p.id =:id", [clazz:Parent.class.getName(),id:params.id.toLong()] )
          def parent = Parent.get(params.id)
          if(!parent) {
              flash.message = "Person not found with id ${params.id}"
              redirect(action:list)
          }
          else {
              redirect(controller:'parent',action:show,id:parent.id)
          }
      }
  

  
  
  ... other techniques to do this create a new object with a new id. That means you can't model a "parent" to "child" relationship this way. Instead turn the relationship around ...
  

  
  class Person {
    Long id
    String name
    static hasMany = ['parents':Person]
  }
  

  
  ... or create a bridge object...
  

  
  class Person {
    Long id
    String name
  }
  class Relationship {
     Person parent
     Person child
  }
  

  
  ... and that works much better.
  


• Do not lock in a domain model based on "big design first" principles. Domain may have to change to fit reality of your system. Sometimes you don't really understand the problem on day 1 or day 10.


• Do not release big, release small, release frequently. It's better to get tiny releases and adjust course in small increments rather than have "big bang" releases.


• Coding and posting while tired only lead to messes you have to clean up later.


• Do not ignore your email.

Hopefully, the insanity has passed.

Web 2.0 is so last year...

Note: I'm still too busy to write full posts. I should be back to a normal schedule soon.

First there was the web, then Web 2.0, now there's Web 3Di...



Actually, it's a very insightful video. The problem is that 3Di is not just twice as hard as 2.0 it's exponentially harder. Increasing dimensions in a system exponentiates the complexity possible in a system. Now, I'm not just riffing on the name. Actually a decade ago I thought that 3D internet ala Second Life would be main-stream today. I even based my Graduate studies on this. My realization is that these things are actually much much harder than we realize. I think the next decade should see real web 3Di take off. That means really useful 3D web sometime between 2010 and 2020. If that time scale is too big for you... just ignore 3Di for now.

However, I look forward to the day that I can bring back my work in VR and marry it up to my Web based work. I may get to see that day sooner than I thought.

You should care about API...

It turns out that API are created constantly in any environment that must share code. API are one of the ways that computers carry our thoughts and ideas. And API can limit or expand the way a company (not just a software company) a company does business. It can stifle their growth and limit their ability to do business or it can foster growth...



BTW: much of this talk also applies to general application design not just API design.

A terrible hibernate hack... for Grails...

I'm still "under water" this week but I felt this really deserved an entry. I found this blog entry by Sorin Postelnicu that solved a problem I was having with my Grails configuration. I've set up my database to use utf8 everywhere but my tables are still getting created as latin1 tables. What to do... using Sorin's example I did this:

package org.hibernate.dialect;
/**
 * Extends MySQL5InnoDBDialect and sets the default charset to be UTF-8
 * @author Sorin Postelnicu
 * @since Aug 13, 2007
 */
public class MySQL5UTF8InnoDBDialect extends MySQL5InnoDBDialect {
    public String getTableTypeString() {
        return " ENGINE=InnoDB DEFAULT CHARSET=utf8";
    }
}

The trick here is that Sorin had over-ridden the default table string from hibernate which says "CHARSET=latin1". I've just named the class differently. I put this class in the right package in my Grails project's "src/java" directory and modify my Datasource.groovy to read like this:

dataSource {
 configClass = GrailsAnnotationConfiguration.class
 pooled = false
 driverClassName = "com.mysql.jdbc.Driver"
 dialect= org.hibernate.dialect.MySQL5UTF8InnoDBDialect
 url = "jdbc:mysql://localhost:3306/grails?zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=UTF-8"
 username = "grails"
 password = "grails"
 dbCreate = "update"
}

... now my GORM generated tables show up in UTF8. But my question to you is ... is this a plugin? is this a patch to hibernate? Is this a hack? You tell me.

A Grails WebFlow observation

Objects created and persisted either explicitly or implicitly to the flow context must implement the java.io.Serializable interface. But, closures can not be serialized. That means that any class that uses a closure such as constraints will probably have to do custom serialization. This is very easy to implement in Groovy since all objects have a properties map and maps can serialize. Just be certain to pick the right keys to serialize.

I've noticed that even objects created by the onChange event handler may end up serialized if the change happens inside an action node of a WebFlow.

EDIT: I stand corrected. It is much easier to mark the closures with transient instead of just def. This just works and requires no further thinking. I should probably put this in an example somewhere.