Maintenance Software Architect?

If you can accept the idea of a Software Architect the idea of a maintenance architect makes sense. However, in the circles I travel in there is actually some resistance to the idea of a software architect at all.

This resistance is usually driven by a misrepresentation of what the role should be. An architect role should be a mentor and guiding hand in the business' technology related initiatives. The architect should not be about the business of "bossing around" developers.

The Maintenance Architect role makes perfect sense if you look at what the job would be involved in doing. Even in the absence of a Software Architect a Maintenance Architect could still make sense. The Maintenance Architect would basically herd all the "cats" (developers) in a direction. The M.A. would steer a Perl shop (for example) toward exposing their CGI as distinct REST services that could then be fronted with Flex for example. In time you could whittle away at the CGI with Grails applications or another technology that fit your new technical stack. Eventually after a great number of iterations you would be transitioned to the new technology. You would "cook the lobster" if you will... slowly transitioning the shop to one of the 21st century software stacks.

It would be a long sighted goal and it would take a long sighted person to pull that off. The guy you'd look for would be someone with patience, endurance, and commitment. They would have to be a stern guiding hand yet be flexible enough to wait out the many development cycles to see their plan fulfilled. Not an easy order to fill and not a common person to find.

Proposed at BarCampRDU: iBatis plugin for Grails

One of the stranger ideas that came out of BarCampRDU this year was that we might do an iBatis plugin for Grails. Such that GORM could then run on-top of iBatis or perhaps you would tweak iBatis to run a faux-GORM in Grails context.

That seems like an exceptionally hard plugin to write. I think I might need some convincing before I even took a serious look at something like that. It is at least technically possible since GORM itself is a plugin. But, that doesn't mean there isn't an iBatis guru out there that would love to learn how to do iBatis + Grails for the folks that needed that.

Anybody out there interested?

BarCamp RDU 2008

This year's BarCampRDU was fantastic. Presentations given this year covered a really diverse set of topics. Frankly, too many topics to cover in this post. Ironically, I forgot to vote...

Talks on Social Networking? Yeah, but that is so last year. Instead I saw talks on iPhone programming and the Fan Programming Language, Cloud Computing using the Amazon EC2, Big Lee himself was there getting a talk going on social controls over Spam, and there were even topics such as Hadoop and Mahout (talk led by Grant Ingersoll no less). Not to mention a crowd of serious people working on incubating technology startups in the RTP area. And yes, a group discussing Groovy and Grails was there and yours truly did more than lurk.

The interesting thing about Grails is how broad an audience it targets. At the moment the core G2One guys (I've only met Jeff Brown) are focused on Java shops. This is probably a good "head of the spear" tactic for them. (See I was paying attention in the startup talk) as the integration story between Java and Groovy and Spring and Grails just totally fantastic. There are only so many hours in the day and only so many people you can talk to. So the choice to focus was probably wise. I think there are some great advocates and allies for Grails waiting out there in frustrated LAMP stack developers who are ready to move to the next level.

The fact that you can dive as deep into Grails as you want to or stay as shallow as you want is so powerful. By leveraging Groovy which by its very construction means I can dig deep into Java if I need to means for small projects that scale out they can be incrementally and iteratively backed off down to Java applications if you need to. Or, you can stay fast an loose on top of the Grails platform cranking out features like there's no tomorrow. And there are many LAMP guys who are going to love that. I think I've met a few smart guys that might actually find a few great ways to help Grails grow into this region.

During the between presentations times I met folks who work at exciting technology companies like rPath, Lulu, Tibco, and through them I've heard about the small Google office here in RTP working on Android related software. And, since I've started to get out and about I know that Sun also has a small group of engineers here in RTP as well as Oracle and the obvious big guns like RedHat, Cisco, Nortel, and IBM.

I think that the RTP area (because of the insane growth seen in Chapel Hill, Durham, Cary, and Raleigh) is just inevitably the next great place to be for new young companies seeking to innovate, invent, and create. If there is a recession it's not here, if any thing I'm seeing a boom town in my backyard. Why aren't there more high profile startups and more technology companies here? I think it's just a perception thing.

I can't mention everyone and everything in this post ... in a scant few hours I got as much out of BarCampRDU as I have some week long conferences ... I'm sure that many of the connections I made at BarCampRDU will help to inform future posts and might even change the course and focus of my career. Yes, it was that good. If you didn't go... you missed out.

Groovy class tricks: Zero Padded Number

Take the example of a SKU number. We need the number to be zero padded. For example the number 10 should print as "0010" and not "10" in any display of this object. These numbers might be generated from some source and handed to us to store as an additional identifier for some object. So the source is a number but the destination is a zero-padded number string.

With groovy we have two ways to tackle this problem. First we could store the value as a string in our domain class. We know that the source data should be a number and the output will always be a string. This has the advantage of showing the zero padded number in our database for when folks do queries.

Now, Groovy automatically creates your getters and setters. You never write them yourself. But the trick here is that the getters and setters still exist that means you can do something like this:

class SkuItem {
    String skuNumber
    public void setSkuNumber(Long number) {
        skuNumber = String.format('%014d',number) // NOTE: zero-pad to 14 characters
    }
    public String toString() {
        return skuNumber
    }
}

Doing the zero-padded number this way means the following code will throw an exception and crash:

def sku = new SkuItem(skuNumber: "10")
println sku

The exception looks like this:

Caught: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object '10' with class 'java.lang.String' to class 'java.lang.Integer'

Which is what we want if we intend to enforce that a skuNumber is actually a number. This code will work:

def sku = new SkuItem(skuNumber: 10)
println sku
sku.skuNumber = 20
println sku

and produces this output:

00000000000010
00000000000020

But now that we know that in Groovy the getters and setters are there we can solve this problem the other way. Storing the string in the domain class means that database reports will see the zero padded numbers but that is inefficient and means that the database is working harder to build string based indexes. These are numbers and working with numbers is faster and more compact.

Knowing that the getters and setters are under the covers even in our groovy constructors we could rewrite our SKU storage class like this:

class SkuItem2 {
    Long skuNumber
    public String getSkuNumber() {
        return String.format('%014d',this.skuNumber)
    }
    public String toString() {
        return this.getSkuNumber()
    }
}

Which has the advantage of storing numbers as numbers but the moment the item is fetched it gets turned into a zero padded string... for example:

def sku2 = new SkuItem2(skuNumber: 30)
println sku2
sku2.skuNumber = 40
println sku2

which now produces the output:

00000000000030
00000000000040

... which just goes to show you get even more out of Groovy when you understand how it works underneath the covers.

Language is a Cognitive Technology

Making the rounds is a story out of the MIT news office about the Piraha tribe whose language does not include any counting numbers. They do have the concepts of more than and greater than and the concepts of few and many. Counting isn't a useful skill for them.

What is most intriguing to me is not the missing counting concepts but this odd quote from the end of the article:

One other discovery of the project is that the Piraha can perform exact matching tasks as long as there is no memory component to them, but once there is a memory component, they approximate their matches. This suggests that language is a cognitive technology that aids humans in memory tasks.

That is an interesting phrase to me: language is a cognitive technology.

Programming languages represent cognitive tools to work with a computer. That is definitely true. If you can consider language itself a technology of the mind or a cognitive tool that is an intriguing idea. Is your ability to only use one language or one programming language akin to only being able to use a screw driver?

If we can consider language itself a technology then we have an interesting new time-line for the progress of technology. We can go back to prehistory tracing the advent of the basic concepts of mathematics building up to today's computers which ostensibly hold the fruit of tens of thousands of years of cognitive technological effort.

And this begs me to question... what if we had started on the wrong path all those millenia ago. What if there are better cognitive platforms to build our systems of thought on top of but we couldn't explore them because our tools of cognition effectively created a platform of thought that prevented us exploring other methods of thinking. There could be ways of thinking and computing we can't even look directly at because they are so alien to us.

This idea cuts both ways. It tells me that if a paradigm is too far removed from mainstream thought it could die because not enough minds can (or will) adopt it. That means there are undoubtedly modern linguistic paradigms that are limiting but also unavoidably entrenched. This is good and bad. One language helps people to be mobile even as it limits them.

Consider how English is changing as speakers in many countries begin to change English to match the paradigms of the local tongues it mingles with. The resultant language may become a world tongue called Panglish in some breathless articles. It's all speculation right now.

But, we see this all the time in Programming languages. We see programmers transplanted from one language into another that bring with them the idioms of the languages they left behind. When this becomes laughable is when what is changing is far more than syntax but entire ways of thinking through problems that are being butchered by forcing the new language to work like the old one.

Here is a counter intuitive idea, but, some limitations are actually liberating. When children play in an open field they will cluster together. If you fence in an area larger than what they are actually occupying they will spread out to fill the space. The same is true about how you think about things.

Too broad a programming framework means that you can't effectively address any specific problem. It means you have too wide open a field. Instead the more a framework leads you to an answer the faster you will move toward a goal. Too efficient and limiting of a framework and you can't steer toward an appropriate design and must instead work around your framework.

Think of it as a volume of space where each point represents a system design. Your framework is like a channel or road driving you toward a region of all the possible system designs you can imagine. There is a volume inside that huge space of all possible systems that contains designs that fulfill your requirements (stated and otherwise) and your framework drives you in the direction of a handful of these. If your framework is too strict there could be a problem where the volume of answers lie in one direction and your framework drives you in another.

That would be an impedance mismatch between the problem and the framework. You would need to modify the problem or the framework to bring them into concert. Or you would have to abandon your framework entirely.

Most of us today are working in a world where our frameworks are pulling us toward solutions that are wholly web-based. That is just a fact of what is in fashion today. It isn't wrong either. The web will finally disassociate the cognitive and the physical. Just as MP3 is slowly killing the CD so too will the web slowly kill off certain legacy computing paradigms.

What do the Piraha know and can think about that we can't? Perhaps they have some insights of genius that are not evident to us who are encumbered with numbered thoughts? I wonder how many of us are like the Piraha happy in our own world but are fast colliding with a world that is driven by new thoughts and ideas utterly alien to us. As alien as numbers are to the Piraha the concepts behind cloud computing and massively parallel computing systems are headed our way. What will the programming languages that help us get there look like? How far off course have we drifted along in the channels of thought carved by our programming infrastructure?

It's just a thought.

PostgreSQL and Grails

Just a few notes on setting up Grails with PostgreSQL and PostgreSQL accounts for JDBC connections.

First you need to have postgreSQL installed. The install varies per OS. I'll let you figure that out since there's plenty of documentation on that part.

Next you need to set up your grails application to connect to the database.

Download a PostgreSQL JDBC driver and copy it into your applications lib folder, for example: myProject/lib/pg74.216.jdbc3.jar in my set up. I don't really need to talk about setting up an application from scratch or anything do I? You've all seen that enough times right?

Next, set up your data-source for using your PostgreSQL driver...

dataSource {
    pooled = false
    dbCreate = "create-drop" // one of 'create', 'create-drop','update'
    url = "jdbc:postgresql://localhost:5432/grails"
    driverClassName = "org.postgresql.Driver"
    username = "grails"
    password = "grails"

    // NOTE: both of these dialects have worked for me. But some people
    // recommend using the net.sf version and not the org.hibernate version.
    // dialect = org.hibernate.dialect.PostgreSQLDialect // honestly, not sure what
    dialect = net.sf.hibernate.dialect.PostgreSQLDialect // the difference is.

}

Right now, this configuration will promptly fail. There is no grails database and no grails user. Setting one up requires a little thinking. Obviously, we need to make the grails database first. Let's get that out of the way and move to the slightly more ... confusing ... stuff.

Setting up a database and user on a fresh install requires you to be the postgres user. This is the username created by the postgres installer that the postgres daemon runs under. I have no idea what my postgres user password is so this is how I run the postgres commands to create the grails user and database...

 $ sudo su
 # su postgres
 > createuser -P grails
 > createdb grails

... as the user postgres still, we set the grails user's password ...

 > psql grails
 grails=# ALTER ROLE grails WITH PASSWORD 'grails' \g
 grails=# \q

... and this would be ready to go except that when we try to login as grails we get an authentication error.

By default Postgres uses the authentication system of the OS for determining the identity of the user. This could be a problem if you plan on having users other than your own username or jetty/jboss/nobody use your databases. Most likely you'll want different users depending on application. So we need to modify the postgres configuration files to allow for users other than the console users to login.

The configuration files are found in /var/lib/pgsql/data or other data directory depending on your specific setup. The file for controlling authentication policies is pg_hba.conf. It is pretty well documented on its own and by default the "method" for authentication is set to ident meaning that if I am username "fred" then I can only use the database as "fred" even if I'm running the application "grails".

So for local and host users to login as a password authenticated user we need to set the "method" to password instead. I've set my pg_hba.conf to read...

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD
# "local" is for Unix domain socket connections only
local   all         all                               password sameuser
# IPv4 local connections:
host    all         all         127.0.0.1/32          password sameuser
# IPv6 local connections:
host    all         all         ::1/128               ident sameuser

... allowing a user from local host to login using a username and password pair. I restart postgres (as my own username not postgres)

 $ sudo /etc/init.d/postgresql restart

... and now from my shell I can login as the grails user to the grails database...

 $ psql -U grails grails
Password for user grails:

... and I supply the password grails and I'm logged in. That means this time when I start my grails project with grails run-app the grails user will authenticate and now my grails project can use postgres for development against the postgres database and we can work against postgres on our development machine.

EDIT:
Your whole Grails configuration file (as of Grails 1.1) should look like this

/** 
 * The top dataSource holds configuration options for ALL
 * environments... I'm presuming you want PostgreSQL in all
 * your environments but you may want to use the default
 * Hypersonic database in development and testing instead.
 */
dataSource {
 pooled = true
 driverClassName = "org.postgresql.Driver"
 // dialect = org.hibernate.dialect.PostgreSQLDialect
 dialect = net.sf.hibernate.dialect.PostgreSQLDialect
}
hibernate {
    cache.use_second_level_cache=true
    cache.use_query_cache=true
    cache.provider_class='com.opensymphony.oscache.hibernate.OSCacheProvider'
}
// environment specific settings
environments {
 development {
  dataSource {
    // one of 'create', 'create-drop','update'
    dbCreate = "create-drop" 
    url="jdbc:postgresql://localhost:5432/dev"
    username = "dev"
    password = "dev"
  }
 }
 test {
  dataSource {
    dbCreate = "update"
    url="jdbc:postgresql://localhost:5432/test"
    driverClassName = "org.postgresql.Driver"
    username = "tester"
    password = "tester"
  }
 }
 production {
  dataSource {
    dbCreate = "update"
    url="jdbc:postgresql://localhost:5432/grails"
    username = "grails"
    password = "grails"
  }
 }
}

API is API, but so is table structure...

It's sometimes hard for developers to remember that table structure is also an API. Yes, it should be considered an internal API for the application you are developing, but it is still part of a system that must be carefully constructed to reflect the reality in which it operates. It is not enough to just normalize data following the rules of database normalization... not anymore anyway... it is important to consider how you will mash-up and add onto a database schema.

Consider RESTful webservices as a unique opportunity to add data into an enterprise system. You have a URI for a resource exposed by a restful API. That URI can then be bookmarked and annotated in a system like del.icio.us for your internal users.

That kind of annotation is powerful. It means you can add tags, wiki, and annotated information to CRM pages, BOM, or other data independent of the underlying system implementation. And, not surprizingly, Grails is well situated for this. Grails projects with Content Negotation can be viewed as amazing polymorphic services that display web pages to browsers, XML to services, and JSON to ajax clients.

That flexibility means you can "bookmark" ala social bookmarking in multiple contexts ... these bookmarks can carry meta data about a linked URI. That meta-data can be anything from contact notes to wiki entries. That could mean a new era of business applications that provide semantic inter-sections of web URI based on simple RESTful principles.

Imagine a BOM bookmarked and annotated in a del.icio.us type interface by different departments. A kitting department links the BOM to a history list that represents activities of kitting. Billing annotates accounting activity by bookmarking and logging the original BOM. The accounting system could be .NET, the kitting in PHP... it wouldn't matter. And you wouldn't need SOAP or known WSDL. The core system would be simpler and concerned with just web-page functions... and (using Grails) a scant few lines for context sensitivity to what the controller renders (XML, HTML, Ajax, JSON, etc.) ...meaning you can develop supporting systems each orthogonal to the data source system.

Think of it as meta-Aspect Oriented Programming if that makes you feel better about it. But, really, it's all about using REST and the web for what they are good at and leveraging the power of the browser as a broker platform. I've heard it called Browser As A Broker (BAAB) and also Enterprise Service Browser (ESB). I think I'll call it... simple and to the point.