2008-07-23

Groovy class tricks: Zero Padded Number

Take the example of a SKU number. We need the number to be zero padded. For example the number 10 should print as "0010" and not "10" in any display of this object. These numbers might be generated from some source and handed to us to store as an additional identifier for some object. So the source is a number but the destination is a zero-padded number string.

With groovy we have two ways to tackle this problem. First we could store the value as a string in our domain class. We know that the source data should be a number and the output will always be a string. This has the advantage of showing the zero padded number in our database for when folks do queries.

Now, Groovy automatically creates your getters and setters. You never write them yourself. But the trick here is that the getters and setters still exist that means you can do something like this:
class SkuItem {
String skuNumber
public void setSkuNumber(Long number) {
skuNumber = String.format('%014d',number) // NOTE: zero-pad to 14 characters
}
public String toString() {
return skuNumber
}
}

Doing the zero-padded number this way means the following code will throw an exception and crash:
def sku = new SkuItem(skuNumber: "10")
println sku

The exception looks like this:
Caught: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object '10' with class 'java.lang.String' to class 'java.lang.Integer'

Which is what we want if we intend to enforce that a skuNumber is actually a number. This code will work:
def sku = new SkuItem(skuNumber: 10)
println sku
sku.skuNumber = 20
println sku

and produces this output:
00000000000010
00000000000020

But now that we know that in Groovy the getters and setters are there we can solve this problem the other way. Storing the string in the domain class means that database reports will see the zero padded numbers but that is inefficient and means that the database is working harder to build string based indexes. These are numbers and working with numbers is faster and more compact.

Knowing that the getters and setters are under the covers even in our groovy constructors we could rewrite our SKU storage class like this:
class SkuItem2 {
Long skuNumber
public String getSkuNumber() {
return String.format('%014d',this.skuNumber)
}
public String toString() {
return this.getSkuNumber()
}
}

Which has the advantage of storing numbers as numbers but the moment the item is fetched it gets turned into a zero padded string... for example:

def sku2 = new SkuItem2(skuNumber: 30)
println sku2
sku2.skuNumber = 40
println sku2

which now produces the output:
00000000000030
00000000000040

... which just goes to show you get even more out of Groovy when you understand how it works underneath the covers.

2008-07-15

Language is a Cognitive Technology

Making the rounds is a story out of the MIT news office about the Piraha tribe whose language does not include any counting numbers. They do have the concepts of more than and greater than and the concepts of few and many. Counting isn't a useful skill for them.

What is most intriguing to me is not the missing counting concepts but this odd quote from the end of the article:
One other discovery of the project is that the Piraha can perform exact matching tasks as long as there is no memory component to them, but once there is a memory component, they approximate their matches. This suggests that language is a cognitive technology that aids humans in memory tasks.


That is an interesting phrase to me: language is a cognitive technology.

Programming languages represent cognitive tools to work with a computer. That is definitely true. If you can consider language itself a technology of the mind or a cognitive tool that is an intriguing idea. Is your ability to only use one language or one programming language akin to only being able to use a screw driver?

If we can consider language itself a technology then we have an interesting new time-line for the progress of technology. We can go back to prehistory tracing the advent of the basic concepts of mathematics building up to today's computers which ostensibly hold the fruit of tens of thousands of years of cognitive technological effort.

And this begs me to question... what if we had started on the wrong path all those millenia ago. What if there are better cognitive platforms to build our systems of thought on top of but we couldn't explore them because our tools of cognition effectively created a platform of thought that prevented us exploring other methods of thinking. There could be ways of thinking and computing we can't even look directly at because they are so alien to us.

This idea cuts both ways. It tells me that if a paradigm is too far removed from mainstream thought it could die because not enough minds can (or will) adopt it. That means there are undoubtedly modern linguistic paradigms that are limiting but also unavoidably entrenched. This is good and bad. One language helps people to be mobile even as it limits them.

Consider how English is changing as speakers in many countries begin to change English to match the paradigms of the local tongues it mingles with. The resultant language may become a world tongue called Panglish in some breathless articles. It's all speculation right now.

But, we see this all the time in Programming languages. We see programmers transplanted from one language into another that bring with them the idioms of the languages they left behind. When this becomes laughable is when what is changing is far more than syntax but entire ways of thinking through problems that are being butchered by forcing the new language to work like the old one.

Here is a counter intuitive idea, but, some limitations are actually liberating. When children play in an open field they will cluster together. If you fence in an area larger than what they are actually occupying they will spread out to fill the space. The same is true about how you think about things.

Too broad a programming framework means that you can't effectively address any specific problem. It means you have too wide open a field. Instead the more a framework leads you to an answer the faster you will move toward a goal. Too efficient and limiting of a framework and you can't steer toward an appropriate design and must instead work around your framework.

Think of it as a volume of space where each point represents a system design. Your framework is like a channel or road driving you toward a region of all the possible system designs you can imagine. There is a volume inside that huge space of all possible systems that contains designs that fulfill your requirements (stated and otherwise) and your framework drives you in the direction of a handful of these. If your framework is too strict there could be a problem where the volume of answers lie in one direction and your framework drives you in another.

That would be an impedance mismatch between the problem and the framework. You would need to modify the problem or the framework to bring them into concert. Or you would have to abandon your framework entirely.

Most of us today are working in a world where our frameworks are pulling us toward solutions that are wholly web-based. That is just a fact of what is in fashion today. It isn't wrong either. The web will finally disassociate the cognitive and the physical. Just as MP3 is slowly killing the CD so too will the web slowly kill off certain legacy computing paradigms.

What do the Piraha know and can think about that we can't? Perhaps they have some insights of genius that are not evident to us who are encumbered with numbered thoughts? I wonder how many of us are like the Piraha happy in our own world but are fast colliding with a world that is driven by new thoughts and ideas utterly alien to us. As alien as numbers are to the Piraha the concepts behind cloud computing and massively parallel computing systems are headed our way. What will the programming languages that help us get there look like? How far off course have we drifted along in the channels of thought carved by our programming infrastructure?

It's just a thought.

2008-07-10

PostgreSQL and Grails

Just a few notes on setting up Grails with PostgreSQL and PostgreSQL accounts for JDBC connections.

First you need to have postgreSQL installed. The install varies per OS. I'll let you figure that out since there's plenty of documentation on that part.

Next you need to set up your grails application to connect to the database.

Download a PostgreSQL JDBC driver and copy it into your applications lib folder, for example: myProject/lib/pg74.216.jdbc3.jar in my set up. I don't really need to talk about setting up an application from scratch or anything do I? You've all seen that enough times right?

Next, set up your data-source for using your PostgreSQL driver...

dataSource {
pooled = false
dbCreate = "create-drop" // one of 'create', 'create-drop','update'
url = "jdbc:postgresql://localhost:5432/grails"
driverClassName = "org.postgresql.Driver"
username = "grails"
password = "grails"

// NOTE: both of these dialects have worked for me. But some people
// recommend using the net.sf version and not the org.hibernate version.
// dialect = org.hibernate.dialect.PostgreSQLDialect // honestly, not sure what
dialect = net.sf.hibernate.dialect.PostgreSQLDialect // the difference is.

}


Right now, this configuration will promptly fail. There is no grails database and no grails user. Setting one up requires a little thinking. Obviously, we need to make the grails database first. Let's get that out of the way and move to the slightly more ... confusing ... stuff.

Setting up a database and user on a fresh install requires you to be the postgres user. This is the username created by the postgres installer that the postgres daemon runs under. I have no idea what my postgres user password is so this is how I run the postgres commands to create the grails user and database...

$ sudo su
# su postgres
> createuser -P grails
> createdb grails

... as the user postgres still, we set the grails user's password ...

> psql grails
grails=# ALTER ROLE grails WITH PASSWORD 'grails' \g
grails=# \q

... and this would be ready to go except that when we try to login as grails we get an authentication error.

By default Postgres uses the authentication system of the OS for determining the identity of the user. This could be a problem if you plan on having users other than your own username or jetty/jboss/nobody use your databases. Most likely you'll want different users depending on application. So we need to modify the postgres configuration files to allow for users other than the console users to login.

The configuration files are found in /var/lib/pgsql/data or other data directory depending on your specific setup. The file for controlling authentication policies is pg_hba.conf. It is pretty well documented on its own and by default the "method" for authentication is set to ident meaning that if I am username "fred" then I can only use the database as "fred" even if I'm running the application "grails".

So for local and host users to login as a password authenticated user we need to set the "method" to password instead. I've set my pg_hba.conf to read...
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD
# "local" is for Unix domain socket connections only
local all all password sameuser
# IPv4 local connections:
host all all 127.0.0.1/32 password sameuser
# IPv6 local connections:
host all all ::1/128 ident sameuser

... allowing a user from local host to login using a username and password pair. I restart postgres (as my own username not postgres)

$ sudo /etc/init.d/postgresql restart

... and now from my shell I can login as the grails user to the grails database...

$ psql -U grails grails
Password for user grails:

... and I supply the password grails and I'm logged in. That means this time when I start my grails project with grails run-app the grails user will authenticate and now my grails project can use postgres for development against the postgres database and we can work against postgres on our development machine.

EDIT:
Your whole Grails configuration file (as of Grails 1.1) should look like this

/**
* The top dataSource holds configuration options for ALL
* environments... I'm presuming you want PostgreSQL in all
* your environments but you may want to use the default
* Hypersonic database in development and testing instead.
*/
dataSource {
pooled = true
driverClassName = "org.postgresql.Driver"
// dialect = org.hibernate.dialect.PostgreSQLDialect
dialect = net.sf.hibernate.dialect.PostgreSQLDialect
}
hibernate {
cache.use_second_level_cache=true
cache.use_query_cache=true
cache.provider_class='com.opensymphony.oscache.hibernate.OSCacheProvider'
}
// environment specific settings
environments {
development {
dataSource {
// one of 'create', 'create-drop','update'
dbCreate = "create-drop"
url="jdbc:postgresql://localhost:5432/dev"
username = "dev"
password = "dev"
}
}
test {
dataSource {
dbCreate = "update"
url="jdbc:postgresql://localhost:5432/test"
driverClassName = "org.postgresql.Driver"
username = "tester"
password = "tester"
}
}
production {
dataSource {
dbCreate = "update"
url="jdbc:postgresql://localhost:5432/grails"
username = "grails"
password = "grails"
}
}
}

2008-07-07

API is API, but so is table structure...

It's sometimes hard for developers to remember that table structure is also an API. Yes, it should be considered an internal API for the application you are developing, but it is still part of a system that must be carefully constructed to reflect the reality in which it operates. It is not enough to just normalize data following the rules of database normalization... not anymore anyway... it is important to consider how you will mash-up and add onto a database schema.

Consider RESTful webservices as a unique opportunity to add data into an enterprise system. You have a URI for a resource exposed by a restful API. That URI can then be bookmarked and annotated in a system like del.icio.us for your internal users.

That kind of annotation is powerful. It means you can add tags, wiki, and annotated information to CRM pages, BOM, or other data independent of the underlying system implementation. And, not surprizingly, Grails is well situated for this. Grails projects with Content Negotation can be viewed as amazing polymorphic services that display web pages to browsers, XML to services, and JSON to ajax clients.

That flexibility means you can "bookmark" ala social bookmarking in multiple contexts ... these bookmarks can carry meta data about a linked URI. That meta-data can be anything from contact notes to wiki entries. That could mean a new era of business applications that provide semantic inter-sections of web URI based on simple RESTful principles.

Imagine a BOM bookmarked and annotated in a del.icio.us type interface by different departments. A kitting department links the BOM to a history list that represents activities of kitting. Billing annotates accounting activity by bookmarking and logging the original BOM. The accounting system could be .NET, the kitting in PHP... it wouldn't matter. And you wouldn't need SOAP or known WSDL. The core system would be simpler and concerned with just web-page functions... and (using Grails) a scant few lines for context sensitivity to what the controller renders (XML, HTML, Ajax, JSON, etc.) ...meaning you can develop supporting systems each orthogonal to the data source system.

Think of it as meta-Aspect Oriented Programming if that makes you feel better about it. But, really, it's all about using REST and the web for what they are good at and leveraging the power of the browser as a broker platform. I've heard it called Browser As A Broker (BAAB) and also Enterprise Service Browser (ESB). I think I'll call it... simple and to the point.

2008-07-02

Hide complex behind simple...

Hide the complex behind the simple. Keep the complexity of a system behind the simplest possible interface. At every level find the simplest way to express the intention of the interface, do that, and get out of the way. In short, make things easy for your user and hard on you. It's your job.

If you are creating API (as I sometimes find myself doing) work to create the interfaces that the users of your API will find intuitive and simple to use. Hide the complexity of this work as succinctly as possible behind the inner implementation of the library.

If you are working on UI (as I also paradoxically find myself doing) work to create an interface that encourages play. I don't mean gaming. I mean, allow a user to "play with" the interface and explore it. Let them discover what it can do and try and make it intuitive to the way that they think about problems.

In the desing of both UI and API you can find yourself altering what "simple" is. You can introduce new ways of thinking about a problem by creating new implicit behaviors. This is an interesting space to work in collaboratively because you can find that users (both programmers and lay-users) have developed preconceptions and notions of how a system "must work" and so doing box themselves in.

In my experience of hiding complex things under simple interfaces I have found that there really is nothing that is truly simple in the whole. Instead things are simple in isolation but produce emergent complexity when placed in large systems. So to fight this the best weapon is to keep systems small. By keeping systems small and isolating their inputs and outputs you have a better chance of artfully controlling their interactions.

The other side of this is that our minds are horrible at containing inherent complexity. I used to think I was very good at this. I no longer think that. I now realize that I might be better at dealing with complexity than some, but my abilities have a distinct upper limit that is reached very quickly.

That is why when we build systems of any size. We developers of systems of software should acknowledge the need to test the system to validate our understanding at every level. Just as we use the scientific method to validate what we believe to be true using experimentation we should validate the way thoughts interact via experimentation.

The irony that what we are testing is in my opinion codified thought is not lost on me. Consider Kurt Gödel and his incompleteness theorems which point to the limitations of all formal systems... they are doomed to incompleteness. That is to say that you can never through proof alone (code alone) create a system to model such complex yet simple things as the counting numbers.

So there is a bug in the universe?

When you or I create software in any capacity we are indeed creating a formal system. A system that will either fail to model the real world problem we are attempting to emulate or will be fraught with inconsistency and incompleteness. So even if we were super-coders that never produced bugs, and you and I could in fact produce a perfect system, that system could be fraught with problems due to its inability to properly model or express the real world system it was based on.

The simplest answer is to devise tests to prove the correctness of our systems.