According to Kurzweil your IT systems will be twice as powerful in 3 years. By 2013 or so we should be able to model a human brain in entirety on the world's most powerful machines. In time that power will hopefully trickle down to you and me. If you are like me, you write software, how are you going to use all that power? How are you going to produce all those features people need so rapidly?
If you've read my blog at all you'll know I'll tout the advance of the DSL and the dynamic language. I see these technologies on the JVM at a key inflection point between backwards compatibility and increased efficacy. They are transitional technologies at a key period in technological history.
The win in DSL, dynamic languages, and frameworks is always how succinctly you can get to your core thought or core idea. I don't really care about the ORM, the Ajax, the email most of the time. Most of the time there is a core idea I'm trying to work out.
In 24 months I need to be in the place where I can effectively divide the problems in front of me down into their key ideas. Keep the competing paradigms apart from each other. And then combine them at the right moments of confluence with a minimum of dissonance.
Get to the core thought, the core idea, ignore as much minutiae as possible, get the problem solved. Keep the idea clean. Keep the core thought from being tangled in a mess of details that have nothing to do with the answer you are after.
Advances in the way we create software will be key to the advance of the technologies of the future. If you are writing software today you will have to be prepared to leap a mighty chasm approaching us at break neck speed. That chasm will be paradoxically caused the increase of computational power available to us.
The problem of "traditional" programming that it is very procedural which can feel plodding at times. It takes forever to say anything meaningful. Yet before long we will have the ability to command computational power that could out strip our ability to speak into these systems with these sometimes unnecessarily verbose languages.
Consider that (in the video linked above) Kurzweil says that a scant 30 million bytes describes the whole of the human brain. The reason 30 million bytes could possibly hold the whole design for the human brain is that the brain's design is fractal and self organizing. A uniform underlying design is repeated with variation to create a whole. This is a similar goal to what we have in DRY (Don't Repeat Yourself) code that means we express an idea and leverage that idea again and again.
In the case of our brains, that assembled whole adapts to inputs to form intelligent variation leading to astounding complexity and abilities. The subtle variation introduced by input creates the mote of a soul... or at the very least all the personality traits we recognize as a living person. If only we could write software that could reorganize itself like that.
Functional programming, rules engines, and dynamic languages promise to free us from those mind-killing details that we don't really care about anymore as application developers. But they don't really address the need to create adaptive systems or systems that are able to span across computational mediums.
I predict that there will have to be new programming languages that run in new ways that allow for fractal designs like the human brain. I happen to think these still need to be languages because they are still conversations between a creator and the created system. I dream about these systems acting like holograms or fractals in their nature.
If you cut a hologram in half you get two identical but smaller and fuzzier holograms. If you look at one part of a fractal you see the whole. I think human designed software can have the same quality. The idea isn't new I've heard the opposite idea called "dirt" and perhaps that's a good word but in my mind these form knots ... asymmetric distributions of thought where one idea unrelated to another must intersect in incongruent ways. Perhaps "dirt" is the antithesis of "fractal" or "crystal" design.
I say "fractal" or "crystal" because "modular" isn't strong enough. You can make modules all day long but if they don't allow for cleanly intersecting concerns then you don't haven't done much more than take a messy room and shovel it's contents into random boxes. You haven't actually organized things you've just cut down on the size of the mess you have to sift through.
Good code focuses on one concern at a time and tries to ignore details that are incidental. It should be that in daily programming activities we can use tools like AOP and DSL to focus on the foremost thought ignoring details of execution. Good code is reusable because it encapsulates one concern in a way that is clean enough and simple enough to be easily lifted out and reused.
That's good stuff because if you're like me you really get paid for some high level thought and some high level idea and not the millions of tiny details you have to move through to get there.
2007-12-27
2007-12-26
Code Size and the Cognitive Gordian Knot
The size of a code base is not the worst enemy of code.
My personal experiences with code bases large and small have given me the insight to realize that size of code is a terrible enemy but not our worst enemy. Our worst enemy is stupidity. Well, not out right stupidity but what I call the cognitive Gordian knot.
The larger a code base is the more material there is to create a knot. The smaller, the less material there is to create a mess with. So there is beauty in the small. I have been arguing for the decomposition of large projects into small reusable code units for most of my career.
Yet, I've seen examples at Coding Horror and The Daily WTF that show in a scant few lines some of the most impressive inability to think I have ever seen. These examples are not necessarily what we would consider mind numbingly, guffawingly, powerfully brutish idiocy but subtle brain warping WTF? These are the thought-knots we can't untie like that knot of legend the Gordian knot.
During my time over seas I have worked on code bases whose ages were measured in decades and sizes estimated in rough fractions of millions of lines. Code bases of this size are surely incomprehensible but the best can have inner structure that can be navigated and internal API that can be understood. The reader of this kind of code doesn't need to know what exactly each call does but only understand the local code and some specific libraries to work on it.
And then later in my career... oh the irony... I maintained a code base of a scant few hundred thousand lines of code whose internal structure shifted upon itself every few lines. Every few hundred lines a new API, a new convention, a re-written API, a half-refactor, a violation of another API. And so on the insanity folded over and over upon itself like a Lewis Caroll narrative without any beautiful onomatopoeia. All of it written by the same person over the course of five years.
Instead of a crystalline structure found in those majestic and old code bases whose patterns fold and unfold like a fractal thus leading the maintainer to an understanding of the whole from only a part... this newer... this smaller... this uglier code base contained no song and no resonance that my mind could fix upon. Instead it was a tangle of swamp weed coated in a toxic sludge that defied understanding and like "Alexander the great and the Gordian knot" I saw no other answer to my problems other than to cut it asunder. Bust it up, break it apart, make it small again.
No, I know better, size is not what kills code. It is the Gordian knot of superficial complexity that kills code. Instead of a crystalline pattern or an enforced internal structure the code is allowed to grow untamed. The key to long life in code is pulling out and separating concerns into libraries and frameworks.
You could argue this tactic constitutes removing some code from the main code base and creating small supporting projects that fit together to create a whole. You'd be right. That's what it is. That is what I mean by saying fractal, crystalline, or internally disciplined. The supporting libraries may be internal to the project but they may constitute sub-projects they divide and conquer the problems and keep the material from building to the point that it can accidentally tie itself in a knot.
You could achieve this using MVC, Services, or libraries. I've seen Domain Driven Designs and Spring annotations to enforce SoC that would work well. You could say this is how Linux and GNU work together to create a full operating system. You could say that is code reuse and proper object orientation. You could say this the weapon we have in the mix-in and dynamic language.
Whatever you call it. It makes code easier to understand because it has a logical structure and what I call a resonance or song. It means you can see part of a code base, understand it, and ignore the other concerns to focus on your particular problem.
If you can dive into one section of code and comprehend it without having to comprehend the whole thing then I say you have a good code base no matter how big or how small it is. The goal is to keep the knotty beast at bay for as long as possible using every trick we have.
My personal experiences with code bases large and small have given me the insight to realize that size of code is a terrible enemy but not our worst enemy. Our worst enemy is stupidity. Well, not out right stupidity but what I call the cognitive Gordian knot.
The larger a code base is the more material there is to create a knot. The smaller, the less material there is to create a mess with. So there is beauty in the small. I have been arguing for the decomposition of large projects into small reusable code units for most of my career.
Yet, I've seen examples at Coding Horror and The Daily WTF that show in a scant few lines some of the most impressive inability to think I have ever seen. These examples are not necessarily what we would consider mind numbingly, guffawingly, powerfully brutish idiocy but subtle brain warping WTF? These are the thought-knots we can't untie like that knot of legend the Gordian knot.
During my time over seas I have worked on code bases whose ages were measured in decades and sizes estimated in rough fractions of millions of lines. Code bases of this size are surely incomprehensible but the best can have inner structure that can be navigated and internal API that can be understood. The reader of this kind of code doesn't need to know what exactly each call does but only understand the local code and some specific libraries to work on it.
And then later in my career... oh the irony... I maintained a code base of a scant few hundred thousand lines of code whose internal structure shifted upon itself every few lines. Every few hundred lines a new API, a new convention, a re-written API, a half-refactor, a violation of another API. And so on the insanity folded over and over upon itself like a Lewis Caroll narrative without any beautiful onomatopoeia. All of it written by the same person over the course of five years.
Instead of a crystalline structure found in those majestic and old code bases whose patterns fold and unfold like a fractal thus leading the maintainer to an understanding of the whole from only a part... this newer... this smaller... this uglier code base contained no song and no resonance that my mind could fix upon. Instead it was a tangle of swamp weed coated in a toxic sludge that defied understanding and like "Alexander the great and the Gordian knot" I saw no other answer to my problems other than to cut it asunder. Bust it up, break it apart, make it small again.
No, I know better, size is not what kills code. It is the Gordian knot of superficial complexity that kills code. Instead of a crystalline pattern or an enforced internal structure the code is allowed to grow untamed. The key to long life in code is pulling out and separating concerns into libraries and frameworks.
You could argue this tactic constitutes removing some code from the main code base and creating small supporting projects that fit together to create a whole. You'd be right. That's what it is. That is what I mean by saying fractal, crystalline, or internally disciplined. The supporting libraries may be internal to the project but they may constitute sub-projects they divide and conquer the problems and keep the material from building to the point that it can accidentally tie itself in a knot.
You could achieve this using MVC, Services, or libraries. I've seen Domain Driven Designs and Spring annotations to enforce SoC that would work well. You could say this is how Linux and GNU work together to create a full operating system. You could say that is code reuse and proper object orientation. You could say this the weapon we have in the mix-in and dynamic language.
Whatever you call it. It makes code easier to understand because it has a logical structure and what I call a resonance or song. It means you can see part of a code base, understand it, and ignore the other concerns to focus on your particular problem.
If you can dive into one section of code and comprehend it without having to comprehend the whole thing then I say you have a good code base no matter how big or how small it is. The goal is to keep the knotty beast at bay for as long as possible using every trick we have.
2007-12-19
Fun with Groovy and the Reflection API
Here's the scenario: You've got dates in the format "MM/dd/yyyy" to be parsed and turned into a real Date object. You would parse the date like so:
And you have hundreds of collections of Java and Groovy objects that your library could be potentially mapping from hashes that look something like this:
You could write a switch statement for each of these... or you could go and ask the class defintions for what the types of foo.id and foo.started are. But if you do this:
... that println will print "null" because the started object is set to null. So how do I know if "started" is a date or not? I need to know the type on the other side of the "started" member. It would be nice to have a reverse mapping I could use... so I wrote this:
... now you can ask the type of a field like so ...
... to get the type of the field on the class you are working with... in a loop you would...
Presuming you knew that all the values of "val" were strings and all the members in "Foo" had constructors that could handle a string argument intelligently. I cache up the type map else where so that the real production classOf doesn't build a new type map on each pass.
But now I can map from arbitrary strings into real objects and back without having to specify a "mapping" for each object.
public Date format(String str) {
def format = new SimpleDateFormat( "MM/dd/yyyy" )
return format.parse(str)
}
public String format(Date date) {
def format = new SimpleDateFormat( "MM/dd/yyyy" )
return format.format(date)
}
And you have hundreds of collections of Java and Groovy objects that your library could be potentially mapping from hashes that look something like this:
["foo":[ "id" : "5", "started":"12/11/2007", "name":"bob" ]]
You could write a switch statement for each of these... or you could go and ask the class defintions for what the types of foo.id and foo.started are. But if you do this:
class Foo {
Long id
Date started
String name
}
def foo = new Foo()
println foo.started.getClass()
... that println will print "null" because the started object is set to null. So how do I know if "started" is a date or not? I need to know the type on the other side of the "started" member. It would be nice to have a reverse mapping I could use... so I wrote this:
def getTypeMap(Class clazz) {
def typeMap = [:]
def members = clazz.getDeclaredFields()
members.each( {
m ->
typeMap[m.getName()] = m.getType()
} )
return typeMap
}
... now you can ask the type of a field like so ...
def classOf(Class clazz, field) {
def typeMap = getTypeMap(clazz)
if(typeMap.containsKey(field)) {
return typeMap[field]
}
else {
return null
}
}
... to get the type of the field on the class you are working with... in a loop you would...
def params = ["id":"5","started":"12/11/2007","name":"bob"]
def foo = new Foo()
params.each({ key,val ->
def clazz = classOf(foo.getClass(),key)
if(clazz && clazz != Date.class) {
foo[key] = clazz.newInstance(val)
}
else if(clazz) {
foo[key] = (Date) format(val)
}
})
Presuming you knew that all the values of "val" were strings and all the members in "Foo" had constructors that could handle a string argument intelligently. I cache up the type map else where so that the real production classOf doesn't build a new type map on each pass.
But now I can map from arbitrary strings into real objects and back without having to specify a "mapping" for each object.
Seven Sins of Bad Groovy Programming
Jörg Staudemeyer has posted on the groovy-user forum at nabble a list of seven sins committed in groovy and why they are bad. A good read for anyone new to groovy.
2007-12-18
Single Sign On with JBoss Portal and Active Directory
No one has asked for it but I've begun work on a SSO project when I'm between official projects. I discovered the power of Acegi and CAS. I have an install of the JBoss portal as well and that requires its own set up too... however it is looking like we are going to use Joomla instead.
I began with understanding how to talk to an ldap server (in my case Active Directory). Once I understood how to query LDAP I began searching around using my groovy scripts to find the object classes, and specific names I would need inside my CAS configuration.
I then followed this tutorial for CAS to authenticate the users:
Next I built in the CAS client into JBoss portal using this wiki page's instructions:
Finding the specific settings were hard. In particular I had to muck about with the ldap_identity-config.xml deep inside our jboss-portal.sar file... it turns out our AD setup is somewhat "special" and needed some extra care beyond what is in this page:
I have no idea how I would have figured out how to do that kind of configuration without that wiki page.
Once I managed to stitch together both sets of instructions I could have my users authenticate via CAS and then the portal would query roles on its own against Active Directory.
Next I'll investigate how to bring Groovy and Grails into this mix. I'll have a project for delivery in March that will need to hook into this SSO system... and likely subsequent projects to link into the CAS system using PHP and Perl.
I began with understanding how to talk to an ldap server (in my case Active Directory). Once I understood how to query LDAP I began searching around using my groovy scripts to find the object classes, and specific names I would need inside my CAS configuration.
I then followed this tutorial for CAS to authenticate the users:
Next I built in the CAS client into JBoss portal using this wiki page's instructions:
Finding the specific settings were hard. In particular I had to muck about with the ldap_identity-config.xml deep inside our jboss-portal.sar file... it turns out our AD setup is somewhat "special" and needed some extra care beyond what is in this page:
I have no idea how I would have figured out how to do that kind of configuration without that wiki page.
Once I managed to stitch together both sets of instructions I could have my users authenticate via CAS and then the portal would query roles on its own against Active Directory.
Next I'll investigate how to bring Groovy and Grails into this mix. I'll have a project for delivery in March that will need to hook into this SSO system... and likely subsequent projects to link into the CAS system using PHP and Perl.
2007-12-12
BPM is mostly wrong
In response to:
The Seven Fallacies of BPM
Mentioned in "The World is Flat" book Business Process Modeling systems were supposed to deliver businesses with a glorious future in which Business Analysts could model processes without the need for a programmer's intervention.
The problem is that the modeling process itself is core to application development and utterly central and important to a properly functioning application. You would no more have a Doctor design your house than you would a Business Analyst design a Database or Application. The Doctor is concerned with much different sets of concerns than an Architect is. The Doctor is concerned about wheel chair access and ventilation while the Architect is concerned with whether the building will stand up to a storm or earthquake.
While both a Doctor and Architect could collaborate to create a house you would not have a Doctor build a house unassisted. If you did you would be inviting disaster. And, you would be offended if I suggested that you take dietary advice from an Architect. That would be absurd.
It is true that many small projects are released without the help of a specialist. I'm certain many buildings are built without an Architect... and many diseases are overcome without a Doctor. But, you should ask yourself what your project is? Are you trying to produce a professional product or are you just "hacking around" looking to make something that works? Do you want to create "professional" systems or just "hobby" systems. Are you producing a reliable car or are you creating a dune buggy in your back yard?
If you can honestly answer that the project you are producing is a "buggy" then by all means forgo the professional assistance ... but don't kid yourself... that's not a Toyota you've got.
The Seven Fallacies of BPM
Mentioned in "The World is Flat" book Business Process Modeling systems were supposed to deliver businesses with a glorious future in which Business Analysts could model processes without the need for a programmer's intervention.
The problem is that the modeling process itself is core to application development and utterly central and important to a properly functioning application. You would no more have a Doctor design your house than you would a Business Analyst design a Database or Application. The Doctor is concerned with much different sets of concerns than an Architect is. The Doctor is concerned about wheel chair access and ventilation while the Architect is concerned with whether the building will stand up to a storm or earthquake.
While both a Doctor and Architect could collaborate to create a house you would not have a Doctor build a house unassisted. If you did you would be inviting disaster. And, you would be offended if I suggested that you take dietary advice from an Architect. That would be absurd.
It is true that many small projects are released without the help of a specialist. I'm certain many buildings are built without an Architect... and many diseases are overcome without a Doctor. But, you should ask yourself what your project is? Are you trying to produce a professional product or are you just "hacking around" looking to make something that works? Do you want to create "professional" systems or just "hobby" systems. Are you producing a reliable car or are you creating a dune buggy in your back yard?
If you can honestly answer that the project you are producing is a "buggy" then by all means forgo the professional assistance ... but don't kid yourself... that's not a Toyota you've got.
2007-12-08
Learning the Eclipse RCP
While I'm learning grails I'm also learning the Eclipse RCP. I'm hoping to merge the two efforts at some point.
2007-12-07
telecomputational
Heard in conversation just now:
It's the dawn of a new era.
You have a telecomputational device don't you?
--Billy Howell
It's the dawn of a new era.
Grails 1.0-RC2 sprites and stogies
So, I've run on over to check out the Grails 1.0-RC2 release and see if all my favorite bugs were fixed. They were! And there were birds chirruping and rainbows and a unicorn ... and a sprite lit a fresh stogy for me and one for Bender. Then I tried to do something a little unusual. Should I? Dare I? Yes.
I tried to do a one-to-many mapping where the "one" side was a Groovy domain class and the many side was a JPA annotated class (You know an EJB3 entity bean). And it failed. Oh, the misery and woe! Well, okay it's not quite that bad.
So the lesson learned children? Don't do drugs, stay in school, use JPA or use GORM, be careful using both. I have filed this bug hoping that next time the stogy is a fine cuban cigar: GRAILS-1983. Bender likes fine cigars.
I tried to do a one-to-many mapping where the "one" side was a Groovy domain class and the many side was a JPA annotated class (You know an EJB3 entity bean). And it failed. Oh, the misery and woe! Well, okay it's not quite that bad.
So the lesson learned children? Don't do drugs, stay in school, use JPA or use GORM, be careful using both. I have filed this bug hoping that next time the stogy is a fine cuban cigar: GRAILS-1983. Bender likes fine cigars.
Subscribe to:
Posts (Atom)