2012-04-18

The answer is 5

Many, many, years ago I got the chance to work with a highly seasoned developer. This guy had helped write tools like make. Yes kids... he worked on an early version of make. So as he came up to speed on our new system I left him be, figuring who the heck was I to direct his work at all? I was just some schlub who started at the company earlier.

This seasoned developer got under the gun on his first project at our little company and we had a monthly UAT cycle. In the process this company used: two weeks out, you would freeze new development and then turn the whole system over to test. The new system development would start on a new branch as we practiced "clean trunk, dirty branch" style of version management. (Today I prefer dirty trunk, clean branch but I digress.)

UAT went badly and as was the policy in this company the responsible developer had to turn around and fix the bugs found by the human testers (and turn it around rapidly) so the whole human driven Testing Cycle could start all over again. Some cycles went smooth. Some cycles were hell as the release date never moved and people would have to work longer and longer hours as deadlines loomed closer and closer.

As was typical, a client might hold off on their contractual obligations to test. They might wait until the day before delivery to test. We had no automated testing at this little company. In those days things like that were not common. There was no clause that let us move the deadline. That meant if that client reported something that prevented the project from shipping we would have to work through the night to fix the bug an then sit attentively waiting for the client to test again and approve.

So when this seasoned paragon of coders came upon just such a situation what did he do? He fixed the bug in nothing flat and got everything through UAT late that night. He was a hero.

The UAT scripts were run that night were all written to test for "5" and so there was a bug in the UAT process. We did not retest *everything* after a failure. We only retested the failed things. We did not retest everything before we shipped... we only tested the failures. But, can you blame the testers? They were tired. They had been up late for days testing. They had to repeat the same boring actions over and over and over... can you blame them for beginning to short cut tests?

I looked over the changes and perhaps it was because I was a fresh pair of eyes... but I swear... I could see it right away. Somehow, the code entered a tautological loop that I don't remember exactly suffice to say... the only answer it could ever return was "5".  But, because of the process I couldn't stop the deploy and I couldn't slip-stream the changes because I had to wait for the next cycle of UAT.

We would end up in an eight month long non-stop fire-fight due to that one mistake. When the fires finally died down we would have a staff party and celebrate our victory. Yet we would do little about our mistake other than remember it and make jokes about "5" being the right answer to everything.

To this day if you ask me a question and I'm tired I may respond with a whimsical "5" ... and all my coworkers from that name-less and thank-less job (you know who you are) will chuckle along with me. We all learned a valuable lesson none of us will ever forget because it is burned into our foreheads with the brand "5" as its name.

So every time I think: "eh, I'll skip the tests." I remember what it is like to have no automated testing at all. I remember what its like to not be able to quickly tell that you've just made a mistake that means your method only returns 5. I remember 8 months forged in hell, that nearly crashed that company, and I run the tests again anyway.

2012-04-13

Clojure User's Group @ Google NYC [Feb, 2011]

This post was sitting in my drafts bin for a while but I decided to finish it off and post it today. My apologies if it is a bit dated and not up on current Clojure syntax.

I was at Clojure User's Group, NYC back in 2011...

And Stuart Halloway flashed this code on the screen:

(describe String (named "last"))

He complained that while it appears simple, it is in fact not simple and showed this code instead (which does the same job).

(->>
   (reflect java.lang.String)
   :members
   (filters
        #(.startWith (str (:name %)) "last"))
   (print-table)
)

He then claimed the new code was simpler. I had to admit I just didn't see it. How is the second listing simpler? The idea is to describe in table format some data right? "describe" seems pretty simple to me.

Over beer that night Stuart illuminated my foggy understanding. The second listing is simpler because it introduces fewer new constructs to the language. It is also simpler because the components are smaller, looser in coupling, and can be re-used later without some special facility or dependency on printable output. The key to reuse being that "print-table" now does tablular output on any incoming data as opposed to hiding that nugget of function deep inside the "describe" machinery.

We went on to discuss the issues with ORM and how mapping data adds a layer of accidental complexity. Being a good disciple of the Java/Groovy camp I believe in hiding complexity behind clever facades like Domain Specific Languages, Frameworks, and API. This is in stark contrast to a good disciple of Lisp's view of the world.

What makes Clojure interesting is the deliberate collision between these two world views. I normally would not be forced to dredge up my decade old exposure to Lisp and place it directly in front of what I've learned from Java the last decade. The Clojure folks force us to do that and you can blame this momentary encounter with Stuart for my talk on Functional Programming in Java and for the thought experiment that is lambda-cache which is an attempt to do caching (via ehCache) using a lambda construct in Plain Old Java Objects.

What is interesting is as we explore Functional Programming on the JVM and as project lambda matures we are going to finally explore how best to reconcile these differing views of simple. That is because simple is anything but simple. In it's purest form simple is really an analogy for beauty when it comes to computer science or mathematics. The value of beauty is debatable. How to achieve beauty is at least in part subjective. 

What is most surprising is that there are objective measures of beauty in many fields. For example humans find adherence to the golden ratio in proportion to be beautiful. There may in fact be a measure of beauty that can be performed mechanically. Similarly, is there a measure for beauty when it comes to computing? Ostensibly simple is the opposite of complex and computational complexity theory is still a dark murky thing. Do we need to shine some light into the corners of complexity theory before we can measure simplicity?