Clojure User's Group @ Google NYC [Feb, 2011]

This post was sitting in my drafts bin for a while but I decided to finish it off and post it today. My apologies if it is a bit dated and not up on current Clojure syntax.

I was at Clojure User's Group, NYC back in 2011...

And Stuart Halloway flashed this code on the screen:

(describe String (named "last"))

He complained that while it appears simple, it is in fact not simple and showed this code instead (which does the same job).

(->>
   (reflect java.lang.String)
   :members
   (filters
        #(.startWith (str (:name %)) "last"))
   (print-table)
)

He then claimed the new code was simpler. I had to admit I just didn't see it. How is the second listing simpler? The idea is to describe in table format some data right? "describe" seems pretty simple to me.

Over beer that night Stuart illuminated my foggy understanding. The second listing is simpler because it introduces fewer new constructs to the language. It is also simpler because the components are smaller, looser in coupling, and can be re-used later without some special facility or dependency on printable output. The key to reuse being that "print-table" now does tablular output on any incoming data as opposed to hiding that nugget of function deep inside the "describe" machinery.

We went on to discuss the issues with ORM and how mapping data adds a layer of accidental complexity. Being a good disciple of the Java/Groovy camp I believe in hiding complexity behind clever facades like Domain Specific Languages, Frameworks, and API. This is in stark contrast to a good disciple of Lisp's view of the world.

What makes Clojure interesting is the deliberate collision between these two world views. I normally would not be forced to dredge up my decade old exposure to Lisp and place it directly in front of what I've learned from Java the last decade. The Clojure folks force us to do that and you can blame this momentary encounter with Stuart for my talk on Functional Programming in Java and for the thought experiment that is lambda-cache which is an attempt to do caching (via ehCache) using a lambda construct in Plain Old Java Objects.

What is interesting is as we explore Functional Programming on the JVM and as project lambda matures we are going to finally explore how best to reconcile these differing views of simple. That is because simple is anything but simple. In it's purest form simple is really an analogy for beauty when it comes to computer science or mathematics. The value of beauty is debatable. How to achieve beauty is at least in part subjective. 

What is most surprising is that there are objective measures of beauty in many fields. For example humans find adherence to the golden ratio in proportion to be beautiful. There may in fact be a measure of beauty that can be performed mechanically. Similarly, is there a measure for beauty when it comes to computing? Ostensibly simple is the opposite of complex and computational complexity theory is still a dark murky thing. Do we need to shine some light into the corners of complexity theory before we can measure simplicity?

A Programmer's Isogloss

One of my coworkers has a degree in linguistics and this idea comes from talking with her about language and isolation. Call it my you should really go to that conference rant if you like. The topic of conversation give me a stark realization that I think anyone that is responsible for the care an feeding of programmers should be cautious of.

Naturally, computer programmers use programming languages and the more complex the language and free form it is the more likely it is to have dialect, idiom, and isogloss just like natural languages.

In particular if a speaker or group of speakers live in isolation long enough they will develop their own little quirky habits. These quirks start off as innocent inside jokes which include conventions that we recognize in API. Given time the isolated group can start to invent new idioms which approach an unintelligible complexity to an outsider. The divisions between various frameworks that share Java as a base language and their definitions of terms is a good example.

So how do you break this cycle of main languages fragmenting and breaking up into new languages? You go out and regularly talk to others about your code, your architecture, and you get a feel for how others say words or convey meanings. You make your community of language larger. Every company needs to get its programmers out there and talking to other developers. One of the best ways is through conferences.

The benefits will be that your corporate architecture will stay closer to the wider world's interpretation of the language and idioms you use (think: style, practice, and API). Your developer's will be more creative and valuable as they benefit from the exchange of ideas. And you will know that you have a pool of similarly talented consultants and developers to seek out when you need them.

Other good ways to let your programmers get out there and learn more is to encourage the use of social networking in responsible and applicable ways. I used this recently to help think about const versus final in our code base.

Another idea I've been toying with is to have at least one small company backed Open Source project that your people are encouraged to contribute to that you can also use for getting outside feedback and perhaps to recruit new people into your company from. This is a novel idea but I think with some sensible boundaries it can pay off even for closed source companies.

Those are just some of my thoughts on the topic. But essentially you want to avoid putting your programmers in the position that all they ever do is "talk" (via programming) to the computer and themselves. You need to talk with other people whether in code or English to keep your skills sharp and to avoid drifting off into your own strange dialect. The dangers to both the company and the employee of drifting off into your own little world are profound. It can mean obsolescence for both parties and that doesn't help anyone ... except maybe your competitors.

Language is a Cognitive Technology

Making the rounds is a story out of the MIT news office about the Piraha tribe whose language does not include any counting numbers. They do have the concepts of more than and greater than and the concepts of few and many. Counting isn't a useful skill for them.

What is most intriguing to me is not the missing counting concepts but this odd quote from the end of the article:

One other discovery of the project is that the Piraha can perform exact matching tasks as long as there is no memory component to them, but once there is a memory component, they approximate their matches. This suggests that language is a cognitive technology that aids humans in memory tasks.

That is an interesting phrase to me: language is a cognitive technology.

Programming languages represent cognitive tools to work with a computer. That is definitely true. If you can consider language itself a technology of the mind or a cognitive tool that is an intriguing idea. Is your ability to only use one language or one programming language akin to only being able to use a screw driver?

If we can consider language itself a technology then we have an interesting new time-line for the progress of technology. We can go back to prehistory tracing the advent of the basic concepts of mathematics building up to today's computers which ostensibly hold the fruit of tens of thousands of years of cognitive technological effort.

And this begs me to question... what if we had started on the wrong path all those millenia ago. What if there are better cognitive platforms to build our systems of thought on top of but we couldn't explore them because our tools of cognition effectively created a platform of thought that prevented us exploring other methods of thinking. There could be ways of thinking and computing we can't even look directly at because they are so alien to us.

This idea cuts both ways. It tells me that if a paradigm is too far removed from mainstream thought it could die because not enough minds can (or will) adopt it. That means there are undoubtedly modern linguistic paradigms that are limiting but also unavoidably entrenched. This is good and bad. One language helps people to be mobile even as it limits them.

Consider how English is changing as speakers in many countries begin to change English to match the paradigms of the local tongues it mingles with. The resultant language may become a world tongue called Panglish in some breathless articles. It's all speculation right now.

But, we see this all the time in Programming languages. We see programmers transplanted from one language into another that bring with them the idioms of the languages they left behind. When this becomes laughable is when what is changing is far more than syntax but entire ways of thinking through problems that are being butchered by forcing the new language to work like the old one.

Here is a counter intuitive idea, but, some limitations are actually liberating. When children play in an open field they will cluster together. If you fence in an area larger than what they are actually occupying they will spread out to fill the space. The same is true about how you think about things.

Too broad a programming framework means that you can't effectively address any specific problem. It means you have too wide open a field. Instead the more a framework leads you to an answer the faster you will move toward a goal. Too efficient and limiting of a framework and you can't steer toward an appropriate design and must instead work around your framework.

Think of it as a volume of space where each point represents a system design. Your framework is like a channel or road driving you toward a region of all the possible system designs you can imagine. There is a volume inside that huge space of all possible systems that contains designs that fulfill your requirements (stated and otherwise) and your framework drives you in the direction of a handful of these. If your framework is too strict there could be a problem where the volume of answers lie in one direction and your framework drives you in another.

That would be an impedance mismatch between the problem and the framework. You would need to modify the problem or the framework to bring them into concert. Or you would have to abandon your framework entirely.

Most of us today are working in a world where our frameworks are pulling us toward solutions that are wholly web-based. That is just a fact of what is in fashion today. It isn't wrong either. The web will finally disassociate the cognitive and the physical. Just as MP3 is slowly killing the CD so too will the web slowly kill off certain legacy computing paradigms.

What do the Piraha know and can think about that we can't? Perhaps they have some insights of genius that are not evident to us who are encumbered with numbered thoughts? I wonder how many of us are like the Piraha happy in our own world but are fast colliding with a world that is driven by new thoughts and ideas utterly alien to us. As alien as numbers are to the Piraha the concepts behind cloud computing and massively parallel computing systems are headed our way. What will the programming languages that help us get there look like? How far off course have we drifted along in the channels of thought carved by our programming infrastructure?

It's just a thought.

By population...

I just happened to run across this statistic that makes me wonder what kinds of effects the additional one billion computers expected in the next five years will have if they are mostly owned by Asian users. Programming languages today are structured around western languages. In particular English. If you are not a native English speaker I wouldn't be surprised if some of the prevalent programming constructs aren't alien because they are tailored to an English speaking mind.

Software Metaphor: Chop Sticks versus Formal Cutlery

Consider the simplicity of chopsticks versus the complexity and elegance of formal cutlery as demanded by a formal table setting.

In the past cutlery was far more complex yielding exceptionally specialized tools for every dining situation. The complexity of formal dining utensils is just plain daunting to the neophyte. A different fork for every function. A different spoon for every food. A knife for butter, a knife for bread, a knife for fish and on goes the list.

Now, consider the chopsticks. Every food imaginable can be eaten with the chopsticks. Need to cut? Scissor your chopsticks to slice. Need to stab your food to get a hold of it? Stab the item of food with a single stick. Need to scoop food? Form a wedge with your sticks and scoop away. There is no need to learn a proliferation of tools, just one tool. Use this one tool well and all of life's dining problems are solved. Or, just avoid the problem entirely and lift the soup bowl to your mouth... have the chef slice the food smaller... if all else fails... don't eat that.

In Software Engineering we face the battle between chop sticks and cutlery every time we start a new project. Should I use a specialized technology? Should I use the chop sticks I know so well? The happiest Software Engineers learn to use chop sticks, and when problems get sufficiently messy, the learn to put down the sticks and pick up the right utensil. They keep the utensils down to knife, fork, and spoon if possible to avoid the over-head of having too many tools.

Don't fall into the "hammer" problem where in you see every thing as a nail needing to be hammered, yet keep away from the Victorian table setting if at all possible.