It's Programmable... It's configurable...

How much difference is there between a program and a configuration?

Consider that a program is a set of instructions for a universal Turing machine. If a configuration determines how a program does something... if the configuration becomes sophisticated enough so that it even includes a DSL when does it cross the line from merely a configuration into a programming language.

When does a program become configuration? When does a configuration become a program? What is the difference?

Technical Debt and Technical Credit

The Martin Fowler entry titled Technical Debt has probably been the single most profound article I've read about software design in the last five years. It honestly describes why quick and dirty is a necessary evil and why shops need to build in development cycles to essentially "fix" working software. The idea of "paying down" technical debt is completely lost on non-programmers because they can't see the result directly. Instead what they see is the result of not paying technical debt.

I've had the misfortune of having this conversation:

Client: How long will this take?

Me: We've estimated that it will take about three weeks pushing just about everything else out of this development cycle.

Client: That's crazy! At my last job they could do the same thing in three days, that task should be trivial! Why will it take you so long?

Me: The legacy system design which we have to account for will never allow those changes without a significant amount of work. I'm sorry. Your other shop must have spent time upfront in preparation for such changes so that you didn't see the cost when you asked for them.


Sadly, the client can hardly help feeling that I'm incompetent. I've been set up for failure by a history of delivering too rapidly. It's something Bruce Webster calls the Dangers of a Successful IT project and I've had the problem in spades.

Obviously, I am working in a different legacy environment than the client's other shop was. To her she only sees working with me and her browser, to me I see working with her, the browser, and the old crufty code she just asked to have changed that has SQL in-lined in JSP and PHP, no CSS, a denormalized database, and cut-and-paste based templates (no templates at all). So changing somethings turn into a repetitive batch-job of a task and require full regression testing to make sure we didn't forget a page.

I'm working in a Technical Debt scenario and I've just been asked to do something that requires I pay back a lot of the technical debt in the system. In the scenario I'm thinking of the company had incurred over eight years of technical debt it had never bothered to try and pay back. The interest payments were huge by the time my "simple change" was asked for.

But, I've observed that are situations of both Technical Debt and Technical Credit. A Technical Credit is a credit that you get for working with tools that are ready made for agility. The Grails Framework for example is perfectly positioned to give you a major Technical Credit. Ready made plugins such as the Searchable plugin and the JFreeChart plugin are examples of quick to get and use tools that will make you seem like a technical guru even if you aren't.

This all applies to clients at the same company as you or clients at different companies. Anyone who is non-technical and has a stake in the project needs to be seen as a client of your work. And you, dear developer, need to cultivate a relationship of trust with them.

I think the analogies of Technical Credit and Technical Debt together can make strong cases for doing the unglamorous work that needs to be done... or adopting things like Open Source tools in the Enterprise. The developers on a project need to remember to sell this idea and tie-in the incidents of debt making and credit making to project time-lines to management. That is a difficult but important task that will build trust between you and your clients.

Language is a Cognitive Technology

Making the rounds is a story out of the MIT news office about the Piraha tribe whose language does not include any counting numbers. They do have the concepts of more than and greater than and the concepts of few and many. Counting isn't a useful skill for them.

What is most intriguing to me is not the missing counting concepts but this odd quote from the end of the article:

One other discovery of the project is that the Piraha can perform exact matching tasks as long as there is no memory component to them, but once there is a memory component, they approximate their matches. This suggests that language is a cognitive technology that aids humans in memory tasks.

That is an interesting phrase to me: language is a cognitive technology.

Programming languages represent cognitive tools to work with a computer. That is definitely true. If you can consider language itself a technology of the mind or a cognitive tool that is an intriguing idea. Is your ability to only use one language or one programming language akin to only being able to use a screw driver?

If we can consider language itself a technology then we have an interesting new time-line for the progress of technology. We can go back to prehistory tracing the advent of the basic concepts of mathematics building up to today's computers which ostensibly hold the fruit of tens of thousands of years of cognitive technological effort.

And this begs me to question... what if we had started on the wrong path all those millenia ago. What if there are better cognitive platforms to build our systems of thought on top of but we couldn't explore them because our tools of cognition effectively created a platform of thought that prevented us exploring other methods of thinking. There could be ways of thinking and computing we can't even look directly at because they are so alien to us.

This idea cuts both ways. It tells me that if a paradigm is too far removed from mainstream thought it could die because not enough minds can (or will) adopt it. That means there are undoubtedly modern linguistic paradigms that are limiting but also unavoidably entrenched. This is good and bad. One language helps people to be mobile even as it limits them.

Consider how English is changing as speakers in many countries begin to change English to match the paradigms of the local tongues it mingles with. The resultant language may become a world tongue called Panglish in some breathless articles. It's all speculation right now.

But, we see this all the time in Programming languages. We see programmers transplanted from one language into another that bring with them the idioms of the languages they left behind. When this becomes laughable is when what is changing is far more than syntax but entire ways of thinking through problems that are being butchered by forcing the new language to work like the old one.

Here is a counter intuitive idea, but, some limitations are actually liberating. When children play in an open field they will cluster together. If you fence in an area larger than what they are actually occupying they will spread out to fill the space. The same is true about how you think about things.

Too broad a programming framework means that you can't effectively address any specific problem. It means you have too wide open a field. Instead the more a framework leads you to an answer the faster you will move toward a goal. Too efficient and limiting of a framework and you can't steer toward an appropriate design and must instead work around your framework.

Think of it as a volume of space where each point represents a system design. Your framework is like a channel or road driving you toward a region of all the possible system designs you can imagine. There is a volume inside that huge space of all possible systems that contains designs that fulfill your requirements (stated and otherwise) and your framework drives you in the direction of a handful of these. If your framework is too strict there could be a problem where the volume of answers lie in one direction and your framework drives you in another.

That would be an impedance mismatch between the problem and the framework. You would need to modify the problem or the framework to bring them into concert. Or you would have to abandon your framework entirely.

Most of us today are working in a world where our frameworks are pulling us toward solutions that are wholly web-based. That is just a fact of what is in fashion today. It isn't wrong either. The web will finally disassociate the cognitive and the physical. Just as MP3 is slowly killing the CD so too will the web slowly kill off certain legacy computing paradigms.

What do the Piraha know and can think about that we can't? Perhaps they have some insights of genius that are not evident to us who are encumbered with numbered thoughts? I wonder how many of us are like the Piraha happy in our own world but are fast colliding with a world that is driven by new thoughts and ideas utterly alien to us. As alien as numbers are to the Piraha the concepts behind cloud computing and massively parallel computing systems are headed our way. What will the programming languages that help us get there look like? How far off course have we drifted along in the channels of thought carved by our programming infrastructure?

It's just a thought.

Hide complex behind simple...

Hide the complex behind the simple. Keep the complexity of a system behind the simplest possible interface. At every level find the simplest way to express the intention of the interface, do that, and get out of the way. In short, make things easy for your user and hard on you. It's your job.

If you are creating API (as I sometimes find myself doing) work to create the interfaces that the users of your API will find intuitive and simple to use. Hide the complexity of this work as succinctly as possible behind the inner implementation of the library.

If you are working on UI (as I also paradoxically find myself doing) work to create an interface that encourages play. I don't mean gaming. I mean, allow a user to "play with" the interface and explore it. Let them discover what it can do and try and make it intuitive to the way that they think about problems.

In the desing of both UI and API you can find yourself altering what "simple" is. You can introduce new ways of thinking about a problem by creating new implicit behaviors. This is an interesting space to work in collaboratively because you can find that users (both programmers and lay-users) have developed preconceptions and notions of how a system "must work" and so doing box themselves in.

In my experience of hiding complex things under simple interfaces I have found that there really is nothing that is truly simple in the whole. Instead things are simple in isolation but produce emergent complexity when placed in large systems. So to fight this the best weapon is to keep systems small. By keeping systems small and isolating their inputs and outputs you have a better chance of artfully controlling their interactions.

The other side of this is that our minds are horrible at containing inherent complexity. I used to think I was very good at this. I no longer think that. I now realize that I might be better at dealing with complexity than some, but my abilities have a distinct upper limit that is reached very quickly.

That is why when we build systems of any size. We developers of systems of software should acknowledge the need to test the system to validate our understanding at every level. Just as we use the scientific method to validate what we believe to be true using experimentation we should validate the way thoughts interact via experimentation.

The irony that what we are testing is in my opinion codified thought is not lost on me. Consider Kurt Gödel and his incompleteness theorems which point to the limitations of all formal systems... they are doomed to incompleteness. That is to say that you can never through proof alone (code alone) create a system to model such complex yet simple things as the counting numbers.

So there is a bug in the universe?

When you or I create software in any capacity we are indeed creating a formal system. A system that will either fail to model the real world problem we are attempting to emulate or will be fraught with inconsistency and incompleteness. So even if we were super-coders that never produced bugs, and you and I could in fact produce a perfect system, that system could be fraught with problems due to its inability to properly model or express the real world system it was based on.

The simplest answer is to devise tests to prove the correctness of our systems.

Symbol System Manipulation

If thought is just the manipulation of symbols. Then computers think. If thought is more than the manipulation of symbols... then how do you know people think?

If you can't accept that the mechanical manipulations of symbol systems carried out by a computer are thought... then how can you accept that the mechanical symbol manipulations carried out by organic beings are thought?

Artificial Intelligence beats real stupidity every time.

Waxing a tad philosophical, perhaps the mechanical symbol manipulations of any system be it a human brain or a computer are the echo of thought. In the case of the computer the program is the echo of a thought left by its programmer. In the case of a brain these are the echoes of thoughts begun in the minds of others and advanced or reproduced yet again.

The idea is that thought-stuff can move between noggins and that the computer offers a similar place of purchase for the thought-stuff as a rather simple minded noggin. So thought-stuff lives in noggins. Thought-stuff can be preserved in medium like a seed of an annual plant it lays dormant on a page. Thought-stuff gets planted in a noggin and sprouts.

Living thought stuff that bears fruit as action and turns to seed is shot across the divide between noggins in speech, on paper, or as code. The result is germinated. In the case of code the germination can occur in the artificial soil of a computer.