Comparing Stats on this blog Feb & July 2008

I've been keeping stats on this blog since Febuary 1st of 2008. Here's some numbers. Don't know if they are good, bad, or whatever.

February hits: 1,503
July hits:     2,681

So I've not yet doubled my traffic in the last six months. What do those numbers over the last six months look like in total?

13,961 Visits
10,954 Absolute Unique Visitors
17,323 Pageviews
1.24 Average Pageviews
00:01:00 Time on Site
85.76% Bounce Rate
78.40% New Visits

... so most folks just bop through here and on their merry way. only about 15% are getting anything from their visit. I wonder what the average visitor's browser looks like? Top five browsers are a small surprise...

1. Firefox / Windows            49.86%  
2. Internet Explorer / Windows  20.06%  
3. Firefox / Linux              13.37%  
4. Firefox / Macintosh          6.88%  
5. Safari / Macintosh           4.61%

Firefox trumps explorer. But then consider the audience... programmers... yeah, I guess that makes sense. Interestingly 70% are Windows users, 13% are on Linux, and about 10% are on Macintosh. That's a bit surprising to see Linux rank higher than Mac.

I won't try and break down the search engine stats here but what I gather is that most visitors to this page are getting here through google or yahoo. Surprisingly, the most used key words are "grails portlet" I'm very happy to see GRAILS-254 coming along. Check out Lee Butts' documentation of his work.

Ironically, I never did follow through on a Grails Portlet because we dropped the use of Portlets in our organization. It would not have paid off for us to do portlet development until we had a much larger constellation of projects. I may have to revisit portlet work in Grails in the coming months judging from this demand.

I'm not a bit shocked that my more technical Grails articles enjoy the most readership so I plan on ending my indulgence of more abstract topics and getting down some serious Grails related business. I've just shipped a major Grails project that represents six months of work so I should be able to dig deep into that experience for a few posts.

What I'd love to hear either formally or on the side is how these numbers stack up to other blogs. Are these good numbers? Are they bad? I'm hanging them out there for everyone to see (or laugh at) because I have no delusions about being a well read blog and no reason to hide them. So I've offered first. You go next.

Why do you read this blog? Apparently about 3,000 of you come back here from time to time. How can I focus my topics to help you better? What would you like to ask me about my last year working in Grails full time? Any pointers from "professional" bloggers will be welcome.

UK Gov't and Lost Personal Data

Slashdot is running a story on UK Gov't Lost Personal Data On 4M People. I follow these stories (as previously posted here) because the management of Personally Identifiable Information is probably one of the hardest and most vital issues we have in front of the internet in the next decade.

As I have observed previously in this blog, computer programmers are beginning to build systems that substitute for subtle key social infrastructure that we (as mere programmers) have no idea that we are replacing. As computers take over the duties of identifying social standing, trust-worthiness, and other faculties that used to be purely human domains the protection of key information for each person either becomes deathly important or our ideas about how to verify identity have to be re-designed on a global scale.

Daunting, serious issues that will probably not be addressed until there are truly severe consequences to either the financial system or civil liberties. These are the core issues I set out to explore in this blog originally. In the simplest terms, the software engineering and the software design of the world's information systems need to grow up. And fast.

Unfortunately, the concept of "data guardian" feels like a hack and not a good one. I think we need to design information systems that obviate the need to keep PII secret at all... or systems that eliminate the need for the transmission of PII. Such systems would probably have to operate in radically different ways on the human level.

A fundamental property of information is that it is easily replicated the idea that we must prevent the spread of a specific class of information is like fighting the tide. Instead redefine identity and authority. If you must have a "secret ID key" (aka Social Security Number in the US) allow it to be easily changed.

Until society adapts to these ideas we all must carefully protect our personal information and those of us that write software to deal with credit, visas, or medical data must be vigilant beyond the immediate call of duty. Your web application might lead to someone's identity being stolen. Please think about that.

Anti-Management

Bruce Webster is fast becoming one of my favorite IT-management authors right now. If you haven't already,1 check out his article The Dead Sea Effect which summarizes why some companies have high turn-over and only retain the worst employees. It's a good read and the main ideas in that article should at least inform your decision about what job offers to accept.

I also listen a lot to Jared Richardson and his advice which is usually about positive examples of how to lead teams and set up software shops. Jared's writings tend to be the exact opposite of Bruce's. Bruce focuses on what people do wrong and Jared tends to focus on what people do right.

I've taken Bruce Webster's writings along with things I've learned from Jared's talks and thought up these anti-examples of management. I call them anti-management tips... some are from Bruce's observations, others are Jared's flipped in reverse.

anti-advice for managers, CEOs, and leaders everywhere:

1. Don't fire anyone, ever. The best way to demoralize troops is to avoid ever taking action against the incompetent, destructive, or otherwise evil employees you have. In fact, you should reward people with negative attitudes, poor work ethic, or destructive personalities by giving them the same or better salaries than the productive and positive people in the company. This way you show people that it doesn't matter if they do well or work hard and you will not remove negative influences or stand up for them. Not firing or ever taking action to remove harmful employees at all shows that you don't care to protect your company, the jobs of your employees, or create a non-hostile work environment.
   


2. Fire people randomly. The less relevant to performance you can make the firing or layoff of employees the better you will destroy their work ethic. Just up and fire people for reasons that nobody can fathom. It will help people feel that your workplace is unfair and that their loyalty is misplaced. If you do this enough you can start your very own "Dead Sea" as competent people leave for new jobs to avoid the chopping block. If you have a fair process that tells an employee if they are doing well or headed for a pink-slip then you might accidentally give people the idea that your work place is fair. Don't do that. Be unfair.
   


3. Never promote from within. If you never promote your people in-house you are telling them that they will never go anywhere and you see no potential in them. In other words, if you work here you must be a chump. If you want to advance in your career... this company is just a stepping-stone!
   


4. Only promote from within. Promote incompetent people, hostile people, people who stuck around long enough. Don't promote for skill. That would show fairness. Never hire outside the good-old-boy network showing that it only matters if you've been with the company a long time. That helps people to either seek new opportunities or hunker-down and wait for the rising tide of high turn-over to eventually lift their ship up the corporate ladder. Whatever you do don't make promotions based on skills, ability, training, or any other metric that might seem fair.
   


5. Never train your people. If your people have no real potential why bother training them. They might get better professionally and leave your company! Don't do that. You need to keep people from becoming talented enough to get other job opportunities. Only promote people who show a dis-interest in self-improvement to help underline this idea.
   


6. Switch skill requirements and retrain constantly. Make people feel like the future is uncertain. Make them feel like any skill they learn will immediately become worthless.
   


7. Never adjust plans. Don't change plans just because reality changed. This way you can further validate that you don't really care to protect your company, profits, and by extension the jobs of your employees. You further promote the image that you are detached, disinterested, and only drawing a paycheck yourself.
   


8. Change plans constantly. Paradoxically changing plans too frequently gives the impression that you have no idea what the hell you are doing, have no long range vision, and are not competent to protect the employment of your staff. Finding the balance requires skill and thoughtfulness and you never want to display those otherwise people might start having hope for the future.
   


9. Never sweat the details. Showing total disinterest in the details helps to foster that sense of detachment and disinterest in protecting the employment of your people. A well functioning company that is profitable and rewards its best employees is essentially ensuring continued profits for share holders and continued employment for the majority of its employees. Sometimes the details mean the difference between profit and loss. Never showing interest in the details shows not only a disinterest in the daily work lives of your employees as people but also a negligent disinterest in the affairs of your company. Smart employees will spot this and be effectively demoralized by it. So, go for two rounds of golf this afternoon.
   


10. Last minute Micromanage. Sweat all the details, preferably only about three weeks before the due date of any project. Swoop in at the last minute and add more managers to a project and start micromanaging details and changing things right up to the deadline. Doing this right necessitates that you create project plans that are multi-year or multi-month and then ignore all status updates for as long as possible. Micromanaging this way helps people who work under you not only feel you don't trust them but you are also incompetent and may try and hide that by blaming them. Micromanaging is bad but most people either learn how to live with it or find a balance with their manager if it is consistent... don't give people the opportunity to think they understand what is expected of them. Inconsistency is how you keep people guessing and uncertain about their futures.
    


11. Solicit feedback, then ignore it. Let people know you are just toying with them and don't take them seriously. Give them a party afterwards to rub it in.


12. Celebrate mediocrity. Celebrate and reward meaningless accomplishments, never acknowledge super-human effort. This will create the impression that you either don't pay attention or that you find above and beyond performance meaningless. Conversely, we celebrate a baby's ability to walk but we don't celebrate a teenager's ability to do the same. Loudly congratulating a baby for taking a step gets a giggle from the baby, loudly congratulating a teenager for taking a step gets nothing but disdain from the teenager. Your employees at different levels of ability operate the same way. Of course, you could also throw a party for the person who didn't screw up this time consider comments in front of the whole group like: Hurray! You didn't totally screw up! We're so proud of you. Who's acting like a competent developer? You are! Yes, you are! Wooshee wooshee woo!
    


13. Only listen to consultants. You need to let people know you think they are incompetent.


14. Never hire expert consultants to help with projects. You need to let people know you will never get them the help they need. Let them know you'll never train them, get them what they need to succeed, or actually care about what they are burning their precious life's flame out on.


15. Never cancel consultant's contract. Let people know you can spend lots of money, just not on them.


16. Never get to know your employees. Keep them at arms length, don't get to know them, tell jokes to other managers in front of them without acknowledging their existence. This keeps the employee feeling distant and detached. You want to let them know you think they are a cog.
    


17. Never tell your employees how the company is doing. Keep them in the dark and avoid telling them what's really going on in the company. That way they learn to use the grape-vine as the ultimate source of information. Your company will run on rumor and innuendo.
    


18. Broadcast everything negative. Anytime something the slightest bit wrong happens run around like chicken-little screaming that the sky is falling. That'll help morale.


19. Contradict subject matter experts. You should also contradict and constantly undermine your subject matter experts to show them that their expertise is worthless. If you contradict the best minds in the industry all the time you'll help people feel you are clueless. This will help heat up your little Dead Sea project.
    


20. Abuse your employees. If you can't physically beat up your employees make sure you set up situations where you can emotionally beat them up. Set up processes with lots of negative interactions with people telling each other "no" a lot. Don't use technology to alleviate the need for negative interactions between people. Make sure everyone is forced to confront their coworkers frequently for the smallest infractions.
    
    Whatever you do... don't create or install any metrics, automated tests, dashboards, continuous integration servers, bug tracking systems, or anything that could impartially handle the tasks of negative feedback that help create a perception of fairness and predictability. That kind of action would positively affect morale. Instead, keep work-life nebulous, unpredictable, and as unfair as possible. Make people battle each other based on emotion, gut feeling, and innuendo forcing politics to rule decisions instead of cold hard facts.
    

Comedy aside, I think building a talented software development team is a lot like building a talented basketball team... or any sports team really. You may need talented developers that are really star players, but, you may have to play without them in which case having a good strategy is key.

Frameworks, delicious frameworks...

This blog has been on "auto pilot" for the last few weeks while I've been on vacation. As my reserve of scheduled articles has dried up I guess it's back to work. I'll be at the trijug meeting tonight. We'll be hearing from Hadrian Zbarcea about Apache Camel which is yet another framework on top of Spring.

As I've noted previously in this blog, the J2EE space is being very effectively invaded by Spring. So effectively so that even J2EE Application Server provider JBoss provides Embeddable EJB3 which is a sub-set (as of this writing) of the full EJB3 environment that can deploy on "light weight" containers such as tomcat.

If that last paragraph made no sense, try thinking of it this way... JBoss, WebSphere, and other Application Servers provide a host of services that are considered part of an "Enterprise" stack. These services sit underneath specific applications. These are things that are conceptually low level like messaging between processes/beans, facilities for looking up shared resources, data persistence and storage, and security. Things like HTTP, HTTPS, and such are considered additional services and are provided by applications. A typical J2EE application looks up the additional resources it needs as it runs. So when your bean initializes it has to do a lot of leg-work looking things up and initializing resources.

The EJB3 application standard is in part an Inversion of Control framework. In some situations using Java 5 annotations EJB3 beans specify what resources they need and the framework wires up those resources so that they are available. Most notably the Entity Manager is usually wired up using annotations and not using JNDI lookups. It's so radically different that we call the new paradigm JEE 5 in some circles... not only skipping the numbers 3 and 4 (which are ugly looking anyhow) and jumping right to (the sexy looking) number 5... but also moving the number to the end of the acronym so we can spot the noobs.

Spring works on this Inversion of Control (IoC) design principle and this is a key feature of Spring that Grails makes heavy use of. The design inverts the J2EE sensibility of having a bean init and then look up everything it needs. It puts the control of a bean's initialization in the hands of the framework. Working this way actually means that the majority of logic for how a application server works moves out of the application server itself and out of the application and into the framework and its configuration files.

That means you can run complex applications in simpler "containers" like tomcat and jetty. That makes server administration simpler for shops running smaller and simpler applications. Shops with larger and more complex applications can migrate their Spring applications to heavier more robust clustering Application Servers. So you have simple to scalable in these lighter frameworks.

But, without the conventions found in Grails, IoC and Spring applications become "XML-er-ific" and choc-full of XML-ly goodness. You end up having to specify nearly all the same lookups and wirings you were having to do under J2EE anyhow. You just move the work from compiled Java to light-weight XML. It's better but you don't really escape the work.

This configuration issue is where Grails shines. Grails removes the need for most configuration work by introducing conventions. Because Spring is still under Grails you can over-ride or change the conventions if you want or you can abide by them and get the full benefit of Grails. So once again Groovy and Grails provide you with the choice of staying as shallow or diving as deep as you need. A great benefit that is missing in most development frameworks which either force you to become an expert of force you to keep the training wheels on forever.

I'm curious to learn about Apache Camel to see how they solved these issues. I'm also curious to see if JBoss will bring along their embeddable EJB3 to compete in this space. Obviously Groovy/Grails is my personal favorite but I'm always up for learning from others.

Software waste

Consider the plight of Microsoft and their rampant success that has also sown the seeds of their current woes. Also consider the plight of the BBC and their strange web platforms. And, I think we can start to see the inside of a very real recurring problem that date back at least to Fred Brooks Jr. and probably point back to problems that have plagued all engineering projects that require much management over-head.

Computers do a great job of covering up wasted labor. That's because they can crunch through stupidity like nobody's business. You have to wonder if management could actually see the product being built if they would take such a stance (both at Microsoft and at BBC).

It turns out that being insular in your development practices means you get to be "the old man on the mountain" and your shop turns into something quite unique... most of the time in a very bad way. The ideas of DRY, open source, and code reuse are ideas that we hold onto in the great wide community of developers to keep us from wandering down dead-end trails. The results of which are utter disasters unless you are Microsoft or the BBC which are both big enough to just absorb the loss.

My message to you is to get out there and get some of your sacred cows challenged. Learn how to defend your choices and your technical abilities or learn to change them. Do some open-source work. Stop making stupid rules to limit the creative talent you employ. Get out there and learn from others.

Software is a system of ideas

Take a system of ideas, turn it into code, let people interact with it. Software that works is software where everyone's ideas match and are consistent with a reality that can be represented in the computer. If your ideas don't match what is in the computer or in the heads of others then there is a bug.

Technical Debt and Technical Credit

The Martin Fowler entry titled Technical Debt has probably been the single most profound article I've read about software design in the last five years. It honestly describes why quick and dirty is a necessary evil and why shops need to build in development cycles to essentially "fix" working software. The idea of "paying down" technical debt is completely lost on non-programmers because they can't see the result directly. Instead what they see is the result of not paying technical debt.

I've had the misfortune of having this conversation:

Client: How long will this take?

Me: We've estimated that it will take about three weeks pushing just about everything else out of this development cycle.

Client: That's crazy! At my last job they could do the same thing in three days, that task should be trivial! Why will it take you so long?

Me: The legacy system design which we have to account for will never allow those changes without a significant amount of work. I'm sorry. Your other shop must have spent time upfront in preparation for such changes so that you didn't see the cost when you asked for them.


Sadly, the client can hardly help feeling that I'm incompetent. I've been set up for failure by a history of delivering too rapidly. It's something Bruce Webster calls the Dangers of a Successful IT project and I've had the problem in spades.

Obviously, I am working in a different legacy environment than the client's other shop was. To her she only sees working with me and her browser, to me I see working with her, the browser, and the old crufty code she just asked to have changed that has SQL in-lined in JSP and PHP, no CSS, a denormalized database, and cut-and-paste based templates (no templates at all). So changing somethings turn into a repetitive batch-job of a task and require full regression testing to make sure we didn't forget a page.

I'm working in a Technical Debt scenario and I've just been asked to do something that requires I pay back a lot of the technical debt in the system. In the scenario I'm thinking of the company had incurred over eight years of technical debt it had never bothered to try and pay back. The interest payments were huge by the time my "simple change" was asked for.

But, I've observed that are situations of both Technical Debt and Technical Credit. A Technical Credit is a credit that you get for working with tools that are ready made for agility. The Grails Framework for example is perfectly positioned to give you a major Technical Credit. Ready made plugins such as the Searchable plugin and the JFreeChart plugin are examples of quick to get and use tools that will make you seem like a technical guru even if you aren't.

This all applies to clients at the same company as you or clients at different companies. Anyone who is non-technical and has a stake in the project needs to be seen as a client of your work. And you, dear developer, need to cultivate a relationship of trust with them.

I think the analogies of Technical Credit and Technical Debt together can make strong cases for doing the unglamorous work that needs to be done... or adopting things like Open Source tools in the Enterprise. The developers on a project need to remember to sell this idea and tie-in the incidents of debt making and credit making to project time-lines to management. That is a difficult but important task that will build trust between you and your clients.

Maintenance Software Architect?

If you can accept the idea of a Software Architect the idea of a maintenance architect makes sense. However, in the circles I travel in there is actually some resistance to the idea of a software architect at all.

This resistance is usually driven by a misrepresentation of what the role should be. An architect role should be a mentor and guiding hand in the business' technology related initiatives. The architect should not be about the business of "bossing around" developers.

The Maintenance Architect role makes perfect sense if you look at what the job would be involved in doing. Even in the absence of a Software Architect a Maintenance Architect could still make sense. The Maintenance Architect would basically herd all the "cats" (developers) in a direction. The M.A. would steer a Perl shop (for example) toward exposing their CGI as distinct REST services that could then be fronted with Flex for example. In time you could whittle away at the CGI with Grails applications or another technology that fit your new technical stack. Eventually after a great number of iterations you would be transitioned to the new technology. You would "cook the lobster" if you will... slowly transitioning the shop to one of the 21st century software stacks.

It would be a long sighted goal and it would take a long sighted person to pull that off. The guy you'd look for would be someone with patience, endurance, and commitment. They would have to be a stern guiding hand yet be flexible enough to wait out the many development cycles to see their plan fulfilled. Not an easy order to fill and not a common person to find.

Proposed at BarCampRDU: iBatis plugin for Grails

One of the stranger ideas that came out of BarCampRDU this year was that we might do an iBatis plugin for Grails. Such that GORM could then run on-top of iBatis or perhaps you would tweak iBatis to run a faux-GORM in Grails context.

That seems like an exceptionally hard plugin to write. I think I might need some convincing before I even took a serious look at something like that. It is at least technically possible since GORM itself is a plugin. But, that doesn't mean there isn't an iBatis guru out there that would love to learn how to do iBatis + Grails for the folks that needed that.

Anybody out there interested?

BarCamp RDU 2008

This year's BarCampRDU was fantastic. Presentations given this year covered a really diverse set of topics. Frankly, too many topics to cover in this post. Ironically, I forgot to vote...

Talks on Social Networking? Yeah, but that is so last year. Instead I saw talks on iPhone programming and the Fan Programming Language, Cloud Computing using the Amazon EC2, Big Lee himself was there getting a talk going on social controls over Spam, and there were even topics such as Hadoop and Mahout (talk led by Grant Ingersoll no less). Not to mention a crowd of serious people working on incubating technology startups in the RTP area. And yes, a group discussing Groovy and Grails was there and yours truly did more than lurk.

The interesting thing about Grails is how broad an audience it targets. At the moment the core G2One guys (I've only met Jeff Brown) are focused on Java shops. This is probably a good "head of the spear" tactic for them. (See I was paying attention in the startup talk) as the integration story between Java and Groovy and Spring and Grails just totally fantastic. There are only so many hours in the day and only so many people you can talk to. So the choice to focus was probably wise. I think there are some great advocates and allies for Grails waiting out there in frustrated LAMP stack developers who are ready to move to the next level.

The fact that you can dive as deep into Grails as you want to or stay as shallow as you want is so powerful. By leveraging Groovy which by its very construction means I can dig deep into Java if I need to means for small projects that scale out they can be incrementally and iteratively backed off down to Java applications if you need to. Or, you can stay fast an loose on top of the Grails platform cranking out features like there's no tomorrow. And there are many LAMP guys who are going to love that. I think I've met a few smart guys that might actually find a few great ways to help Grails grow into this region.

During the between presentations times I met folks who work at exciting technology companies like rPath, Lulu, Tibco, and through them I've heard about the small Google office here in RTP working on Android related software. And, since I've started to get out and about I know that Sun also has a small group of engineers here in RTP as well as Oracle and the obvious big guns like RedHat, Cisco, Nortel, and IBM.

I think that the RTP area (because of the insane growth seen in Chapel Hill, Durham, Cary, and Raleigh) is just inevitably the next great place to be for new young companies seeking to innovate, invent, and create. If there is a recession it's not here, if any thing I'm seeing a boom town in my backyard. Why aren't there more high profile startups and more technology companies here? I think it's just a perception thing.

I can't mention everyone and everything in this post ... in a scant few hours I got as much out of BarCampRDU as I have some week long conferences ... I'm sure that many of the connections I made at BarCampRDU will help to inform future posts and might even change the course and focus of my career. Yes, it was that good. If you didn't go... you missed out.