2008-01-27

Paradox/Parabox of Choice

A late night post due to a late night. Some things are bothering me and I decided to put them in words.

Consider this post on slashdot this is problem is real and a real burden to many shops. If you were a LAMP shop and that 'P' was Perl and then you add PHP because marketing wants something... and then you add RoR because somebody thought it was cool. Then you add Java to the mix. How many platforms are you supporting? How many different server configurations... and what is that costing you?

Even if you are a Java shop which kind of Java shop are you?

Both Neal Ford and Scott Davis talk about this idea called "The Paradox of Choice." The problem is that in Java land there are far too many choices between frameworks for anyone to comprehend and deal with. The paradox is that when given too many choices people can't make any choice.

Java has so many frameworks for so many different environments that it has become a joke amongst techies: "How many Java frameworks are there? Dunno, a new one comes out every second so what time is it?"

In typical tongue-in-cheek fashion I will instead refer to The Farnsworth Parabox of choice. This is a technique that I think will help. See the Parabox is a box that Professor Farnsworth put the universe inside... or is it a box with a universe inside it? Anyway at the end of the episode the Professor had created a box that contained our universe. Grails is your box. Grails contains the universe of Java but it fits neatly into a box.

Think inside the box. Check to see what works well with Grails. No seriously. If you use the Grails "defaults" you can effectively eliminate the need for making dozens of choices. Which framework to use? Well, what is the Grails default? Spring it is. Which application server to use? Which one comes with Grails? Jetty it is.

When you reach the edge of the box or can't fit a need you have into that Groovy/Grails box that's okay because the whole universe of Java is inside that box too. All you have to do is reach down deeper beneath the Groovy skin and dig into Hibernate, Spring, or EJB3 and that whole cosmos is ready and waiting for you. Conversely, if you are comfortable floating around in the gigantic universe of Java then you can reach down into the box and take out what you need.

Grails is the Java universe in a box.

2008-01-19

the vessel of ideas

Earlier I said:
The word is the vessel of ideas. The written word is the preservation of ideas. The computerized word is the execution of ideas.

Programming is about thinking. It is nothing else. Programs only execute the rules we believe to be true as we have written them down. What do we know? What do we believe?

Take the idea of person, parent, or child these are very basic ideas on the surface but surprisingly few people understand in context the ironic trap that the idea of an
Object then introduces. In the sense of a program...

An Object is a thing.

An Object may have things.

So it seems natural that a parent is a person and a child is a person. That makes sense. A parent has children. A child has parents. That all makes perfect sense. In the now.

We might say:

class Person {
def name
def birthday
def gender
}

... to say a person has a name, a birthday, a gender...

class Parent extends Person {
static hasMany = [children:Person]
}

... to say a parent has one or more children ...

class Child extends Person {
Person mother
Person father
}

... to say a child has one and only one mother and one and only one father... (in the real world we would probably fold Child attributes back into Person)

The terms I've used so far are more or less permanent. Once you become a parent you are never not one. Once you become someone's child that status is never removed. But sometimes there are statuses that are more flexible that aren't modeled correctly by using the "is a" idea. This becomes more apparent when dealing with a term such as employee.

A person is an employee as long as they are employed. You might be employed, unemployed, and re-employed multiple times with the same employer. So should "person is a parent" be modeled the same as "person is an employee" or is being an employee different?

Could it be that ...

class Employee extends Person {
static hasMany = [employers:Employer]
}

... such that we we would add and remove employers over time to an employee?

But that doesn't really get at the idea of being an employee. You are one from one date to another and you might be again. So really you would want to say:

class Employment {
Person employee
Employer employer
Date start
Date end
}

So that a person is an employee if a person has an employment that has a start before now and an end after now. So many other relationships turn out share this same kind of temporal basis. Sadly, marriage does too... some of the data I have to model deals with people who have children and get divorced. The parent-child relationship is unbreakable and begins on the date of birth of the child. However, marriages don't work that way and at least once we've had the problem of modeling a marriage, divorce, and re-marriage. The systems can't handle that idea because they weren't built to hold it. They are poor vessels for these ideas.

It stands to reason that the attribute of being something when it comes to people is not in fact the same as being something when it comes to objects. People are objects in every sense but the being is not the same. A person has attached to them attributes ... like tags more than attributes ...which makes them one thing or another.

The relationship itself is an entity or an object.

The person is an employee by the virtue of possession of a relationship. Once in existence the relationship of employment carries its own being. It has its own data, attributes, and validity independent of the employee or employer.

Likewise the idea of a relationship between people carries similar intrinsic existence independent of the people. For example parent. The person as an object is not really modified but rather a new external relationship is created. In this case a child is created that then points back to the parent.

That means the "class Parent" is actually redundant. A person is a parent if any other people point back to them as a mother or father. If you wanted you could add a deceased date that would remain unused inside the person class and leave it undefined during life and set after death. But that's getting into the morbid.

Consider the employee again. This is a person object who has a record of employment. The employment is distinct and independent of the person. The relationship itself can have attributes without involving the people.

This is all pretty basic but if you continue to expand these ideas shifting your attitude about what an object is you find that mapping objects and relationships starts to get easier. You get to a model that has fewer impedance mis-matches with a relational database for example.

If the vessel will not hold your ideas change the vessel not your ideas.

2008-01-18

Deploying Grails on JBoss 4.2.1

I'm deploying Grails applications into a shared JBoss environment. Now if you've read the Grails FAQ there's a nice bit in there about deployment isolation in JBoss. Specifically you'll use jboss-web.xml and isolate the deployment. If that works for you kudos. Didn't work for me.

When I isolate my deployment I see this error:

17:02:08,115 INFO [STDOUT] [5] digester.Digester Digester.getParser:
java.lang.ClassCastException: org.apache.xerces.jaxp.SAXParserFactoryImpl
at javax.xml.parsers.SAXParserFactory.newInstance(SAXParserFactory.java:107)
at org.apache.tomcat.util.digester.Digester.getFactory(Digester.java:487)
at org.apache.tomcat.util.digester.Digester.getParser(Digester.java:692)
at org.apache.tomcat.util.digester.Digester.getXMLReader(Digester.java:900)
at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1562)
at org.apache.catalina.startup.TldConfig.tldScanStream(TldConfig.java:507)
at org.apache.catalina.startup.TldConfig.tldScanTld(TldConfig.java:544)
at org.apache.catalina.startup.TldConfig.execute(TldConfig.java:294)
at org.apache.catalina.core.StandardContext.processTlds(StandardContext.java:4450)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4257)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:761)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:741)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:553)
at sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:297)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.apache.catalina.core.StandardContext.init(StandardContext.java:5310)
at sun.reflect.GeneratedMethodAccessor229.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:297)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.web.tomcat.service.TomcatDeployer.performDeployInternal(TomcatDeployer.java:301)
at org.jboss.web.tomcat.service.TomcatDeployer.performDeploy(TomcatDeployer.java:104)
at org.jboss.web.AbstractWebDeployer.start(AbstractWebDeployer.java:375)
at org.jboss.web.WebModule.startModule(WebModule.java:83)
at org.jboss.web.WebModule.startService(WebModule.java:61)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:978)
at $Proxy0.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:417)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
at $Proxy182.start(Unknown Source)
at org.jboss.web.AbstractWebContainer.start(AbstractWebContainer.java:466)
at sun.reflect.GeneratedMethodAccessor212.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
at org.jboss.mx.interceptor.DynamicInterceptor.invoke(DynamicInterceptor.java:97)
at org.jboss.system.InterceptorServiceMBeanSupport.invokeNext(InterceptorServiceMBeanSupport.java:238)
at org.jboss.ws.integration.jboss42.DeployerInterceptor.start(DeployerInterceptor.java:93)
at org.jboss.deployment.SubDeployerInterceptorSupport$XMBeanInterceptor.start(SubDeployerInterceptorSupport.java:188)
at org.jboss.deployment.SubDeployerInterceptor.invoke(SubDeployerInterceptor.java:95)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
at $Proxy183.start(Unknown Source)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:1025)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:819)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:782)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
at $Proxy9.deploy(Unknown Source)
at org.jboss.deployment.scanner.URLDeploymentScanner.deploy(URLDeploymentScanner.java:421)
at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:634)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:263)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:274)
at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:225)
My first guess is this has something to do with a conflicting version of the Xerces libraries between JBoss and Grails. But I'd rather not resolve this issue...

If I turn off deployment isolation and then hit a JSP page in a non-Grails application (carefully selecting a JSP that has never been hit before) I get this error:

Caused by: java.lang.AbstractMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext;
at org.apache.jasper.runtime.PageContextImpl.proprietaryEvaluate(PageContextImpl.java:903)
17:11:58,316 INFO [STDOUT] [2008-01-18 17:11:58,314] DEBUG core.ApplicationDispatcher.:185 servletPath=/Login.jsp, pathInfo=null, queryString=null, name=null
17:11:58,317 INFO [STDOUT] [2008-01-18 17:11:58,316] DEBUG core.ApplicationDispatcher.doForward:375 Path Based Forward
17:11:58,319 INFO [STDOUT] [2008-01-18 17:11:58,318] DEBUG servlet.JspServlet.service:249 JspEngine --> /Login.jsp
17:11:58,321 INFO [STDOUT] [2008-01-18 17:11:58,320] DEBUG servlet.JspServlet.service:250 ServletPath: /Login.jsp
17:11:58,323 INFO [STDOUT] [2008-01-18 17:11:58,321] DEBUG servlet.JspServlet.service:251 PathInfo: null
17:11:58,325 INFO [STDOUT] [2008-01-18 17:11:58,324] DEBUG servlet.JspServlet.service:252 RealPath: /home/shawn/jboss/jboss-opengate/server/default/./deploy/VIF.war/Login.jsp
17:11:58,327 INFO [STDOUT] [2008-01-18 17:11:58,326] DEBUG servlet.JspServlet.service:253 RequestURI: /VIF/Login.jsp
17:11:58,329 INFO [STDOUT] [2008-01-18 17:11:58,328] DEBUG servlet.JspServlet.service:254 QueryString: null
17:11:58,331 INFO [STDOUT] [2008-01-18 17:11:58,330] DEBUG servlet.JspServlet.service:255 Request Params:
17:11:58,335 ERROR [STDERR] [2008-01-18 17:11:58,333] ERROR [/VIF].[jsp].invoke:719 Servlet.service() for servlet jsp threw exception
java.lang.AbstractMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext;
at org.apache.jsp.Login_jsp._jspInit(Login_jsp.java:25)
at org.apache.jasper.runtime.HttpJspBase.init(HttpJspBase.java:52)
at org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:159)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:323)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:687)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:403)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
at org.apache.catalina.authenticator.FormAuthenticator.forwardToLoginPage(FormAuthenticator.java:316)
at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:244)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:491)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:241)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:381)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:595)
17:11:58,336 WARN [FormAuthenticator] Unexpected error forwarding to login page
javax.servlet.ServletException: java.lang.AbstractMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext;
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:274)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:687)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:403)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
at org.apache.catalina.authenticator.FormAuthenticator.forwardToLoginPage(FormAuthenticator.java:316)
at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:244)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:491)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:241)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:381)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.AbstractMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext;
at org.apache.jsp.Login_jsp._jspInit(Login_jsp.java:25)
at org.apache.jasper.runtime.HttpJspBase.init(HttpJspBase.java:52)
at org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:159)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:323)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266)
... 20 more
Which I know thanks to this post has something to do with the Jasper compiler. So I turn off isolation remove the jasper, xalan, and xerces libs and viola it all works.

That's because JBoss has this Unified Class Loader... or something. Once a class is seen it's loaded and available to all the non-isolated archives on the system. That's what is solving that first exception when we tried to deploy the WAR while it was isolated. Instead of having our copy of the class and JBoss's copy of the class we have one and only one copy of any given class.

It turns out that the deployment needs to hand back a copy of a org.apache.xerces.jaxp.SAXParserFactoryImpl to the JBoss environment. The classes don't match and boom! your deploy blows up. Remove isolation and you get the Jasper problem. Which doesn't manifest until after you try and use a JSP outside Grails that has never been used before.

The second stack trace is caused by the Jasper libraries embedded in your Grails WAR bleeding out into JBoss land and generally mucking up the JSP compilers in the JBoss container. To stop this remove all the jasper Jars from your application.

This doesn't strike me as best practice but it's the best I've got on short notice.

So the short answer? Remove these JAR files from your Grails WAR when deploying into JBoss 4.2.1GA if you don't use jboss-web.xml:
  • jasper-compiler-5.5.15.jar
  • jasper-compiler-jdt-5.5.15.jar
  • jasper-runtime-5.5.15.jar
  • xalan.jar
  • xerces-2.8.1.jar
  • xercesImpl-2.6.2.jar
  • xercesImpl.jar

Without deployment isolation on a newer JBoss you don't need any of the hibernate JARs either and it's tempting to remove even more JARs to trim the deploy. I haven't done this yet but it seems that because Grails doesn't know what your deployment environment already has in it you can't expect grails to find this combination of classes for you.

If you do use jboss-web.xml to isolate your deployments then you need to make certain your xercesImpl.jar, xalan.jar, and other XML jars are the same version or at least compatible with the JBoss libs.

EDIT
I don't have to do any of this when using Grails 1.0 so this only applies to older Grails versions.

2008-01-16

Behold the Power of Pipe!

This is a repost of an article on pipe I posted several years ago. Pipe is a very powerful tool, behold:

tar -czf - [list of dirs and files] | \
ssh [user]@[server] "cd /path/to/place/files; tar -xzf -"


I learned this one from one of my bosses a few jobs back. There are a few unixy tricks in that command that took me a while to grasp. I've been using this trick for years now and I thought perhaps you would like to see it.

You create a tar archive and specify the file "-" which is your standard out. Then you list the dirs and files you wish to grab. Since the compressed tar is being sent to stdout you can catch it with a pipe "|" and send it to ssh (the "\" is just a way of breaking a command between lines).

With ssh we log into a server as a given user. Then we ask ssh to run a command for us. Since the ssh session will login to the machine at the user's home dir we ask the command session to change to a path, then run the tar command. This time we are extracting the tar archive we caught with pipe by specifying the file "-" to the tar command.

The result is that the files get tarred, compressed, and shot over the network to a remote machine which then immediately untarrs decompresses and drops the files in place. We let tar do all the work of error handling and handling non-regular files, compression is optional, and there are no archives left over.

If, however, we wanted to create a remote archive without ever consuming storage on the local machine we could instead do this:

tar -czf - [list of dirs and files] | \
ssh [user]@[server] "cat > /path/to/place/file.tar.gz"

If processing power was limited on the local machine then we could do this:

tar -cf - [list of dirs and files] | \
ssh [user]@[server] "gzip -c > /path/to/place/file.tar.gz"

2008-01-15

Tautology and the Programmer

I've decided to extract and simplify an example of why a programmer should know what a tautology is... I found very nearly exactly this code in production a few years ago:

if(!a && !b) {
if( ! (a || b) ) {
// stuff
}
}
// ... more unrelated stuff


The if inside another if creates an implicit and ... further obviated by the fact that there is no else to the inner if statement.

But, what's worse the inner if is automatically true if the outer if is also true. The nested if statements form a tautology! The result is that a maintenance programmer must read and comprehend that he is looking at a tautology and understand how to edit the code.

What is particularly bad is if the maintenance programmer doesn't understand that there is a tautology in the nested if statements and is tempted to put an else after the inner if. That nested else will never fire... and the programmer may waste hours or days trying to figure out why.

2008-01-14

Testing & Design Philosophy

I'm continually surprised at how many shops don't test their software or don't have test plans. The ROI on a test plan is being able to say: "I did my due diligence." When you have a contract on the table you can point to a record of testing to say that every possibility that could be thought of was, a plan was made and tested. Without a formal testing plan you are just hanging yourself out there hoping to see nobody calls you out on your mistakes.

I understand that if you are working time and materials that all the company that contracted you is paying for is your time an any materials you consume. They technically aren't paying for a working product. That seems far too under-handed to me. How could anyone sleep at night knowing that's how they worked?

I personally strive to make certain that everything that I produce first does no harm and second does what's expected and third does what's required. Sometimes I've found that these goals are in conflict. But first I want to be certain what I write isn't worse than nothing or worse that total failure.

I suspect that I might be scratching at the core of some kind of oath.

2008-01-08

Fun with Grails configuration files

In a Grails project you can put your own configuration inside the grails-app/conf/Config.groovy file.

In this post is a sample of a configuration I'm using for a WebClient driven service. Certain web client configurations have been pushed out into Config.groovy so that if I need to I can run different configurations for different combinations of products. NOTE: My service performs web automations using the com.gargoylesoftware.htmlunit.WebClient that would otherwise be a screen-scrape operation becoming very brittle.


webclient {
crm {
scheme = "http" // see: RenderRequest.getScheme
server = "192.168.1.40"
port = 8080

agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9'

username = "guest"
password = "guest"

timeout = 12000 // timeout in milliseconds for browser
waitTime = 12000 // time to wait between failed attempts

home {
url = "/CRM/ui.jsp"
}

login {
url = "/CRM/security_check"
params {
username
password
}
}

logout {
url = "/CRM/logoff.jsp"
}
}


Anything you need to do involving multiple data sources, multiple servers, or anything you need to configure to be sensitive to deployment environment can go in the configuration file. For example the following allows for the over ride of different configurations (in your own configs) in different environments... very handy...


environments {
development {
webclient {
crm {
server = "192.168.1.40"
username = "guest"
password = "guest"
}
}
}
test {
webclient {
crm {
server = "192.168.1.41"
username = "guest"
password = "testing"
}
}
}
production {
webclient {
crm {
scheme = "https" // see: RenderRequest.getScheme
server = "192.168.1.42"
port = 443
username = "automatic"
password = "production"
}
}
log4j {
logger {
grails="error,stdout"
}
}
}
}


For more on web automation I found PLEAC very helpful.