As I'm working out SOAP + Langauge du jour problems I continue to be impressed with the utility embedded in the JVM.
Rarely do you get to develop software in a completely greenfield project. Most of us have to integrate our software with other systems and that often means dealing with SOAP. I asked about what people are doing with Groovy + SOAP over at Google+ and had a few side conversations with folks.
It turns out that when forced to deal with the abject insanity that is SOAP, most Groovy users drop back and pick up those Java tools like JAX-WS. It's tools like these that keep developers coming back to the JVM and that Java to Groovy integration story is still one of the best stories going for Groovy.
In part, you have to wonder if ... in order to understand SOAP's madness you don't need a little madness to get the job done. This insanity might have to continue to exist in practical long-lived projects for quite some time. Keeping that madness bottled up in some pre-baked libraries is honestly the best thing you can do for your own continued sanity and the sanity of your own projects.
Showing posts with label groovy. Show all posts
Showing posts with label groovy. Show all posts
2013-01-10
2011-11-01
The Highly Monitored Grails Application @ SpringOne 2GX 2011
My second talk at SpringOne 2GX was "The Highly Monitored Grails Application" and it had a great turn out from a large and lively audience. This was fantastic especially considering I was presenting as the very last presenter of the conference. This was one of the best received talks I've ever given.
The talk was demo heavy and so, while there's not much in the slides we spent a lot of time in my IDE, at the command line, and running through ways to monitor and work with live Grails applications. We even manipulated the hibernate settings in a running Grails application! Great fun!
Definitely the highlight of my SpringOne 2GX conference!
The talk was demo heavy and so, while there's not much in the slides we spent a lot of time in my IDE, at the command line, and running through ways to monitor and work with live Grails applications. We even manipulated the hibernate settings in a running Grails application! Great fun!
Definitely the highlight of my SpringOne 2GX conference!
2011-10-27
Performance Tuning Grails applications at SpringOne 2GX 2011
I just got done with my SpringOne 2GX talk on Performance Tuning Grails Applications. When I proposed this talk six months ago there was no material on this subject. Since then a fantastic webinar came out. At the start of the presentation I mentioned this video which I would call the essential resource. And, you can watch it any time.
I deliberately cut out of my talk much of the subject matter there, why present the same information in a talk that you can download at any time? Wanting to present new and original material at SpringOne 2GX I rewrote my own talk from scratch. This time focused on the performance tuning of the server side code itself.
I decided to focus on real world performance metrics associated with Groovy Code itself. In the talk I demonstrated four implementations of a service three in Groovy and one in Java. The result is a bit surprising. Restructuring your Groovy code yields order-of-magnitude greater performance improvement than simply shedding Groovy in favor of Java.
The lesson to be learned is that the largest expenses your application will pay are in data marshalling, unmarshalling, network transmission time, and database querying. The Groovy itself can be restructured to be more performant without resorting to pure Java. Pure Java is there for us if we really need it but the majority of its benefit is not in that it is Java but in that it removes the most elegant features of functional programming and dynamic type systems from our pallet. I ask if the trade-off is really worth the small difference we see when simple Groovy code restructuring can give us five times better performance.
Slides:
Code:
https://github.com/hartsock/folksonomy
I deliberately cut out of my talk much of the subject matter there, why present the same information in a talk that you can download at any time? Wanting to present new and original material at SpringOne 2GX I rewrote my own talk from scratch. This time focused on the performance tuning of the server side code itself.
I decided to focus on real world performance metrics associated with Groovy Code itself. In the talk I demonstrated four implementations of a service three in Groovy and one in Java. The result is a bit surprising. Restructuring your Groovy code yields order-of-magnitude greater performance improvement than simply shedding Groovy in favor of Java.
The lesson to be learned is that the largest expenses your application will pay are in data marshalling, unmarshalling, network transmission time, and database querying. The Groovy itself can be restructured to be more performant without resorting to pure Java. Pure Java is there for us if we really need it but the majority of its benefit is not in that it is Java but in that it removes the most elegant features of functional programming and dynamic type systems from our pallet. I ask if the trade-off is really worth the small difference we see when simple Groovy code restructuring can give us five times better performance.
Slides:
Code:
https://github.com/hartsock/folksonomy
2011-03-23
GR8Conf
I'm very pleased to be speaking at GR8Conf this year. I'll be covering two topics. I hope the Groovy communities find them interesting.
The first up, how to use Domain Specific Languages in order to reduce accidental complexity. I have a very long post that I'm holding on to based on this topic (inspired by my trip to Manhattan and Google NYC earlier this year). In this talk I'll really take time to explore this idea practically. I will create a Domain Specific Language before your eyes as I do live coding with Test Driven Development. It's performance art with Unit Tests!
The second topic is on Grails and the JPA. I'm currently working on a lab for this topic. We'll explore some of the nastier Object Relational Mapping problems I've encountered working with Grails and with JPA in isolation and together. If time permits I'll work in some performance troubleshooting in this lab too. In this format we'll have the chance to work together and really get a deep understanding of what's going on with Grails persistence and how you can use GORM and/or the JPA in places you may have thought were impossible. I really enjoy the lab format and I hope to see a good turnout.
For me, getting the opportunity to speak at the GR8Conf is a major milestone and I'm very pleased that Open Software Integrators is allowing me to participate. The fine folks at Open Software Integrators have an experienced consulting staff that can help you with all things JVM related. I've had a great time working with the other consultants here over the last few months learning a great deal about performance tuning and scalability.
So please check me out at GR8Conf and I look forward to seeing you there!
2011-02-16
Groovy/Grails: How to disable a controller
Let's say you have a plugin. That plugin includes a controller. You don't want to use the controller provided by the plugin (for whatever reason) but instead you want use your own controller.
What do you do?
Filters my friend. Filters.
Next to your Config.groovy file in the grails-app/conf directory create for your self a file called DisableFilters.groovy and then add a class definition. Here's a filter definition that disables everything in the application... (making your grails application headless).
class DisableFilters {
def filters = {
all(controller:'*', action:'*') {
before = {
// return false
}
after = {
}
afterView = {
}
}
}
}
Now you'll notice there is a before closure and it returns false. This closure fires before the request is passed to its destined controller/action. The * means match all. So if you had a specific controller name or specific controller name pattern you wanted to disable you could easily disable all these by matching the name here.
Notice the nifty after which would execute after the controller and action complete. Finally you have the afterView closure which would execute after the entire call chain was done and the browser has left. I'll leave the uses for these up to your imagination.
In the filter body you have access to everything you have in a grails controller so you have a great deal of flexibility here with what you can do. You could look at usernames, session data, whatever your heart desires. You can even redirect the request to another URL using the same conventions as you would have in a controller. So have at it!
2011-02-09
double entendre
I have said "The Future of Java is Groovy" and I meant it deliberately as a double entendre. The future of Java is indeed groovy and is indeed Groovy. This does not mean Java will be supplanted by Groovy, or some other language. Instead, the future of the JVM and the technology stack that calls the JVM home is tied more to technological off-shoots like Groovy than it is to any other purist re-engineering of the JVM or the core Java language.
Java is what it is at this point. Tools like Groovy give developers the chance to get a gentle introduction to new techniques such as Functional Programming, Parallel Computing, Domain Specific Langauges, Runtime Metaprogramming, and Compiletime Metaprogramming, as well as a whole host of other techniques and technologies.
The beauty of Groovy as a programming language is how accessible it makes all these advanced techniques to a beginner and how readily it will get out of the way of an expert. This is a beautiful balance of the elegant and the ugly that few other languages manage to pull off well.
Do I think Groovy will supplant Java on the JVM? No, not really. Do I think any other language will make that claim? I seriously doubt it. Do I think Java needs a successor? No. No I do not.
The closest ecosystems that we can draw analogies to for the modern JVM are the .Net ecosystem and the C/C++ ecosystem. The C/C++ ecosystem is probably a better environment to draw lessons from since C/C++ long ago shed its lock-in with a single vendor (that being AT&T long long ago).
I think Java is growing up the same way the C/C++ ecosystem did and C hasn't really changed much over the years. I think Java needs to become the same kind of solid foundation. If other languages come and go on the JVM that needs to be groovy with everyone.
I don't steer any ships in the Groovy navy, I don't direct any plans or strategies, so what I say here is just what I say, and what I say is:
That's my opinion anyway. I happen to be one those kinds of guys who likes what I can do with a language. I tend to not get hung up on whether it is pretty, or if it is pure. It's all groovy man.
Java is what it is at this point. Tools like Groovy give developers the chance to get a gentle introduction to new techniques such as Functional Programming, Parallel Computing, Domain Specific Langauges, Runtime Metaprogramming, and Compiletime Metaprogramming, as well as a whole host of other techniques and technologies.
The beauty of Groovy as a programming language is how accessible it makes all these advanced techniques to a beginner and how readily it will get out of the way of an expert. This is a beautiful balance of the elegant and the ugly that few other languages manage to pull off well.
Do I think Groovy will supplant Java on the JVM? No, not really. Do I think any other language will make that claim? I seriously doubt it. Do I think Java needs a successor? No. No I do not.
The closest ecosystems that we can draw analogies to for the modern JVM are the .Net ecosystem and the C/C++ ecosystem. The C/C++ ecosystem is probably a better environment to draw lessons from since C/C++ long ago shed its lock-in with a single vendor (that being AT&T long long ago).
I think Java is growing up the same way the C/C++ ecosystem did and C hasn't really changed much over the years. I think Java needs to become the same kind of solid foundation. If other languages come and go on the JVM that needs to be groovy with everyone.
I don't steer any ships in the Groovy navy, I don't direct any plans or strategies, so what I say here is just what I say, and what I say is:
Groovy allows rapid access to advanced computing concepts with a minimum extraneous cognitive load.This is the Groovy language's strength. It is immediately accessible to anyone with any programming background. This accessibility is the vital engine for the success we are seeing in Groovy adoption. It is also the weakness of the language. It means that Groovy must necessarily avoid the traps of becoming too "beautiful" of a language forcing people into certain forms steepening its learning curve too sharply.
That's my opinion anyway. I happen to be one those kinds of guys who likes what I can do with a language. I tend to not get hung up on whether it is pretty, or if it is pure. It's all groovy man.
2010-10-14
On the topic of Grails Audit Logging Plugin version 0.5.4
Last night, I released a new version of my Audit Logging Plugin. Release notes are here. There had been an outstanding compatibility issue with Grails 1.3.x based on JUnit 4's Integration tests. The bug documented here has been open for several months as I haven't had time to come back to the project to work on it.
Normally I don't blog about tickets but I figure this one requires some attention because I've run into a few plugins that I need in my day job that are "broken" in 1.3.x and I'm not sure who else is having these troubles. In the interest of helping the community as a whole I want to cover my experience.
It turns out that GRAILSPLUGINS-2243 (titled "Integration tests with auditable domain class do not observe test transaction boundary") is actually a more accurate description of the actual bug. The symptom is during integration testing you can't delete entities that are audited. If you take the same code and run it in development or production mode you can delete the objects.
The issue with my code is in this block... (extraneous logging and debugging code removed)
Keep in mind this block functions exactly as expected in Grails 1.2 under all environments I know of. The one environment this code is horribly wrong is Grails 1.3.x in Integration testing. Only at that time does this code cause a problem.
So let me ask you... what's wrong with this code?
Before addressing the real issue, questions I anticipate:
So ... what is wrong with this code again?
Apparently, under integration testing in Grails 1.3.x if you start a transaction in the way I have you will cause a problem with the other independent transactions in the rest of the application. This is simply not the behavior I see in my production systems under Grails 1.2 or in my staging systems running Grails 1.3 for that matter.
My solution is a bit of a work around. In the application context initialization for the Audit Logging plugin (the bit that runs just as you start up your grails application I have a block that does this...
... which simply says in environments that are not production (Notice I'm using the Grails 1.1 API to find that breaking compatibility with Grails 1.0) the boolean flag "transactional" is set to false. I then wrapped the begin and commit calls to the transaction around a check on this boolean. For good measure, I gave the end developer the ability to override this behavior using the setting "auditLog.transactional" in their Config.groovy file. (It would figure that I would release this as a feature and someone needs to use those transactions in their integration testing because they run their testing environment against Oracle or something so here's an out for you to get the old behaviors)
What took so much time to find this bug? It's not clear cut basically. I stumbled on this answer by blindly trying different things until I found a combination that worked. Then I tried to build a theory around this result that fit the facts that I had on hand.
My theory is the JUnit 4 integration environment does something to the session factory so it does not really produce new sessions such that the session that I get in my plugin's listener code is not really a separate session or transaction. I call commit on this transaction and when the session is returned to the normal GORM code it is already committed so the actual entity delete call can't happen and is swallowed.
What is disturbing about this is I didn't invent this technique for Audit Logging I'm implementing here. I created this code after being educated by the Hibernate wiki. This means if my theory holds it's not just a Grails application that will observe this strange-ness ... pretty much any Hibernate application with this kind of Audit Logging will be affected when integration tested under JUnit 4.
Beyond just this anyone developing applications that open their own sessions independent of the GORM session may potentially see bugs during integration testing that do not manifest in other environments. That can be a very difficult situation to deal with. So I've over documented things here for the benefit of the community at large.
In other news...
In other places I've noticed a change in Groovy 1.6 and 1.7's builder API. In particular I use RichUI in one project which makes calls like...
... under Groovy 1.6 yeild was a builder method.
But, under Groovy 1.7 this method is not present. That means the HTML rendered by code that used the yield and yieldUnescaped calls now create tags <yield> and <yieldUnescaped> which breaks some pages (but not all).
So far these are the only surprising issues I've run into that has kept Grails 1.3.x off of my production servers. I haven't yet seen and great benefit to 1.3 so I can't say I'm deeply compelled to move to the new version.
I am looking to rewrite the plugin in light of GRAILS-5725 but this will be a breaking change. Frankly, it's such a large change I'm surprised it was not pushed to Grails 1.4 ... however, it does not appear to interfere with the operation of plugins using the listener injection technique I am also using so no harm no foul.
I see I've lost a whole star in my plugin ratings so, my apologies to my end users who have not had a plugin that works with your Grails 1.3 integration tests. I sincerely hope I can earn back that extra star from you. Thanks for your support and help. I am reviewing code submissions for another version of the plugin coming up and I greatly appreciate your support.
Normally I don't blog about tickets but I figure this one requires some attention because I've run into a few plugins that I need in my day job that are "broken" in 1.3.x and I'm not sure who else is having these troubles. In the interest of helping the community as a whole I want to cover my experience.
It turns out that GRAILSPLUGINS-2243 (titled "Integration tests with auditable domain class do not observe test transaction boundary") is actually a more accurate description of the actual bug. The symptom is during integration testing you can't delete entities that are audited. If you take the same code and run it in development or production mode you can delete the objects.
The issue with my code is in this block... (extraneous logging and debugging code removed)
Session session = sessionFactory.openSession()
def trans = session.beginTransaction()
def saved = session.merge(auditLogObject)
session.flush()
trans.commit()
session.close()
Keep in mind this block functions exactly as expected in Grails 1.2 under all environments I know of. The one environment this code is horribly wrong is Grails 1.3.x in Integration testing. Only at that time does this code cause a problem.
So let me ask you... what's wrong with this code?
Before addressing the real issue, questions I anticipate:
- Why are you opening your own session?
Because on a transactional database in some cases the audit log must fire after the database insert so that I can log the automatically assigned object Id. This means you can't use the open session without invalidating it.
- Why are you using merge and not save?
Because there are edge cases where the AuditLog object will be saved twice and I don't want to dictate the logic for that in this block to keep it simple
- Why are you flushing the session?
Because we want to make sure the audit log is being written outside the regular transaction cycle and this particular session is going to perform no other job we want to ensure the log is written out to the database
So ... what is wrong with this code again?
def trans = session.beginTransaction()
Apparently, under integration testing in Grails 1.3.x if you start a transaction in the way I have you will cause a problem with the other independent transactions in the rest of the application. This is simply not the behavior I see in my production systems under Grails 1.2 or in my staging systems running Grails 1.3 for that matter.
My solution is a bit of a work around. In the application context initialization for the Audit Logging plugin (the bit that runs just as you start up your grails application I have a block that does this...
if( Environment.getCurrent() != Environment.PRODUCTION && ConfigurationHolder.config.auditLog?.transactional == null ) {
transactional = false
}
... which simply says in environments that are not production (Notice I'm using the Grails 1.1 API to find that breaking compatibility with Grails 1.0) the boolean flag "transactional" is set to false. I then wrapped the begin and commit calls to the transaction around a check on this boolean. For good measure, I gave the end developer the ability to override this behavior using the setting "auditLog.transactional" in their Config.groovy file. (It would figure that I would release this as a feature and someone needs to use those transactions in their integration testing because they run their testing environment against Oracle or something so here's an out for you to get the old behaviors)
What took so much time to find this bug? It's not clear cut basically. I stumbled on this answer by blindly trying different things until I found a combination that worked. Then I tried to build a theory around this result that fit the facts that I had on hand.
My theory is the JUnit 4 integration environment does something to the session factory so it does not really produce new sessions such that the session that I get in my plugin's listener code is not really a separate session or transaction. I call commit on this transaction and when the session is returned to the normal GORM code it is already committed so the actual entity delete call can't happen and is swallowed.
What is disturbing about this is I didn't invent this technique for Audit Logging I'm implementing here. I created this code after being educated by the Hibernate wiki. This means if my theory holds it's not just a Grails application that will observe this strange-ness ... pretty much any Hibernate application with this kind of Audit Logging will be affected when integration tested under JUnit 4.
Beyond just this anyone developing applications that open their own sessions independent of the GORM session may potentially see bugs during integration testing that do not manifest in other environments. That can be a very difficult situation to deal with. So I've over documented things here for the benefit of the community at large.
In other news...
In other places I've noticed a change in Groovy 1.6 and 1.7's builder API. In particular I use RichUI in one project which makes calls like...
builder.yield(" var ${attrs.yuiVariableName}DataSource = new YAHOO.util.XHRDataSource(\"${attrs?.action}\");\n", false)
... under Groovy 1.6 yeild was a builder method.
But, under Groovy 1.7 this method is not present. That means the HTML rendered by code that used the yield and yieldUnescaped calls now create tags <yield> and <yieldUnescaped> which breaks some pages (but not all).
So far these are the only surprising issues I've run into that has kept Grails 1.3.x off of my production servers. I haven't yet seen and great benefit to 1.3 so I can't say I'm deeply compelled to move to the new version.
I am looking to rewrite the plugin in light of GRAILS-5725 but this will be a breaking change. Frankly, it's such a large change I'm surprised it was not pushed to Grails 1.4 ... however, it does not appear to interfere with the operation of plugins using the listener injection technique I am also using so no harm no foul.
I see I've lost a whole star in my plugin ratings so, my apologies to my end users who have not had a plugin that works with your Grails 1.3 integration tests. I sincerely hope I can earn back that extra star from you. Thanks for your support and help. I am reviewing code submissions for another version of the plugin coming up and I greatly appreciate your support.
2010-07-09
Three Small Ideas
I was glad to be interviewed by DZone a while ago. The second interview was on my current Open Source work and I gave what is probably a non-traditional answer on why Open Source contribution is important for a developer. The google appengine project I'm talking about is the qrcode linkulator. And, for the record, I've got a better hair-cut now.
My statistics are from this article on Functional Fixedness at wikipedia. The over-lapping concept I'm trying to tease out is that there are problem solving exercises that are not served by tightly focused environments like business. As a developer you need to be able to work in spaces that allow for creative freedom and the lessons you learn there (for better or worse) translate back into your "high pressure" environment. The result is a better developer that is able to think outside the parameters of the problem.
A nice animation covering the idea popularized by Daniel Pink (who is an infinitely better speaker) is on youtube:
My contribution is: not only is open source a product of intrinsic motivation... it creates intrinsic motivation to improve yourself as a developer. These improvements translate back to your "day job" ... so for purely selfish reasons you *should* create open source code. You will be forced to learn. You'll get interaction with other developers. You'll learn much more than if you stay locked in your office and never get challenged by people who never would think like you do and have no reason to pretend they do.
Even if the project is a failure and your contributions are insignificant... you'll learn something.
My statistics are from this article on Functional Fixedness at wikipedia. The over-lapping concept I'm trying to tease out is that there are problem solving exercises that are not served by tightly focused environments like business. As a developer you need to be able to work in spaces that allow for creative freedom and the lessons you learn there (for better or worse) translate back into your "high pressure" environment. The result is a better developer that is able to think outside the parameters of the problem.
A nice animation covering the idea popularized by Daniel Pink (who is an infinitely better speaker) is on youtube:
My contribution is: not only is open source a product of intrinsic motivation... it creates intrinsic motivation to improve yourself as a developer. These improvements translate back to your "day job" ... so for purely selfish reasons you *should* create open source code. You will be forced to learn. You'll get interaction with other developers. You'll learn much more than if you stay locked in your office and never get challenged by people who never would think like you do and have no reason to pretend they do.
Even if the project is a failure and your contributions are insignificant... you'll learn something.
2010-06-29
Grails + Acegi + CAS + Proxy Ticket
Should you have Acegi + CAS setup in Grails and you want to proxy a URL secured behind the same CAS instance you should do something like this:
This only works if your CAS supports proxying and you have a working proxy call back. For details on troubleshooting this with Grails-Acegi 0.5.3 see GRAILSPLUGINS-2231 which includes a working proxy callback controller. Hopefully this will be fixed in an upcoming version so you won't need this reference for long.
If your setup is not working properly the proxyRetriever embedded in the Principal object will be null. If it is null then getProxyTicketFor returns null no matter what. Once you get your CAS and SecurityConfig.groovy working properly the proxyRetriever starts getting magically injected into your application. This doesn't mean you actually have proxying working yet but you can almost rule out client-side configuration errors.
Your Grails application should also have something at /secure/receptor that can catch responses issued by the CAS server. You should see this URI respond to something just after you successfully log in. That URI also needs to be open so the CAS server can post there without needing to login to itself.
NOTE: your CAS server must be able to resolve the URL of the proxy call back. That means if you are running your proxy callback on localhost the responding CAS server must be able to post to that URL. Why? When you request a proxy granting ticket CAS will send the ticket back to your Grails application at the URI /secure/receptor by default. That means your application's URL must be resolvable by the CAS server for proxy tickets to function.
In practice, I run a small CAS server on localhost at an obscure port with the URL to proxy also on localhost when testing proxy ticket granting and other CAS secured proxy services. I work around the SSL issues by using a custom implementation of javax.net.ssl.HostnameVerifier that is installed during bootstrap in development and test-app modes only. The custom verifier (written in Java) looks like:
Other work arounds would involve a staging CAS server that was able to resolve the development or testing host during work on proxying. These are up to you.
URL url = new URL(urlString)
SecurityContext ctx = SecurityContextHolder.getContext();
CasAuthenticationToken auth = ctx?.getAuthentication();
Assertion assertion = auth?.getAssertion();
Principal principal = assertion?.getPrincipal();
assert principal.proxyRetriever != null
// the above line asserts that proxy system is properly setup and working
String proxyTicket = principal?.getProxyTicketFor(url?.toString());
assert proxyTicket != null
URL readUrl = new URL(url.toString() + "?ticket=${proxyTicket}")
This only works if your CAS supports proxying and you have a working proxy call back. For details on troubleshooting this with Grails-Acegi 0.5.3 see GRAILSPLUGINS-2231 which includes a working proxy callback controller. Hopefully this will be fixed in an upcoming version so you won't need this reference for long.
If your setup is not working properly the proxyRetriever embedded in the Principal object will be null. If it is null then getProxyTicketFor returns null no matter what. Once you get your CAS and SecurityConfig.groovy working properly the proxyRetriever starts getting magically injected into your application. This doesn't mean you actually have proxying working yet but you can almost rule out client-side configuration errors.
Your Grails application should also have something at /secure/receptor that can catch responses issued by the CAS server. You should see this URI respond to something just after you successfully log in. That URI also needs to be open so the CAS server can post there without needing to login to itself.
NOTE: your CAS server must be able to resolve the URL of the proxy call back. That means if you are running your proxy callback on localhost the responding CAS server must be able to post to that URL. Why? When you request a proxy granting ticket CAS will send the ticket back to your Grails application at the URI /secure/receptor by default. That means your application's URL must be resolvable by the CAS server for proxy tickets to function.
In practice, I run a small CAS server on localhost at an obscure port with the URL to proxy also on localhost when testing proxy ticket granting and other CAS secured proxy services. I work around the SSL issues by using a custom implementation of javax.net.ssl.HostnameVerifier that is installed during bootstrap in development and test-app modes only. The custom verifier (written in Java) looks like:
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLSession;
import javax.net.ssl.HttpsURLConnection;
public class CustomHostnameVerifier implements HostnameVerifier {
public String[] exemptNames = {"localhost","testing.com","example.com"};
// whatever hosts need exempting from having a "real" SSL certificate
public boolean verify(String hostname, SSLSession session) {
boolean exempt = false;
if(exemptNames != null) {
for(String name:exemptNames) {
if(hostname.toLowerCase().endsWith(name)) {
exempt = true;
}
}
}
return exempt;
}
// call from BootStrap.groovy or other opportune initializer
public static void init() {
HttpsURLConnection.setDefaultHostnameVerifier(new CustomHostnameVerifier());
}
}
Other work arounds would involve a staging CAS server that was able to resolve the development or testing host during work on proxying. These are up to you.
2010-03-28
QRCode Linkulator speaking Gaelyk
I've just written my first application using Gaelyk a Groovy based framework for Google App Engine. I ported the qrcode plugin I had written for Grails over to Gaelyk in a matter of hours.
This isn't really an application for iPhone or Android but it relies on the Barcode Scanner on Android or an app like
Barcodes on the iTunes AppStore for iPhone. Combined with a tiny "bookmarklet" I found the result is quite useful for moving a link between my desktop and my phone. I made a demonstration video since this can be a little hard to explain.
You can get a gander at the app here: QRCode Linkulator. I make use of the HTTP referer header property to determine where you came from so if you were to click http://qrcodelinkulator.appspot.com/300 for example then you would get a QRCode for the URL you were at *before* you clicked the link. The error mode (should you have no "referer" associated with your browser (for example if you were to right-click the link and open it in a new window) is just the logo for the app.
You might notice I tested out this feature in my blog. When you hit a given page in my blog there will be a QRCode for the URL you are currently looking at embedded in the page. This is done with simple image tags and no javascript.
Right now, I'm just hosting the app on the free AppEngine account and hoping that is enough CPU + Bandwidth for the application. I'm also hoping to stress-test the app a little to get a feel for how things scale. So far Gaelyk is running error free. I'm quite pleased with the simplicity of the framework and the price is right for App Engine hosting.
I doubt I'll get enough traffic to have to worry about paying for the service if the app gets that popular I'll have to figure out how to get it to pay for itself. Any ideas?
This isn't really an application for iPhone or Android but it relies on the Barcode Scanner on Android or an app like
Barcodes on the iTunes AppStore for iPhone. Combined with a tiny "bookmarklet" I found the result is quite useful for moving a link between my desktop and my phone. I made a demonstration video since this can be a little hard to explain.
You can get a gander at the app here: QRCode Linkulator. I make use of the HTTP referer header property to determine where you came from so if you were to click http://qrcodelinkulator.appspot.com/300 for example then you would get a QRCode for the URL you were at *before* you clicked the link. The error mode (should you have no "referer" associated with your browser (for example if you were to right-click the link and open it in a new window) is just the logo for the app.
You might notice I tested out this feature in my blog. When you hit a given page in my blog there will be a QRCode for the URL you are currently looking at embedded in the page. This is done with simple image tags and no javascript.
Right now, I'm just hosting the app on the free AppEngine account and hoping that is enough CPU + Bandwidth for the application. I'm also hoping to stress-test the app a little to get a feel for how things scale. So far Gaelyk is running error free. I'm quite pleased with the simplicity of the framework and the price is right for App Engine hosting.
I doubt I'll get enough traffic to have to worry about paying for the service if the app gets that popular I'll have to figure out how to get it to pay for itself. Any ideas?
2009-12-10
Domain Specific Languages: fighting accidental complexity in a sea of technology
I had a great time presenting at UNC's Programming Practices group today. This was the first time I've presented in about a decade so I'm glad that folks stuck around and gave me feed back.
Slides from the presentation:
I wrote speaker's notes for nearly each slide. If you click the menu button and click the link to the full document you can find a button that will pop up the speaker's notes for you.
For code samples, take a look at this example by Tim Yates. My live coding example was a reimplementation of this using a Test Driven Design (TDD) technique. TDD doesn't translate well into a recorded medium. I tried to show how we could stumble toward the same (or similar) design by using TDD and what we knew to extend our knowledge and grow beyond it.
I made a simple extension on Tim's example. I added a simple way to make his internal DSL into an external DSL. I use the GroovyShell object:
Finally, I tried to tie the idea that Domain Specific Languages hide accidental complexity of a system behind "simpler" API and externalize-able syntax. These goals provide not just more focused ways for application developers to write software. They outline a design aesthetic. An aesthetic that fits well with the idea of empowering amateur programmers.
I ended with the assertion that this sea of technology will end in a world where everyone programs. But not everyone will be a programmer. We will think of programming the way we think of writing. Just because you can write it doesn't make you a writer. The ubiquity of computing means humanity will eventually view computer programming the same way... as ubiquitous as reading and writing.
One way to make sure this happens is to develop Domain Specific Languages that empower domain experts to write software.
Slides from the presentation:
I wrote speaker's notes for nearly each slide. If you click the menu button and click the link to the full document you can find a button that will pop up the speaker's notes for you.
For code samples, take a look at this example by Tim Yates. My live coding example was a reimplementation of this using a Test Driven Design (TDD) technique. TDD doesn't translate well into a recorded medium. I tried to show how we could stumble toward the same (or similar) design by using TDD and what we knew to extend our knowledge and grow beyond it.
I made a simple extension on Tim's example. I added a simple way to make his internal DSL into an external DSL. I use the GroovyShell object:
// def list = DateDSL.interpret(stringOfCode) // could be a file?
static interpret(String code) {
def shell = new GroovyShell();
Closure closure = shell.evaluate("{->" + code + "}")
closure.delegate = new DateDSL()
closure.call()
}
Finally, I tried to tie the idea that Domain Specific Languages hide accidental complexity of a system behind "simpler" API and externalize-able syntax. These goals provide not just more focused ways for application developers to write software. They outline a design aesthetic. An aesthetic that fits well with the idea of empowering amateur programmers.
I ended with the assertion that this sea of technology will end in a world where everyone programs. But not everyone will be a programmer. We will think of programming the way we think of writing. Just because you can write it doesn't make you a writer. The ubiquity of computing means humanity will eventually view computer programming the same way... as ubiquitous as reading and writing.
One way to make sure this happens is to develop Domain Specific Languages that empower domain experts to write software.
2009-11-10
Simple Grails Plugin Repository Mirror Script
We needed a local offline repository for Grails plugins. I thought this would be a good chance to contribute a grails application to the community. It turns out there already is one called plugrepo that you can just download and use.
The plugrepo grails application is a very nice looking and if I'd known about it I wouldn't have spent the effort building my own.
What started this is a command line Groovy script I had hooked to a cron job on our Ubuntu Linux Apache server to create a mirror. I'll go ahead and post this here in case someone would rather have a flat Apache served archive... otherwise take a look at plugrepo.
To use the archive setup your grails app with this additional configuration file:
The following groovy script is meant to be run on the Ubuntu command line and
populate the /var/www/grails directory as if it were a grails plugin repository.
The plugrepo grails application is a very nice looking and if I'd known about it I wouldn't have spent the effort building my own.
What started this is a command line Groovy script I had hooked to a cron job on our Ubuntu Linux Apache server to create a mirror. I'll go ahead and post this here in case someone would rather have a flat Apache served archive... otherwise take a look at plugrepo.
To use the archive setup your grails app with this additional configuration file:
// grails-app/conf/BuildConfig.groovy
grails.plugin.repos.discovery.myRepository="http://myhost.test.com/grails/"
The following groovy script is meant to be run on the Ubuntu command line and
populate the /var/www/grails directory as if it were a grails plugin repository.
#!/usr/bin/env groovy
import groovy.xml.MarkupBuilder
/**
* Shawn Hartsock
*
* This is a quick and dirty Groovy CLI script for creating Grails plugin
* repository mirrors.
*/
def uri = "http://myhostname.test.com/grails"
def destinationRoot = "/var/www/grails"
def version = "grails-1.1.1"
def separator = System.getProperty("file.separator")
def latestOnly = true
if(!args) {
println """
this script is designed to take plugins-list.xml files (as arguments) from
a ~${separator}grails${separator}1.1.1 (or other version) folder and download each
plugin to the $root working directory.
It will then output a modified a $destinationRoot${separator}.plugin-meta${separator}plugins-list.xml
file modified to reflect the new locations of the files.
"""
System.exit(1);
}
try {
def destination = destinationRoot + separator + version
new File(destination).mkdir()
def writer = new StringWriter()
def xml = new MarkupBuilder(writer)
List files = []
args.each { arg ->
if(arg == "-a") {
latestOnly = false
}
if(new File(arg).exists()) {
files.add(arg)
}
}
xml.plugins(revision:2) {
files.each { file ->
process(file, destination, separator, uri + "/" + version, xml, latestOnly)
}
}
new File(destinationRoot + separator + ".plugin-meta").mkdir()
def outFile = destinationRoot + separator + ".plugin-meta" + separator + "plugins-list.xml"
def out = new File( outFile )
if(out.exists()) {
if(out.delete()) {
out = new File( outFile )
}
}
out << writer.toString()
} catch(Exception ex) {
System.err.println(ex.message);
System.exit(0);
}
void process(String fileName, String destination, String separator, String uri, xml, latestOnly) throws Exception {
def text = new File(fileName).text // may throw file access exceptions
def plugins
try {
plugins = new XmlParser().parseText(text)
assert plugins?.plugin.size() > 0
assert plugins.plugin[0].release != null
assert plugins.plugin[0].release.file != null
} catch(java.lang.AssertionError ae) {
throw new Exception("File $fileName does not contain plugin definitions: " + ae.message)
} catch(Exception ex) {
throw new Exception("File $fileName could not be parsed as plugin XML: " + ex.message)
}
println "${plugins.plugin.size()} plugins found in $fileName "
println "\t"
plugins.plugin.each { p ->
xml.plugin( name:p.'@name', 'latest-release':p.'@latest-release') {
p.release.each { r ->
def get = true
if(latestOnly) get = (r.'@version' == p.'@latest-release')?true:false
if(get) {
def f = download(r.file.text(), destination, separator)
if( f ) {
def url = uri + "/" + f
println "\tregistering:\n\t " + url
release(type:r.'@type',tag:r.'@tag',version:r.'@version') {
title(r.title.text())
author(r.author.text())
authorEmail(r.authorEmail.text())
description(r.description.text())
documentation(r.documentation.text())
file(url)
}
}
}
}
}
}
}
def download(String urlString, String dest, String separator) {
// note: this is a web URL so you tokenize on "/" not file.separator
def fileName = urlString.tokenize("/")[-1] // last name on URL
def destFileName = dest + separator + fileName
if( new File(destFileName).exists() ) {
println "\t $destFileName already present in the repository"
return fileName
}
print "\tdownloading $fileName to $dest directory\t"
def file = new FileOutputStream( destFileName )
def out = new BufferedOutputStream(file)
out << new URL(urlString).openStream()
out.close()
println "."
return fileName
}
2009-11-02
Apache Ivy: Componentization? What's hot and what's not?
I'm working with Apache Ivy over the next few weeks. The problems I'm trying to solve are around the testability of a J2EE application and its functional decomposition. I have chosen Ivy after an evaluation period due to its simplicity and how easy it is to port Ivy into existing Ant build systems.
In the case of the Grails applications I'm working with this problem domain is all rather straight forward. The Grails applications can be functionally decomposed easily along plugins and modules. They were all developed with a Test Driven Design (TDD) mentality and so have a large suite of test automations around the application. NOTE: I have not yet picked a code-coverage tool for this environment, however, so don't ask about code coverage for now okay... suggestions are welcome.
The problem still lies with the J2EE application. I have several components I can identify that have not changed in years since the original system designers bequeathed the code to their heirs. The original system had no automated testing but had ample "test scripts" which were human driven. In some respects I'm shocked at this approach to testing but I can't say it surprises me.
I've been in work environments that literally employed armies of testers. (No really, it was literally the Army... literally armies of testers.) And while there is a place for this if you can afford it ... doesn't it make sense to focus those human-level testers on testing human-level problems? I'm not talking about getting rid of those armies of testers just focusing them on the most interesting problems, saving their collective power for the big issues.
So, here's my hypothesis about how you should decompose an application already long in development, with no automated test harness so it can be better managed. It's a work in progress so please help me knock off the rough edges or if you think I'm daft... let me know.
I'm operating under the theory that the human testing in fact exercises all relevant existing code as it is compiled and bundled up as a part of the whole system. Any components we identify were in that tested whole. Any components we create will be tested under the unified whole. Therefore any movement of components creates no net change in the whole. This initially appears pointless but positions us to begin creating automated unit and integration tests around the identified components. The outcome creates no user-visible results initially but is incredibly important since it makes adding features much more certain.
The end goal of the introduction of Ivy is to identify stable framework components and put those components under tests that mimic the current human-based test scripts. The end result will be the ability to identify the volatile system components and isolate them for focused testing and design work. This is desirable because you isolate the system's accidental complexity and help keep it away from its intrinsic complexity. You should in the process be able to identify layers of abstraction in the system.
NOTE: An interesting side-effect is that classes designed for use with RMI may not change frequently right now but we have observed that they will "break" backwards compatibility at seemingly random intervals. Since our system is distributed this poses a problem. Decomposing these RMI interfaces and classes into their own Jar (compiling them separately and only on the event that they are changed) means actions that change them thus breaking compatibility between nodes will become very apparent since it will be harder to make the change accidentally.
Working theory: Componentization of large existing systems
I'm thinking that (in the large existing system) you want to only decouple packages of classes from the grand unified build that have little change between revisions. You should be able to identify these "stable components" by creating a "heat map" of the repository and watching the rate-of-change in the change control system. The more frequent the changes the "hotter" the class. The "hotter" the class the closer to the other frequently changing components it should be... ostensibly going under a new round of full tests with them. I would only select the coldest classes to be moved into components for control by Ivy.
I will take these carefully selected classes and move them to a separate module to be built and packaged as a single Jar file. These will be placed in the Enterprise Ivy repository for management by Ivy. At build time the project will download these Jar files, just like other Ivy dependencies, from the Enterprise's Ivy repository.
When the unified whole goes under test part of that whole will be the independent Ivy managed Jars. These Jars can be instrumented during our human-driven tests to see how they are exercised by those armies of humans. With that documentation I can then devise unit and integration tests to reenact those human-level tests on the isolated Jars. That means, the next round of the application's life cycle will have a set of automated tests that document how the system operates.
Once we know what the colder classes do we can begin to formulate a framework based on them. And that begins the first steps to identifying and targeting changes to the system to add to what it can do... or designing a replacement... or designing a new feature. Each time behavior changes in the future it will be more explainable and thus more controllable.
And you start getting there by identifying what's hot and what's not.
Have I gone wrong? Commentary?
In the case of the Grails applications I'm working with this problem domain is all rather straight forward. The Grails applications can be functionally decomposed easily along plugins and modules. They were all developed with a Test Driven Design (TDD) mentality and so have a large suite of test automations around the application. NOTE: I have not yet picked a code-coverage tool for this environment, however, so don't ask about code coverage for now okay... suggestions are welcome.
The problem still lies with the J2EE application. I have several components I can identify that have not changed in years since the original system designers bequeathed the code to their heirs. The original system had no automated testing but had ample "test scripts" which were human driven. In some respects I'm shocked at this approach to testing but I can't say it surprises me.
I've been in work environments that literally employed armies of testers. (No really, it was literally the Army... literally armies of testers.) And while there is a place for this if you can afford it ... doesn't it make sense to focus those human-level testers on testing human-level problems? I'm not talking about getting rid of those armies of testers just focusing them on the most interesting problems, saving their collective power for the big issues.
So, here's my hypothesis about how you should decompose an application already long in development, with no automated test harness so it can be better managed. It's a work in progress so please help me knock off the rough edges or if you think I'm daft... let me know.
I'm operating under the theory that the human testing in fact exercises all relevant existing code as it is compiled and bundled up as a part of the whole system. Any components we identify were in that tested whole. Any components we create will be tested under the unified whole. Therefore any movement of components creates no net change in the whole. This initially appears pointless but positions us to begin creating automated unit and integration tests around the identified components. The outcome creates no user-visible results initially but is incredibly important since it makes adding features much more certain.
The end goal of the introduction of Ivy is to identify stable framework components and put those components under tests that mimic the current human-based test scripts. The end result will be the ability to identify the volatile system components and isolate them for focused testing and design work. This is desirable because you isolate the system's accidental complexity and help keep it away from its intrinsic complexity. You should in the process be able to identify layers of abstraction in the system.
NOTE: An interesting side-effect is that classes designed for use with RMI may not change frequently right now but we have observed that they will "break" backwards compatibility at seemingly random intervals. Since our system is distributed this poses a problem. Decomposing these RMI interfaces and classes into their own Jar (compiling them separately and only on the event that they are changed) means actions that change them thus breaking compatibility between nodes will become very apparent since it will be harder to make the change accidentally.
Working theory: Componentization of large existing systems
I'm thinking that (in the large existing system) you want to only decouple packages of classes from the grand unified build that have little change between revisions. You should be able to identify these "stable components" by creating a "heat map" of the repository and watching the rate-of-change in the change control system. The more frequent the changes the "hotter" the class. The "hotter" the class the closer to the other frequently changing components it should be... ostensibly going under a new round of full tests with them. I would only select the coldest classes to be moved into components for control by Ivy.
I will take these carefully selected classes and move them to a separate module to be built and packaged as a single Jar file. These will be placed in the Enterprise Ivy repository for management by Ivy. At build time the project will download these Jar files, just like other Ivy dependencies, from the Enterprise's Ivy repository.
When the unified whole goes under test part of that whole will be the independent Ivy managed Jars. These Jars can be instrumented during our human-driven tests to see how they are exercised by those armies of humans. With that documentation I can then devise unit and integration tests to reenact those human-level tests on the isolated Jars. That means, the next round of the application's life cycle will have a set of automated tests that document how the system operates.
Once we know what the colder classes do we can begin to formulate a framework based on them. And that begins the first steps to identifying and targeting changes to the system to add to what it can do... or designing a replacement... or designing a new feature. Each time behavior changes in the future it will be more explainable and thus more controllable.
And you start getting there by identifying what's hot and what's not.
Have I gone wrong? Commentary?
2009-10-01
Should I put that logic in my Grails domain class, service class, or a controller?
I don't honestly know. I do have a feel for when this is appropriate and when it isn't. For example I think Anemic Domain Model is a valid anti-point. It completely jibes with how I feel the benefit of an Object Oriented Programming (OOP) system should work... in that I don't think domain classes should be just a representation of data.
The whole point of OOP is to encapsulate data and methods in a modular way keeping logic bundled together with the data it operates on. Let's work on a concrete example... assume this is a domain class in a Grails project.
So far so good. So if I want to figure out a person's age... where do I put that? The class seems like a good place...
... in fact anything that operates on the domain data in the object seems like it fits well here. If we run generate-all on this class we get a UI that looks like this...
Marking non-persistent services as transient keeps them from getting sent to the database should we need them... for example if I had a "fooService" then I would get this injected in my domain class like so...
So when does the logic belong in a service or a controller? Well, when it stops being directly driven by the domain model. For example things could get messy if we start injecting security checks into the domain model. I would find that a bit messy to maintain.
Similarly if there was a Person to Person relationship that required some logic that checked states of both objects...
... while this works it feels a tad clunky because I have to do this bit ...
... and since we have to somehow modify the other class too and it feels odd to have the this person object modify the other person object.
... but if there was a minister service I could write this...
... and this feels a tad better since the Person class no longer needs to carry around the knowledge of all the possible marriage rules that a lawyer might enact. This also models real life better since people visit ministers, justices, or captains to get married in real life... and we cleanly avoid the issue of people modifying each other.
There's other rules that I could probably explain if I was more articulate. Has anyone else thought about this? What tweaks would you put on these ideas? How would you express this rule?
Note: this is a quick reply to this tweet.
The whole point of OOP is to encapsulate data and methods in a modular way keeping logic bundled together with the data it operates on. Let's work on a concrete example... assume this is a domain class in a Grails project.
class Person {
String firstName
String lastName
Date birthDate
}
So far so good. So if I want to figure out a person's age... where do I put that? The class seems like a good place...
class Person {
String firstName
String lastName
Date birthDate
def getAge() {
def now = new Date()
now[Calendar.YEAR] - birthDate[Calendar.YEAR]
}
}
... in fact anything that operates on the domain data in the object seems like it fits well here. If we run generate-all on this class we get a UI that looks like this...
Marking non-persistent services as transient keeps them from getting sent to the database should we need them... for example if I had a "fooService" then I would get this injected in my domain class like so...
... but I'll not bother with an example just now.
class Person {
transient fooService
// ... snip ...
So when does the logic belong in a service or a controller? Well, when it stops being directly driven by the domain model. For example things could get messy if we start injecting security checks into the domain model. I would find that a bit messy to maintain.
Similarly if there was a Person to Person relationship that required some logic that checked states of both objects...
class Person {
Person spouse
//... snip ...
void marry(Person other) {
if(!other.spouse || other.spouse.id == this.id ) {
other.spouse = person
}
else {
throw new Exception("These two cannot wed")
}
}
... while this works it feels a tad clunky because I have to do this bit ...
if(!other.spouse || other.spouse.id == this.id )
... and since we have to somehow modify the other class too and it feels odd to have the this person object modify the other person object.
... but if there was a minister service I could write this...
class MinisterService {
//... snip ...
void marry(Person left, Person right) {
if(!left.spouse && !right.spouse) {
left.spouse = right
right.spouse = left
}
else {
throw new Exception("These two cannot wed")
}
}
// ... snip ...
}
... and this feels a tad better since the Person class no longer needs to carry around the knowledge of all the possible marriage rules that a lawyer might enact. This also models real life better since people visit ministers, justices, or captains to get married in real life... and we cleanly avoid the issue of people modifying each other.
There's other rules that I could probably explain if I was more articulate. Has anyone else thought about this? What tweaks would you put on these ideas? How would you express this rule?
Note: this is a quick reply to this tweet.
2009-09-02
Working on a new Audit plugin release
I'm working on a new release for the Grails Audit Logging Plugin that deals with issues some people were having with transactional databases.
I'd like to test this on multiple databases and with multiple configurations. So I'm putting the word out that I'd like some help with testing this. I don't have more than four databases to test on so I'd like some help.
If you can help out let me know. Contact me through this blog, twitter, or carrier pigeon. Thanks.
I'd like to test this on multiple databases and with multiple configurations. So I'm putting the word out that I'd like some help with testing this. I don't have more than four databases to test on so I'd like some help.
If you can help out let me know. Contact me through this blog, twitter, or carrier pigeon. Thanks.
2009-08-21
Groovy Automatic type casting tricks...
I've been working on a Domain Specific Language. We wanted to hide the fact that you were passing string parameters to methods... so you could say things like:
...and they would be equivalent. As the project progressed I needed to make calls like these...
... again we want the two calls to be equivalent. But then I needed to reference the implicit tree of properties created by calling 'foo.bar' and 'foo.bar.baz' when I was interpreting these statements. So I needed to start tracking the symbols.
So my text-book meta-programming trick ...
... stopped working since foo.bar would result in a property missing on the String object. I could do some meta-programming on the String object but I also wanted to preserve the tree relationship for other work later. So with that in mind I wrote a class to wrap the symbols.
But there's a problem with that. If I introduce this new class... there are many places I just want the symbol like 'foo.bar.baz' and not the complex relationship. And by this point in the project I've written a lot of code to deal with things as strings.
Wouldn't it be great if I could ignore the complex nature of the object most of the time... but pick out the complex bits when I wanted them? To that end I created this class ComplexSymbol to do that work. I empowered it to know how to turn itself into a string silently in all sorts of situations.
So I wrote some tests to express what I wanted to see...
... and I wanted to see the class of Symbol silently swap in for String whenever the method that was being called was expecting a string and not a complex symbol.
Take a look at the class I came up with to do this job...
I've got lots of explicit typing in places to document the use of the class since I expect this class to show up in Java and Groovy code. The typing is really for documentation as it should be.
In the class where I want ComplexSymbol to stand in for naked strings I now do this:
... which sets things up nicely so that the following works ...
... and someMethod should get called with a string "foo.bar.baz" but before we get to that point I need to sanity check things.
Now to test the class with some methods... I just dumped the text for the above class into a file called symbols.groovy and stuck these methods at the end of the script so I could call them.
This one should just see if it gets a string and I'll pass my ComplexSymbol to it and see if it gets the string.
This method explicitly takes the complex symbol and uses the asType explict cast on it. This should show that the cast works.
The implicit cast is my holy grail. If I can get the type system to see String whenever it asks for a String or the full ComplexSymbol type when it can handle that then I've got the whole enchilada.
... command line output of this ...
Notice that the last assert (my holy grail) fails. What I have is a very nice taco if not the whole enchilada.
Methods like print and println are going to implicitly handle the ComplexSymbol properly since they'll call the toString() on the object. And for methods that don't call ".toString()" we can either call it for them or explicitly cast to a String just before calling them.
All and all, it's a nice compromise with static and dynamic typing that allows me to use this class in plain Java code or in Groovy code very easily.
If someone has some thoughts on how I can get assert implicitlyCast(symbol) to work I'd love to hear them. With out it though, it's simple enough to spot the problem in a test case and put "as String" in the call like so...
... and as I have incorporated this class into my work I've found points where I don't want the implicit cast to work all the time! I actually want to go into ComplexSymbol and ask for the symbol property directly. It turns out I actually need to hold the leaf nodes by their leaf node value sometimes and having them automatically resolve to their whole name would cause problems.
So in the end I'm glad the implicitlyCast test doesn't work the way I imagined.
something 'foo'
something foo
...and they would be equivalent. As the project progressed I needed to make calls like these...
something 'foo.bar'
something foo.bar
... again we want the two calls to be equivalent. But then I needed to reference the implicit tree of properties created by calling 'foo.bar' and 'foo.bar.baz' when I was interpreting these statements. So I needed to start tracking the symbols.
So my text-book meta-programming trick ...
def propertyMissing(String name) {
return name
}
... stopped working since foo.bar would result in a property missing on the String object. I could do some meta-programming on the String object but I also wanted to preserve the tree relationship for other work later. So with that in mind I wrote a class to wrap the symbols.
But there's a problem with that. If I introduce this new class... there are many places I just want the symbol like 'foo.bar.baz' and not the complex relationship. And by this point in the project I've written a lot of code to deal with things as strings.
Wouldn't it be great if I could ignore the complex nature of the object most of the time... but pick out the complex bits when I wanted them? To that end I created this class ComplexSymbol to do that work. I empowered it to know how to turn itself into a string silently in all sorts of situations.
So I wrote some tests to express what I wanted to see...
assert symbol == "foo"
assert symbol.bar == "foo.bar"
assert symbol.baz.bing == "foo.baz.bing"
assert symbol.baz.blat == "foo.baz.blat"
assert symbol.bar.blat == "foo.bar.blat"
... and I wanted to see the class of Symbol silently swap in for String whenever the method that was being called was expecting a string and not a complex symbol.
Take a look at the class I came up with to do this job...
public class ComplexSymbol {
ComplexSymbol parent
String symbol
Map symbols = [:]
ComplexSymbol(String str) { symbol = new String(str) }
ComplexSymbol(String str, ComplexSymbol other) {
symbol = new String(str);
parent = other
}
String toString() {
if(parent) {
return parent.toString() + "." + this.symbol
}
return symbol
}
def propertyMissing(String sym) {
if(symbols.containsKey(sym)) {
return symbols[sym]
}
def obj = new ComplexSymbol(sym,this)
symbols[sym] = obj
return obj
}
def asType(Class clazz) {
Object obj = null
switch(clazz) {
case java.lang.String:
obj = this.toString()
break
}
return obj
}
boolean equals(Object other) {
this.toString().equals(other.toString())
}
}
I've got lots of explicit typing in places to document the use of the class since I expect this class to show up in Java and Groovy code. The typing is really for documentation as it should be.
In the class where I want ComplexSymbol to stand in for naked strings I now do this:
def propertyMissing(String name) {
return new ComplexSymbol(name)
}
... which sets things up nicely so that the following works ...
someMethod foo.bar.baz
... and someMethod should get called with a string "foo.bar.baz" but before we get to that point I need to sanity check things.
Now to test the class with some methods... I just dumped the text for the above class into a file called symbols.groovy and stuck these methods at the end of the script so I could call them.
boolean stringify(String str) {
return str != null
}
This one should just see if it gets a string and I'll pass my ComplexSymbol to it and see if it gets the string.
boolean stringify(ComplexSymbol sym) {
return stringify(sym as String)
}
This method explicitly takes the complex symbol and uses the asType explict cast on it. This should show that the cast works.
boolean implicitlyCast(String sym) {
return sym != null
}
The implicit cast is my holy grail. If I can get the type system to see String whenever it asks for a String or the full ComplexSymbol type when it can handle that then I've got the whole enchilada.
/** Here's the tests again **/
def symbol = new ComplexSymbol("foo")
assert symbol == "foo"
assert symbol.bar == "foo.bar"
assert symbol.baz.bing == "foo.baz.bing"
assert symbol.baz.blat == "foo.baz.blat"
assert symbol.bar.blat == "foo.bar.blat"
assert stringify(symbol)
println "-" * 40
println symbol
println "-" * 40
println symbol as String
println "-" * 40
println symbol.toString()
println "-" * 40
assert implicitlyCast(symbol)
... command line output of this ...
$ groovy symbol.groovy
----------------------------------------
foo
----------------------------------------
foo
----------------------------------------
foo
----------------------------------------
Caught: groovy.lang.MissingMethodException: No signature of method: symbol.implicitlyCast() is applicable for argument types: (ComplexSymbol) values: [foo]
at symbol.run(symbol.groovy:71)
Notice that the last assert (my holy grail) fails. What I have is a very nice taco if not the whole enchilada.
Methods like print and println are going to implicitly handle the ComplexSymbol properly since they'll call the toString() on the object. And for methods that don't call ".toString()" we can either call it for them or explicitly cast to a String just before calling them.
All and all, it's a nice compromise with static and dynamic typing that allows me to use this class in plain Java code or in Groovy code very easily.
If someone has some thoughts on how I can get assert implicitlyCast(symbol) to work I'd love to hear them. With out it though, it's simple enough to spot the problem in a test case and put "as String" in the call like so...
assert implicitlyCast(symbol as String)
... and as I have incorporated this class into my work I've found points where I don't want the implicit cast to work all the time! I actually want to go into ComplexSymbol and ask for the symbol property directly. It turns out I actually need to hold the leaf nodes by their leaf node value sometimes and having them automatically resolve to their whole name would cause problems.
So in the end I'm glad the implicitlyCast test doesn't work the way I imagined.
2009-06-04
Grails in a J2EE world - The Database @ GroovyMag
In this month's GroovyMag I talk about sharing a database between a J2EE application and a Grails application that is using GORM. If you haven't spent anytime working with concurrency in database applications you will really want to check this out. If you have a Container Managed Persistence (CMP) application that you are working with and you have a green-light on using Grails you might consider moving directly from EJB2 to GORM in a phased manner sharing parts of the database on the way.
Thanks again to Craig Wickesser for editing.
Please head over to GroovyMag and check out the article and give me some feedback.
EDIT
I'll also be giving out 5 free PDF copies of GroovyMag to the first five people who can Direct Message me on Twitter the answer to this riddle: What do you call an organized body of related information?
EDIT
That went fast! Congrats to @vintagedance, @evanbooth, @paulk_asert, @iansari, and @unixfudotnet
@unixfudotnet has offered his coupon up to the first person who wants it. Congrats to @dferrand.
Our answer? A database
Thanks again to Craig Wickesser for editing.
Please head over to GroovyMag and check out the article and give me some feedback.
EDIT
I'll also be giving out 5 free PDF copies of GroovyMag to the first five people who can Direct Message me on Twitter the answer to this riddle: What do you call an organized body of related information?
EDIT
That went fast! Congrats to @vintagedance, @evanbooth, @paulk_asert, @iansari, and @unixfudotnet
@unixfudotnet has offered his coupon up to the first person who wants it. Congrats to @dferrand.
Our answer? A database
2009-05-01
Grails in a J2EE world - The Web @GroovyMag
The new GroovyMag is available. This month I've made a contribution titled "Grails in a J2EE world - The Web" as I've been exploring ways that Grails and J2EE applications can interact I've found that the topic area is so rich that I can't possibly cover the whole thing in just one article. Instead I'll be making contributions in installments on the topic.
This particurlar article is inspired by the Kevin Kelly TED talk: Predicting the next 5,000 days of the web and how that has caused me to reevaluate my attitude toward software design and construction.
The article also covers some of the techniques I've found involving Java platform web technologies. I had a much longer narrative in mind but, honestly, I've come to enjoy some of writing of the authors who are much more succinct. I'm trying to emulate that style in my work. So for me writing involves one long session of pouring out followed by a session of cutting down. So in the end half of what I write gets deleted.
In addition, I had worked out a few more techniques that I could not fit in for time concerns. Every example code listing and technique comes from real working code. Often I have to address the issues of clarity and code ownership so I don't always get to share everything that I know.
I would like to thank my editors who helped with this month's article. Thanks to both Darryl Bloch and Craig Wickesser your feedback was very helpful and I really appreciate it. You both helped me tighten up my writing style quite a bit.
Please head over to GroovyMag and pick up a copy! As always your feed back is appreciated and will only go toward making me a better author and developer. Thanks!
This particurlar article is inspired by the Kevin Kelly TED talk: Predicting the next 5,000 days of the web and how that has caused me to reevaluate my attitude toward software design and construction.
The article also covers some of the techniques I've found involving Java platform web technologies. I had a much longer narrative in mind but, honestly, I've come to enjoy some of writing of the authors who are much more succinct. I'm trying to emulate that style in my work. So for me writing involves one long session of pouring out followed by a session of cutting down. So in the end half of what I write gets deleted.
In addition, I had worked out a few more techniques that I could not fit in for time concerns. Every example code listing and technique comes from real working code. Often I have to address the issues of clarity and code ownership so I don't always get to share everything that I know.
I would like to thank my editors who helped with this month's article. Thanks to both Darryl Bloch and Craig Wickesser your feedback was very helpful and I really appreciate it. You both helped me tighten up my writing style quite a bit.
Please head over to GroovyMag and pick up a copy! As always your feed back is appreciated and will only go toward making me a better author and developer. Thanks!
2009-04-27
Grails 1.1, JBoss 4.2.x, and Oracle xmlparserv2.jar
If you want to use Grails 1.1 and the XMLType from oracle (the class is found in the Oracle jar xdb.jar) then you're going to need the Oracle xmlparserv2.jar in your Grails project. The problem is that Grails 1.1 will die if you just drop xdb.jar and xmlparserv2.jar into the lib directory of Grails. Why?
Well it turns out that this post tells you why. It was not cheap to find this answer at all... it cost me days of web surfing!
When you use JBoss in the $JBOSS_HOME/lib directory you'll find and endorsed sub-directory in there are two jars of particular interest to us. The xalan.jar and the xercesImpl.jar these provide the following services defined in their META-INF/services directories... xercesImpl.jar provides javax.xml.parsers.DocumentBuilderFactory and javax.xml.parsers.SAXParserFactory and the xalan.jar provides javax.xml.transform.TransformerFactory.
These three services are also supplied by the Oracle provided jar file xmlparserv2.jar and will effectively evict the JBoss provided implementations of these Java services. So what you have to do is replace the class references inside the xmlparserv2.jar with the same ones inside JBoss's endorsed libraries.
In short, explode xmlparserv2.jar then copy over the javax.xml.parsers.DocumentBuilderFactory and javax.xml.parsers.SAXParserFactory from xercesImpl.jar and the javax.xml.transform.TransformerFactory from the xalan.jar... re-jar the archive and redeploy.
Well it turns out that this post tells you why. It was not cheap to find this answer at all... it cost me days of web surfing!
When you use JBoss in the $JBOSS_HOME/lib directory you'll find and endorsed sub-directory in there are two jars of particular interest to us. The xalan.jar and the xercesImpl.jar these provide the following services defined in their META-INF/services directories... xercesImpl.jar provides javax.xml.parsers.DocumentBuilderFactory and javax.xml.parsers.SAXParserFactory and the xalan.jar provides javax.xml.transform.TransformerFactory.
These three services are also supplied by the Oracle provided jar file xmlparserv2.jar and will effectively evict the JBoss provided implementations of these Java services. So what you have to do is replace the class references inside the xmlparserv2.jar with the same ones inside JBoss's endorsed libraries.
In short, explode xmlparserv2.jar then copy over the javax.xml.parsers.DocumentBuilderFactory and javax.xml.parsers.SAXParserFactory from xercesImpl.jar and the javax.xml.transform.TransformerFactory from the xalan.jar... re-jar the archive and redeploy.
2009-01-08
Data validation teirs
Data is either valid or invalid right? Well, sort of. I'm working with medical data like the Blood CHEM-7 test. This data has the possibility of being valid and being abnormal. So there is this concept of valid capture data and data that is out of range.
For example here's the CHEM-7 data definition in English:
I might code this as an Object and write validators to make sure that the saved data is in range. My Groovy object for GORM might look like this:
... but I've got a problem. The very reason that we conduct the CHEM-7 blood test is to find irregularities. The constraints that I've expressed only detail the normal range of data for a CHEM-7... so what should I do?
If I make the table definition in the database very generic. Allowing the table to accept NULL values or numbers of any kind for each of these values then I can do this with a chem7 object:
... as long as the table underneath the Grails application can accept the data then I'll save my irregular chem7 test results. This would mean that anyone who was editing the chem7 would get a warning about how it was not a normal test result. Yet no matter how irregular the test was we could still save the results and log who entered them.
So imagine a "Forced Save" button next to the normal save button on your Grails application. Simple enough to write. But it allows us to use validation in a tiered fashion. Validation at the controller and validation at the database. The important thing to keep in mind is that the validation must be weakest at the database level.
For example here's the CHEM-7 data definition in English:
- BUN: 7 to 20 mg/dL
- CO2 (carbon dioxide): 20 to 29 mmol/L
- Creatinine: 0.8 to 1.4 mg/dL
- Glucose: 64 to 128 mg/dL
- Serum chloride: 101 to 111 mmol/L
- Serum potassium: 3.7 to 5.2 mEq/L
- Serum sodium: 136 to 144 mEq/L
I might code this as an Object and write validators to make sure that the saved data is in range. My Groovy object for GORM might look like this:
class Chem7 {
static belongsTo = [patient:Patient]
Integer bun
Integer co2
Float creatinine
Integer glucose
Integer serumChloride
Float serumPotassium
Integer serumSodium
static constraints = {
bun(nullable:false,range:7..20)
co2(nullable:false,range:20..29)
creatinine(nullable:false,range:0.8..1.4)
glucose(nullable:false,range:64..128)
serumChloride(nullable:false,range:101..111)
serumPotassium(nullable:false,range:3.7..5.2)
serumSodium(nullable:false,range:136..144)
}
}
... but I've got a problem. The very reason that we conduct the CHEM-7 blood test is to find irregularities. The constraints that I've expressed only detail the normal range of data for a CHEM-7... so what should I do?
If I make the table definition in the database very generic. Allowing the table to accept NULL values or numbers of any kind for each of these values then I can do this with a chem7 object:
chem7.save(validate:false)
... as long as the table underneath the Grails application can accept the data then I'll save my irregular chem7 test results. This would mean that anyone who was editing the chem7 would get a warning about how it was not a normal test result. Yet no matter how irregular the test was we could still save the results and log who entered them.
So imagine a "Forced Save" button next to the normal save button on your Grails application. Simple enough to write. But it allows us to use validation in a tiered fashion. Validation at the controller and validation at the database. The important thing to keep in mind is that the validation must be weakest at the database level.
Subscribe to:
Posts (Atom)