2006-09-25

Bit Plumber

It seems that most of my life is spent either waiting for, calling new, or building data transfers. I spend my Monday morning reviewing logs from the previous weekend's data transfers, I start a refresh of my development environment's database, and I work through last week's progress on the web services I created. This weeks project is to design a custom export of data from one database to an XML format and back into a different database. So much of this is repetitive and so much of it is the same from week to week I have to wonder if there isn't a group of tools to automate these tasks.

Let me introduce you to the wonderful world of ETL tools. These are tools that Extract, Transform, and Load data. In short they have the potential to replace an entire stack of SOA tools that currently are written painstakingly entity by entity by your talented Web Service programmers. They do all their by work by pragmatically by-passing the entire ORM process (that then demands you to Serialize the Objects to get XML) and skip directly to the XML.

I expect to see more applications built around these ideas as SOA catches on more. Why can't an ETL be used to generate Ajax messages or SOAP calls? After all aren't many SOAP calls and Ajax calls just loading up data from a database and transforming it into the right message format?