2014-05-30

pyvmomi-community-samples the tags and what they mean

This week on the pyVmomi Community Samples project I created a list of tasks to be accomplished.

The pyVmomi library itself is a very bare binding onto the vSphere Management API. This is by design since the library has to match very closely to the API which only changes with each vSphere release. That means we need to be careful to only alter that library when vSphere gets altered. Breaking out the samples to its own project frees the software to change at a much more dynamic rate and opens up the possibility that we can have a range of collaborators outside VMware contribute to those samples.

Since I've heard a lot of folks are wanting to help out but don't know how, I've created a set of issues marked 'help wanted' at any point if you want to help and don't know how... pick up a task marked 'help wanted' and comment on there that you're working the issue. Once you have a pull request ready reference the issue number in the comment.

Some issues as I notice they are being worked on will get marked 'in progress' to let you know that the issue is taking a bit to work on. I'll be tagging issues with other special tags to make lists to send to VMware staff internally in order to solicit help and support from folks working inside VMware that may not normally spot issues in public.

Currently I happen to be working on issue 40 and it's opened a whole different can of worms. The samples currently have a tools package. This tools package is the first stop on our way to identifying useful tools for general pyvmomi development.

More on that next week...

2014-05-23

How pyVmomi is something different

I'm largely responsible for the current set of Java samples vSphere WS Management SDK released with vSphere 5.5 last fall.  It was really cool for me, for the first time in a 20 some-odd year career to actually be involved in a shrink-wrapped product roll out. That may be out of fashion these days but it was still cool.

This was also an unusual project for me mainly because I couldn't really talk about it at the time. I've worked most of my career on websites, COTS or GOTS modifications, or OpenSource customizations and that implicitly entails some freedom to talk about things that closed source projects don't. The original plan was that I was going to start writing about vSphere 5.5 once it rolled out, but then OpenStack happened... (more on that in coming weeks).

My job at VMware wasn't just to write samples but also to provide a link between the API developers inside VMware and developers outside VMware. Since the Java SDK was my first big job with the company I did things the way that people at the company were used to, I followed the normal channels. I really wanted to be a good VMware citizen first before I started rocking the proverbial boat.

 If you've seen the vSphere Management SDK for vSphere 5.5 then you'll notice I took it in a very JPA-like direction. I wanted the SDK samples to feel like working with any other high level Java code. The theory was that in tools like JPA magic annotations take care of wiring up an Object to a Table. The effect is that the Java programmer gets to focus on data modeling and less on details that don't pertain to their application. I tried to emulate that with the samples... magic annotations wire the POJO getters and setters up to the Command Line Interface (CLI).

The major changes involving the annotations and the framework that supported them were done in just three weeks. However, testing, verifying, and repairing the samples took the rest of nine months with a team of testers. That second part was long, labor intensive, and frankly not very satisfying.

Unfortunately, because this was all closed-source and behind the scenes development I couldn't get feedback from the most critical audience that these changes affected... you the developer outside VMware. Inside VMware these changes have been well received and I gave the new Java SDK maintainer explicit permission to completely nuke the vSphere 5.5 Java samples, annotations, and framework when I shifted to OpenStack work in January 2013. The current maintainer has told me he likes the feel of the new samples and I hope VMware will eventually open source the annotation framework that takes CLI options and wires them to the POJO. The vSphere samples that arrive on the next release will be managed by a new developer and I'm not involved with that new framework anymore.

As I mentioned in early 2013 I started to transition to OpenStack Nova work full time. Again, my typical strategy is to observe a project for one cycle before rocking the proverbial boat. I mostly watched the the Havana release and I more actively participated in the Icehouse release. In all this I felt there were some real problems with how the vSphere Management SDK was getting used. Some of these problems were related directly to how people understood the SDK and others were directly related to the SDK itself.

I asked that we either bring in someone full time to work the pyVmomi library, its community, and its problems or that I be moved to the pyVmomi project full time. Guess which happened?

As of this April, I took over the public facing parts of the pyVmomi project. Part of that has been not rocking the boat for a bit as I observe how the sausage gets made. The other part of has been finally rocking the boat in regards to how samples are made. With pyVmomi I've been asked to write samples again but this time it's different.

The project pyVmomi Community Samples is a really radically different way to do SDK samples for VMware. The project is an Apache 2 Open Source samples project and we're getting samples from non-VMware contributors. This new project structure means that you help write which samples you want to see, you help write them the way you want to see them, and you can use the awesome power of github to send pull requests. The samples become a durable, sharable, and living community effort.

I've tentatively slated a new release of pyVmomi should come out every 6 months. That means releases every December and July. That does mean a bit of boat rocking early on for me... but then... if you'll help me out I think we can get this ship sailing in the right direction.

More next week...

2014-05-16

Let's start over again...

After many adventures and troubles I find myself here again. Let's try this one more time. From the top.

I'm Shawn Hartsock and I happen to write software for a living. I don't specialize. I'm a generalist. I've worked on the Linux Kernel and written Kernel modules (but I wasn't allowed to contribute these to the OSS community at large this is something I hope to fix one day), I've architected web sites that scaled to millions of users and conducted billions of dollars of revenue annually. I've invented new API and lectured at multiple Universities. I've worked with Knowledge Bases and Databases, rules engines and expert systems. I've done 3D rendering software on super computers and embedded applications for phones. I've even worked with weapons systems and simulators for them.

Lately I've been focused on this thing we call "The Cloud" and my mercenary adventures have very recently made me the care taker of the pyVmomi library. That means for the first time since 2011 I'll be working on publicly facing Open Source code again.

Each week I'll post on what we've been working on with pyVmomi, how that's gone, and what that might mean. Each step of the way, I'll talk to you here candidly. There may be entries from official blogs, or comments on official channels, but this is where you'll talk to me... the architect, developer, and author... unfiltered.

So, let's start again.