This week in pyVmomi & rbVmomi

pyVmomi

Not much motion this week. I've spent the majority of the week focused on coming up to speed on rbVmomi. Of note we did have a bug report of sockets getting left open and I'll be spending quality time debugging that next week.

• The next major revision of pyVmomi is slipped to December now in light of my new work commitments. I will keep github up to date on how these projects are going.
• Disconnect doesn't close socket is this a pervasive bug or a quirk? TBD I'll carve time from my schedule next week to attempt to reproduce and resolve the bug as either a quirk or create a fix. The fix will go in 2014.2 at minimum but might be bad enough to cut 2014.1.2 if needed.
• We need some more education and evolution on tasks. I've shelved my work on pyvmomi-tools temporarily while I come up to speed on rbVmomi. But, tasks are one of those areas I feel the raw API is too rich for new developers to grasp right away. Some syntax sugar should help.
• Currently stalled on my desktop is an effort to introduce an SMS "plugin" (plugin isn't the right word but it's close) which I wanted to introduce through pyvmomi-tools. I'll ask for review inside VMware and from key thought leaders in the python and pyVmomi communities before shipping it to pypi ... more on this as we have time.
• We changed version conventions yet again. On pyVmomi we wanted the version number to communicate freshness and compatibility. So the year-dot-quarter method was used. Then releases moved to every six months so it's 1st half and 2nd half now. Then we found out `-` means something in pypi so now it's just a `.` character. Finally we have 5.5.0.2014.1 and 5.5.0.2014.1.1 to indicate 1st half of 2014 bug fix 1 is compatible with vSphere 5.5.0. Part of this is we still don't know precisely how pyVmomi github and pyVmomi vSphere will interact on a code-base level going forward. That's TBD be the vSphere teams in charge of that. The admittedly complex interaction between these two things 

rbVmomi

Just coming up to speed on the modern state of Ruby gem management since it's been a number of years since I've had to work with them. I'm also moving cautiously since there's a standing development practice and tradition in place that I need to be mindful of.

• A prerelease for rbVmomi 1.8.2 is available on rubygems now please test!
• A regular release for 1.8.2 will happen October 1st barring any negative testing results
• A 1.8.3 release is tentatively set to got to pre-release October 17th (or so) depending on how well we can manage the process of upstreaming pull requests and this is very tentative. Note: this is a change I had intended to hold the release 2 weeks for myself to gain familiarity first. Instead I like this pre-release, delay for 3rd party testing/review, then release strategy and I'll likely adopt it as my normal operating style from now on.
• Long term planning will have to happen for a 1.9.1 release but I'll not comment much on that until I know what my long term relationship to the project looks like.
• I opened a low priority research ticket for [higher performance XML](https://github.com/vmware/rbvmomi/issues/57) which seems to be a minor scalability problem rbVmomi faces. My initial research is in the ticket. It's a help wanted item so feedback is much appreciated.
• NOTE: any item I tag 'help wanted' is an area I feel a 3rd party can either easily help with (usually also tagged 'low hanging fruit') or an area we can't really move on without outside help.
• NOTE: any item I tag 'VMware staff assistance needed' is something that I feel a 3rd party should probably avoid working on since the issue is fuzzy and bleeds into VMware internal technical details, decisions, or other issues muddy the ticket such that it's not going to move without someone (usually me) bridging the Open Source and VMware internal engineering communities.

General Status

My Friday posts will probably take this two part pyvmomi then rbvmomi flavor as long as I'm bridging both communities. Sorry if that confuses things but that's a function of Conway's law and at the moment the Open Source to VMware and back communications conduit is basically me for both projects so you have to expect a bit of that.

Thanks for your support! The following freenode IRC channels will at least have my IRC 'bot in them #pyvmomi , #pyvmomi-dev , #rbvmomi , #rbvmomi-dev , #govmomi , #govmomi-dev and I'll eventually see IRC messages in those channels through my 'bots. The -dev channels are for advanced programming topics while the other channels will be open for newblings and will need community support since I'm pretty bad at dealing with neophyte level problems these days.

More on each of these threads as progress is made...

On the topic of rbVmomi

Earlier this week I started helping out on the rbVmomi project. My involvement is just to help take some pressure off of Christian Dickmann around reviewing and releasing the next version of rbVmomi.

Compared with pyVmomi, the rbVmomi project has a longer history in Open Source. While pyVmomi has existed longer, rbVmomi has the benefit of being designed from the ground up for the type of collaborative effort an API binding like this can represent. I'll want to be cautious and observant of that established project workflow and primarily act as an accelerant for the processes already there.

To address the initial problem of there being no release this year, I've begun drafting v1.8.2 which should be ready for release in a few days. Having learned from pyVmomi, I've set up a follow up release v1.8.3 to try and catch anything we miss in the earlier release. The follow up release should hit in mid-October if we need it. If it turns out we don't or can't hit mid-October I'll drop the milestone and we'll move to long-term planning. Future release schedules are TBD.

For pyVmomi, this means my attentions are divided and I've shifted out milestones for that project to match my expectations. The second release of 2014 for pyVmomi is now tentatively set for mid-December. (I hope that pyVmomi reaches feature parity with rbVmomi by that release.) Our big additions for the next release of pyVmomi should be SMS and SPBM API support provided we don't hit any hidden issues. We'll hopefully have time for EAM as well.

My role on rbVmomi at this time is just to assist in the current release plan. Christian is very much still the lead developer. I'll be using tags on the open issues and pull requests to help decide what to call out for Christian's direct attention. There's also a bit of process for this project that I'll need to work out on the fly.

More on these topics as things happen...

How to use Conway's Law for good and not evil

If you've been paying attention to our github repository you'll have noticed some noise around the following API additions to pyVmomi...

• SMS
• SPBM
• EAM

These are all presently scheduled for the next major release of pyVmomi which is currently set for December of 2014 and will be called 5.5.0-2014.2 barring any major issues.

I'll be working with the various feature teams inside VMware to figure out how best to Open Source their existing API bindings. More importantly, I'll be work with these teams to find a way that allows the effect described in Conway's Law to function beneficially on this set of projects.

In other words, pyVmomi isn't one large project with one BDFL directing the entire API. The reality is that the APIs potentially exposed by pyVmomi are in fact the result of multiple collaborating teams. Each of these teams will tend to be their own unique snowflake.

Couple this with the fact that we will also have the opportunity to leverage contributions from interested third parties, vendors, and partners and you can see that the previous monolithic structure might not survive the strain. I'll be taking some vital time to implement at least one or more extension strategies for the library and have them evaluated. This will probably cost 3 or more weeks worth of time but it will be worth the pain because these are long-lived projects with minimum life-spans on the order of 3 or more years.

As part of this research I've come back time and again to a talk by Doug Hellmann titled Dynamic Code Patterns.

Not to diminish Doug's work at all, but his Stevedore project is an implementation of the Extensibility Pattern which is itself part of the previous generation's Software Design Patterns movement. The design patterns technique came under understandable criticism for not really contributing much to programming and more than occasionally over complicating things. It's these concerns about over complication that are going to slow me down a bit. After all, making things simple is not easy and my goal is to make things as simple as possible and no simpler.

The Stevedore project does offer something to a library like pyVmomi. It is a tool to create extensions. The term 'plugin' is not really accurate when we're talking about modules like SMS. The SMS module is neither a driver, nor is it a hook, but it is an extension. These are new classes and methods that provide new capabilities from the primary facade of the pyVmomi library itself. You can call these plugins but not in the normal sense.

There is also danger in approaching the problem of extending into these new API as a plugin. It quite possibly introduces the problem of turning pyVmomi into a framework and such a transformation would be wholly inappropriate. In particular, I am considering the use case for pyVmomi as a helper library to deliver a driver for SaltStack. What happens to the outer project if pyVmomi itself becomes a framework and does this make those consumer projects unnecessarily more complex?

We want extensibility but not at the cost of adding the need to be aware of more details. We want the library to appear as a unit even though it is actually composed of multiple sub-units. That's something that requires some subtlety. It's not something to bother with if we don't need it so... why do I think we need it?

Over the next 2 weeks I'll be experimenting with SMS support provided in a couple of different ways in the hopes of finding a sustainable, flexible, and robust mechanism to use Conway's law as a means of enforcing a structure of collaboration between teams. In particular there's a set of existing organizational and structural relationships that I feel the current design violates..

In particular, at VMware, I do not personally manage, dictate, or control what the structure or design a given ... feature interface (or sub-API) looks like. It's impractical to think I could. My role is to act as a bridge between VMware and the developer communities interested in building software on-top of the VMware platforms.

Some details about the practical nature of how modules like SMS, SPBM, and EAM for use in pyVmomi are produced directly informs our library design considerations. For example...

• Feature teams (and their management)
• design, maintain, and release their own API
• dictate what API are official, unofficial, and deprecated
• have their own schedules, priorities, and deadlines
• are fundamentally disinterested in low-level concerns

Knowing this, any design I create for pyVmomi that dictates that a feature team must talk to me first and then convince me to do things a certain way are very likely to fail. Why? Well, let's take the opposite approach as a thought experiment and see what happens.

If I were to decide that I was going to tightly control pyVmomi and force feature teams to first get approval for API from me before I clicked my big "approved" button and there by uploading their API to github that would mean I could conceivably derail their priorities or deadlines. 

What do you think would happen? I could cause a vital product or feature to slip it's release; I could force an unnecessary conflict between managers; I could find my project usurped by something the feature team felt could disentangle themselves from my meddling in their affairs. Any or all these and other scenarios could happen. In short, I would create an unnecessary friction and unintentional power-struggle as people tried to focus on issues wholly unrelated to something as trivial as providing API to people like yourself.

So, if I want to instead create a successful project in this environment I have to engineer my software so that it can function with the natural (and proper) social boundaries already present. The structure of the code itself will also influence how social interactions occur around the code base. And finally, the modularity of the code will allow me to potentially delegate power, authority, and autonomy to other people.

So what are our design concerns?

• For feature creators adding to pyVmomi we should,
• leverage existing library features
• hide low-level concerns
• allow independent ownership
• simplify the process of creating a binding as much as possible
• For integrating developers working with pyVmomi in their separate projects we should,
• present a single unified "surface" for them to use
• hide accidental complexity but expose essential complexity
• follow a rule of least surprise

I admittedly may think of more as I work through the problems but this captures what we're after in a nutshell. This library in the world of Conway's Law represents a bridge between multiple parties and it's structure will end up reflecting how these groups relate. The pyVmomi software sitting between these groups will be least painful to work with for everyone if it can be molded to it present reality.

After we tackle Open Sourcing these three new API I'll have a better picture of what that reality is. And, that's what large-scale software development is really about, not as much computers and code so much as people, relationships, and communication. Our tools affect our lives and better tools make better lives.

More on this another time...

Developer Community Engagement

If you've not been following along at home, pyVmomi was run in a different manner from most VMware Open Source projects. It's been a bit of a social experiment. The last two weeks since our release, I've been working on distilling lessons learned from the past five months of the project.

I did not plan on also looking into rbVmomi ... but ... at the same time just after VMworld a certain blog post started making the rounds on social media. It's clearly an opportunity to examine what we're doing at VMware around OpenSource development projects.

rbVmomi is a more typical VMware fling project. These start as a developer driven POC project and these are developed on a best-effort basis. The rbVmomi project has closed 6 issues and closed 10 pull requests during its entire lifetime as a VMware project.

pyVmomi has benefited from having my attention full time since April/May of 2014. The total number of issues closed to date is 59 with a total number of 70 merges. These differences in numbers shouldn't be surprising, that's to be expected when you go from free time development to full time development. My personal stats have become quite impressive due to the full-time activity on GitHub.

That's all nice for me, but, what does that really mean for the library? What does that mean for developer use, experience, and over-all for VMware? It's a matter of audience and SDK adoption.

Over on stackoverflow, you can see that in its entire life-span rbVmomi has had 9 questions asked as of this writing. That indicates 9 times people who are likely to seek help on stackoverflow have sought help and those 9 times only 4 of those questions got answers.

Taking a look at the same search for pyVmomi yields 24 questions over a much shorter life-span. 

13 of these questions have been answered and 19 of times people voted on the questions where as with rbVmomi no one voted. If you take a closer look, 17 of those questions occur after my full time commitment to the project. In the shorter time-frame my public commitment and effort to the library has helped increase developer engagement with the library improve by an order of magnitude.

The next question is... is this effort worth it? And how do we determine that? 

I'm open to suggestions. What else should I be looking at?

This week: Retrospectives

This was a short week, so I would like to take a moment and reflect on where we've been since May.

This week I've been getting ready for a creating a set of new spikes around a few of our unanswered sample requests. Out of these will come my development plan is to create new features to include in pyvmomi-tools. By the way, if you're new to the projects or haven't been following along, part of the reason for breaking out pyvmomi from pyvmomi-tools is to allow us to develop different release cycles and development standards and styles. It would allow a quicker-turn around on some items and also allow some experimentation in pyvmomi-tools that we might not want to risk in the core pyvmomi library.

This is all part of a social experiment I'm conducting trying to help VMware at large find the best way to engage with the open source communities. Earlier in the week someone called out this negative reaction in the rbVmomi community. Ironically, rbVmomi is much older, more robust, and feature complete than pyVmomi but the community reception to pyVmomi has been much warmer in recent weeks.

I'm getting reports of a number of python projects shifting from other vSphere API bindings to pyVmomi as the confidence in the library grows. I've been getting almost nothing but positive feedback from users of the library and I think we're well on our way to becoming the best way for integrators to work with vSphere. And, we're reaching that goal by encouraging better developer practice.

From some IRC chats earlier in the week I found out that at least one shop has traded from vijava to pyVmomi running in jython. I've not tested or verified jython for use with pyVmomi but I'm encouraged to hear the work is progressing well. If anyone else is attempting this kind of work I would appreciate hearing and/or reading about the experience.

I also find this language switch curious because vijava is actually quite well done. As a side note I am doubly curious because of my own past involvement in alternative languages for the JVM. I've not attempted jython with Java hybrid projects before and I'm curious as to how and why these fit together.

 I'll be presenting these stories to VMware teams to help build the case for this style of community engagement and I plan on having a series of discussions around rbVmomi as well. As I've mentioned multiple times in this blog, merely building pyVmomi up is only one of the objectives toward my much larger goal of helping change the way we write software for the cloud.

Part of that larger goal is testing, process, and engagement.

More on this next time...