OK Labs Story (8): Competitors and Markets
Early on we found that we were competing with two other players: Jaluna, later renamed VirtualLogix, and Trango, both from France (although Jaluna was headquartered in California).
Jaluna goes back to the early microkernel days: INRIA-spinout Chorus Systèmes, who prided themselves of having the first commercial microkernel (in the 1980s). Like other microkernels of the first generation, its performance sucked and it was practically unusable. They early on virtualised Unix on Chorus, but due to poor performance moved the Unix kernel into the Chorus kernel (like these days BSD is co-located with Mach in the Mac OS X kernel), which is no-longer virtualisation. Liedtke’s original L4 (in 1993) out-performed Chorus by a factor of ten!
Chorus had some deployments in the network-infrastructure market, and were eventually bought by Sun, but later closed down, and bought out by the original founders as Jaluna. 20-30 years after the original Chorus, their technology was still essentially Chorus, they don’t seem to have learnt much. They were “virtualising” Linux now, still by putting it into the kernel, something I call pseudo-virtualisation. In an earlier blog I joked about them taking my Advanced OS course so they could learn how to build performant microkernels… At some time they changed their name to VirtualLogix.
Their most noteworthy “achievement” was to demonstrate their loserdom with an incredibly shoddy paper published in the IEEE CCNC conference. They “showed” that their performance was way better than ours, which I knew not to be true (plenty of potential customers did their own evaluation showing this). I had fun in a series of blogs ripping that paper to pieces, and students in my Advanced OS course did similar. That paper was so full of fairly obvious defects, it should never have passed peer review. Needless to say, the conference is on my blacklist.
What almost certainly happened with their “performance” measurements is that they took our open-source version as is and built it blindly. The release had debugging and profiling enabled by default, which, of course is a performance killer. Any ethical comparative benchmarking would have disabled that.
VirtualLogix were later bought by Israeli device-management-software provider RedBend.
The other was Trango Virtual Processors, founded by a bunch of ex-ST Microelectronics employees. Trango was a hypervisor for ARM (and MIPS) processors. Their performance was much better, but bought with a 100% assembler implementation! This is good for performance, but implies huge costs for maintenance and adapting to changing conditions. And, while their founders might have been good hackers, they weren’t technology leaders the way we were.
Trango was later bought by VMware. I never understood why, and I don’t think VMware did. In fact, I believe they soon realised they bought a lemon. They marketed the Trango technology for a while as MVP (mobile virtualisation platform) but then went quiet, and about a year later came out with a new product under the same name, which shared nothing (probably not a line of code) with the original. It was essentially turning the Android Linux kernel into a hypervisor (KVM-style), to support running a second OS in a virtual machine. This was targeted to the “BYOD” business market, where enterprises would install a VM with the business (logical) phone on an employee’s handset. The problem is that this adds very little to security over just using the vanilla phone for accessing the enterprise IT infrastructure: As the hypervisor is the Android OS of the private (logical) phone, compromising it will automatically compromise the VM in which the business phone runs. They used encryption of data at rest to mitigate this, but that’s not really much more than window dressing.
Neither VirtualLogix nor Trango ever were a real threat: We never lost a deal we were directly competing with either of them (they may have won some we didn’t know about).
And we had many opportunities. Basically none were created by our (overpaid) sales force (except for Motorola deals generated by Tony/Josh). Instead, people were approaching us, from across a range of industry sectors. Some of them, especially automotive, appeared to me as great opportunities from the beginning. Of course, a startup has to focus, and Steve decided to focus on mobile. Why? There were no technical reasons, no real analysis of our value add, it was a simple calculation of >1 billion devices sold worldwide each year, if we can get on 20% of them and get $1 per unit, we’re rich. There was never any justification of why these figures were realistic, but Steve told the investors what they wanted to hear, and they lacked the insight to realise it was all bullshit.
For a while almost the whole marketing of our competitor VirtualLogix was directed at “single-core feature phones”, i.e. running a simple application OS and the modem stack on the same core. It was obvious that this would be, if at all, a very short-lived niche, given that the incremental cost of an additional core trended towards negative. In fact, I joked internally that if I was them and it’s all I had, I’d be very worried. Imagine my dismay when, about a year later, our marketing homed in on exactly that use case! Except we called it “mass-market smartphone”. The “mass-market” bit was to refer to the simple ARM9-based hardware of the then feature-phones, and the “smartphone” on the ability to run a smartphone OS (Android).
The thought that smartphone apps (even ignoring OS overheads) require grunty processors, and that it made no sense to share those with the baseband, wasn’t understood in Chicago (and, as I mentioned earlier, critical thinking was treason).
In this context I did a study of industry sectors, the role virtualisation could play in them, and attempted to quantify the value-add. I presented this at a phone conference of board members in October 2009. The conclusions I drew was that I could not see a significant value-add for our technology in mobile, no killer apps, and a low bar for entry for competitors, resulting in very low margins. In addition, VMware had it in their hands to reduce these margins to zero: royalties from the mobile hypervisor could hardly be central to their business model, so they could afford giving it away for free, and thus eliminating our margins. And soon a new competitor appeared: an approach from Columbia University to virtualise Android at the OS ABI level, which has the advantage of simplicity (the paper was published at SOSP in October ’11 and I had in fact reviewed and shepherded it, so I knew all about it). They created a startup going after the same market.
In my analysis, I concluded that I could not see the total addressable market exceeding $100M/a in this space, and that would be shared between many competitors with shrinking margins. Not an attractive place to be.
Instead my analysis identified two promising areas: automotive and strong security.
At the time we actually had traction in automotive: several component manufacturers had approached us. Despite Steve’s best attempts at chasing them away, some were highly persistent. I was particularly excited about one potential partner: OpenSynergy, a startup in Germany that came out of the research lab of a major car manufacturer. They had approached us and I had visited them as far back as July ’07. They started developing an automotive virtualisation product on top of our open-source microkernel and tried to negotiate a partnership agreement with us. Steve’s idea of “partnership”, however, was indistinguishable from “customer”: you pay and we deliver something to you (and that after long and painful negotiations Steve-style). They were perfect as a real partner: they had all the domain knowledge as well as excellent networks in the industry, both of which we were lacking.
Steve stuffed them around for two full years until they finally decided to go with someone else: German Sysgo (now bought by Thales) who had an aerospace (DO178-B) certified L4 clone named PikeOS. They would hardly have been happy with that marriage: I knew that PikeOS’s performance was miles away from ours, and we also knew from other engagements how performance-sensitive the automotive use cases were. Nevertheless, we managed to create a competitor out of nothing in a space we could (and should) have owned! Clearly this takes a special kind of talent.
Actually, in my “bullshit” mail folder I still have the mail Steve sent around OK Labs when OpenSynergy’s partnership with Sysgo was announced. He mailed around their press release with the subject line “1 Loser + 1 Loser makes for the Biggest Loser!” The conclusion was correct (although not as intended): the Biggest Loser was us, and that was immediately clear to me.
Our other engagement in the automotive space was with a company called ADIT, which is a joint venture of the two biggest automotive suppliers Bosch (Germany) and Denzo (Japan). Their German arm was developing integrated infotainment head units for Bosch, and they needed virtualisation to run automotive real-time components (with very strict real-time and performance requirements) concurrently with Linux-based infotainment stuff. Again, OK head office stuffed them around, apparently trying to make them feel that we’d be doing them a huge favour by engaging. Even when we had a development contract, they were not taken seriously, and insufficient resources allocated.
I still vividly remember one of the few customer meetings I had at that time. I was in Germany for CeBIT representing NICTA, and Abi thought I could really help rescuing the ADIT project, so I joined him for a half-day meeting in Hildesheim. The meeting superficially went fine, but during a break, ADIT’s Chief Purchaser took me aside and gave me (in German) the most serious dress-down I had experienced in my business career, calling OK Labs “the most unreliable outfit he’s ever dealt with”. This hurt deeply, because on the one hand I could see he was right, but on the other hand I knew that Abi (the technical sales guy whose project it was) as well as our engineers were doing their very best. In fact, ADIT engineers held ours in high regard (a recurring pattern). It was just head office fucking them around endlessly. Fortunately, things improved a lot after that meeting, and the project was, in the end, completed on schedule and to spec, and ADIT was happy with the outcome.
In automotive I could clearly see the value-add, and, while total numbers were less than in mobile, the addressable market looks way bigger. (Also, there’s a place for more than one hypervisor per car, which changes numbers significantly.) So I recommended making this area a main focus (despite having already missed the chance to own this market outright).
The second attractive space was strong security. Not the soft notion of “security” enterprises seem to be content with, where there are lots of simple “good-enough” approaches, such as VMware’s MVP, but the more paranoid groups like the national-security sector. This is where strong isolation, as we provided it, would matter. A typical use case (but there are others) would be to turn a more-or-less COTS phone into a secure communication device, a secure (logical) phone running on the same hardware as a soldier’s or emergency responder’s personal phone.
While this is a market with established suppliers (Green Hills, Wind River, LynuxWorks) I knew from what I could gather about their technology (as well as learning about evaluations from customers) that their hypervisors were unable to deliver the performance required, whereas we could. Obviously, in this space the number of sold units is much smaller than automotive (leave alone consumer mobile), however margins are much bigger. And we also had a number of exploratory projects, so there was clearly interest. Also, this was the space for which NICTA’s seL4 was made and would give us an unmatchable advantage, and a close collaboration with NICTA was core to my strategy.
Hence, my conclusions for the OK strategy were clear: avoid being trapped in a market with dubious value-add and shrinking margins (consumer mobile) and focus on the two domains where I could see an addressable market of significant size and we had significant competitive advantage: automotive and security.
© 2014 by Gernot Heiser. All rights reserved.
Permission granted for verbatim reproduction, provided the reproduction is of the complete, unmodified text, is not made for commercial gain, and this copyright note is included in full. Fair-use abstracting permitted.
From → Open Kernel Labs