Mesosphere thinks outside the Kubernetes box

With new support for data services like Confluent and DataStax, DCOS and Mesos show how Kubernetes could be better without morphing into its competition.

What's the difference between Mesos and Kubernetes? It's a big question, especially since both are major movers: Mesos recently hit 1.0, and OpenStack is reworking itself around Kubernetes.

To Mesosphere, which makes the Mesos-powered DCOS for managing apps at scale, the difference is that while Kubernetes is about containers, Mesos and DCOS are about containers plus everything else that goes with them. This includes the data used by containerized apps, the services that manage that data, and the other functions that oversee and monitor all of the above.

Now, Mesosphere is announcing support for the Confluent Platform for data streaming management. It's built around the open source Apache Kafka project and the Apache Cassandra-based DataStax Enterprise.

Distributed database Cassandra and widely used message queuing system Kafka figure into the "services" part of the picture. The former is used as a repository for state across applications, while the latter supplies applications both with live streams of data and wiring for exchanging messages.

Mesosphere uses the term "container 2.0" to describe this bundling of data and functions. "Companies only start seeing real, impactful business improvements when they move beyond individual containers and start operating with higher-level abstractions," the company wrote in a blog post.

Parts of the container 2.0 concept were also in DCOS's original, precontainer pitch: applications in a datacenter ought to be addressed and managed as abstractions at scale. Take a given application, wrap it in a set of common APIs that allow it to be managed with simple commands, and leave the heavy lifting (setup, teardown, load balancing) to the infrastructure.

Mesos still has a strong container focus, but always with an eye toward the surrounding infrastructure and its management. Its 1.0 version added a generic container runtime with some sophisticated features like native support for apps that use GPU acceleration.

Kubernetes, by contrast, is more focused on containers as a standard unit. To that end, its original container support was essentially hard-wired to Docker. (It can now use interchangeable third-party container runtimes, but that work is also still in the early stages.)

The underlying emphasis for Mesos, as DCOS amplifies, is on running applications that have a lot of state -- that is, big-data apps. The GPU acceleration feature fits into that picture as well, as GPU-powered data analysis is fast becoming a commodity and not a luxury. Dealing with state and data, though, remains the big differentiator, if only because they have long been handled inelegantly by container frameworks.

Even if Kubernetes and Mesos (and its big brother DCOS) aim for markedly different targets, there's a lot Kubernetes can pick up from the competition, especially when it comes to handling stateful apps and services that manage state. Those tasks aren't tied to DCOS's low-level approach; if anything, Kubernetes's higher-level approach would benefit from them.

Some work in this vein already appears to be on the way. Earlier this year, Kubernetes 1.2 added features for app configuration and deployment. Its future road map also covers providing better support for application state.

It's unlikely Kubernetes will expand to the point where it'll become an entire DCOS-like solution. Kubernetes probably won't become more like Mesos, which is aimed at a far lower and broader set of goals than Kubernetes. But that doesn't mean the big K can't learn a few tricks.