Hacker News new | past | comments | ask | show | jobs | submit login
What Makes Containers at Scale So Difficult (theplatform.net)
26 points by orrsella on Oct 3, 2015 | hide | past | favorite | 11 comments



This is an advertorial for Mesos, which attacks some of these problems at a deep level.

It's (not yet) a full PaaS, though Mesosphere is driving there along with Docker and Hashicorp and everyone else who built parts of the solution and then realised there's not much profit in selling cogs.

I like Cloud Foundry -- which is a PaaS -- probably because I've worked on and seen how that sausage gets made. But there's also OpenShift. And Heroku, which people often forgot pioneered the whole space.

The bin packing problem is hard in theory, but at the scale we're talking about, it becomes less about the perfect fit and more about the fact that keeping up-to-date information is hard under realistic conditions. You can show linear scaling for short tasks. Wait until you're running in someone's whacky private DC.

Diego (the container scheduler in the next generation of Cloud Foundry) doesn't try to maintain a perfect view of state in order to do its job. It's basically impossible to really do so. Instead it satisfices by turning container placement into an auction, then periodically goes out to sync up with reality.

If you want to play with it, the best option is to install Lattice[0].

Standard disclaimer: I work for Pivotal Labs, a division of Pivotal, which is the largest donor of engineering effort to the Cloud Foundry Foundation. So I guess my comment is an advertorial too.

[0] http://lattice.cf/


I am not sysadmin, so, thinking out loud here: why not build software services in a way that does not require this large and sprawling infrastructure? The future could be simple binaries that run in user space and cache everything in simple data structures all in RAM. 64GB is cheap and will handle C1M easily.

I want my future server to abstract the hardware (memory, socket too), provide a few simple API calls, run my runtime (C++, rust, .NET, etc.), and then I'll build my application business logic to handle all my customers waiting on the other end of the socket.

Perhaps it the open source way where you have 40+ dependencies. I recently built my own webserver and found it very liberating.

Think outside the container.


The goal of Containers is to fully abstract the application. That can only go so far - the underlying infrastructure is still a huge part of the application. The responsiveness of a web application doesn't just involve efficient algorithms and storage calls, but also backend call latency, and transport time back to the user.

The infrastructure doesn't have to be large, but distributing parts will help the entire service stay online. Front-end nodes could be distributed to different datacenters, and back-end could also be mirrored or replicated between locations. With some kind of global load balancing, users can be routed to the nearest server for the fastest response.

The binaries have to run somewhere, either in the virtualized unit or its host. And Open Source or Proprietary, you're going to have dependency bundles - .NET, Java, etc. It might be more efficient to lump Containers by dependency so fewer of them need to be loaded by a Host.


64Gb is cheap.

So is 640Gb, compared to developer time.

Abandoning well-understood technologies is usually more expensive than just evolving what you have to run on a smarter substrate.

[Edited for confusing redundancy]


Generic containers are an awkward mid-way point between special-purpose containers (a Wordpress instance or a rails app on heroku) and an actual machine.

You get the hassle of maintaining your own instances, without the flexibility or well-defined performance characteristics of an actual box.

I just don't see the market.


The market you don't see is the market for PaaSes, which make the question of "how do I run this, anyway?" basically vanish.

Continers are the fundamental building block for modern PaaS design. There's a reason OpenShift 3 was built on top of Kubernetes, or why Cloud Foundry was built on top of a pre-Docker container system.


The one thing I can say for Docker is that before it launched, it seemed most companies using container technology were not that open about it. Docker appears to be the company that spoke about the unspeakable and caused lots of software to suddenly be developed in the open where it was all behind the scenes previously. Maybe I was just clueless, but the more Docker pushed their releases, the more other companies suddenly had all this code to release.


I think Docker deserves pretty much all the credit for making containers a mainstream concept. In particular, they tied it together the existing primitives into a uniform concept with a developer-friendly interface.

The very fact we call them "containers" is due to the analogy Docker made to shipping containers.


> The very fact we call them "containers" is due to the analogy Docker made to shipping containers.

I don't think so. I imagine it stems from the use of that term by Solaris starting in 2004:

https://en.wikipedia.org/wiki/Solaris_Containers

The evolution of containers on Linux was heavily influenced by Solaris. More so than it was by BSD, from what i remember.

Whether Solaris picked up the name from an even earlier use, i don't know.


Well there you go. I remember them being almost exclusively referred to as Solaris Zones, but some Googling shows that Solaris Containers was used interchangeably as early as 2005. You're right.

However, Docker did deliberately make the shipping container analogy -- the name, the logo and the early blogposts all consciously focused on the similarity with the invention of shipping containers. So I was half-right.


Because not enough memes and hype and buzzwords.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: