Scaling Docker containers in the real world

Question

I have a few basic questions on scaling Docker containers:

I have 5 different apps. They are not connected to each other. Before having containers I would run 1 app per

Answer 1

There are several options, but none that I know that does it all: you will need 2 things: autoscaling hosts according to signals, then autoscale containers on the hosts.

The following are the solutions to deploy and scale containers on the hosts (not necessarily auto-scale though):

Kubernetes is an orchestration tool which allows to schedule and (with the optional autoscaler) to autoscale pods (groups of containers) in the cluster. It makes sure your containers are running somewhere if a host fails. Google Container Engine (GKE) offers this as a service, however i am not sure they have the same functionalities to autoscale the number of VMs in the cluster as AWS does.

Mesos: somewhat similar to Kubernetes but not dedicated to running containers.

Docker Swarm: the Docker multi-host deployment solution, allows you control many hosts as if they were a single Docker host. I don't believe it has any kind of 'autoscaling' capability, and I don't believe it takes care of making sure pods are always running somewhere: it's basically docker for cluster.

[EDIT] Docker supports restarting failing containers with the restart=always option, also, as of Docker 1.11 Docker Swarm is a mode in Docker Daemon, and supports rescheduling containers on node failure: it will restart containers on a different node if a node is no longer available.

Docker 1.11+ is becoming a lot like Kubernetes in terms of functionalities. It has some nice features (like TLS between nodes by default), but still lacks things like static IPs and storage provisioning

None of these solutions will autoscale the number of hosts for you, but they can scale the number of containers on the hosts.

For autoscaling hosts, solutions are specific to your cloud provider, so these are dedicated solution. The key part for you is to integrate the two: AWS allows deployment of Kubernetes on CoreOS; I don't think they offer this as a service, so you need to deploy your own CoreOS cluster and Kubernetes.

Now my personal opinion (and disclaimer)

I have mostly used Kubernetes on GKE and bare-metal, as well as Swarm a about 6 months ago, and i run an infra with ~35 services on GKE:

Frankly, GKE with Kubernetes as a Service offers most of what you want, but it's not AWS. Scaling hosts is still a bit tricky and will require some work.

Setting up your own Kubernetes or Mesos on AWS or bare metal is very feasible, but there is quite a learning curve: it all depends if you really strongly feel about being on AWS and are willing to spend the time.

Swarm is probably the easiest to get working with, but more limited, however homebuilt script can well do the job core job: use AWS APIs to scale hosts, and Swarm to deploy. The availability guarantee though would require you monitoring and take care of re-launching containers if a node fails.

Other than that, there are also container hosting providers that may do the job for you:

• Scalingo is one i know of but there are others. https://scalingo.com/

• OVH Sail Above has this service in alpha. https://www.runabove.com/sailabove.xml

Answer 2

I would take a look at Tutum(Recently acquired by Docker actually). It ties into CI and I believe it has autoscaling capabilities.

https://www.tutum.co/

Answer 3

UPDATE: this is supported by AWS ECS with Task Placement Constraints.

1. Have your ECS cluster served by two auto-scaling groups (ASGs).

2. In the first ASG, set min, max and desired size all to 1.

   Tag this instance with a custom attribute ALLOW_ALL_APPS = TRUE. Do this in the user data script.

3. In the second ASG, set min and desired size to 0 and max size to 1 (I'm assuming you only want 2 instances).

   Tag this instance with custom attribute ALLOW_ALL_APPS = FALSE. Again in the user data script.

4. The scale up alarm for the second ASG will be determined by load on the first ASG.

   If you know when peak times are for app 3 you could bump it up preemptively with a scheduled scaling action.

5. Scaling down for the second ASG is when its load drops enough so that the first ASG can handle it on its own.

6. In the service definitions for apps 1, 2, 4 and 5 you would have placement constraints restricting them to run only on nodes where ALLOW_ALL_APPS = TRUE.

7. In the service definition for app 3 there are no placement constraints.

8. Service auto scaling is configured for all apps based on container or application metrics.