Docker1.12 on AWS : 10 command(ment)s to manage Swarm cluster
Introduction
In July 2016, Docker Inc. released 1.12 version with many great features including Swarm Mode ( built-in orchestration ). Latest Swarm Mode along with docker-machine tool greatly simplifies the the job of creating/managing a cluster of docker containers on different cloud providers. In this blog post, I am going to explain how to create/manage a 3 node Swarm Cluster on AWS using docker-machine.
Note : These steps are only applicable to docker 1.12 and above !
Pre-requisite Steps
a] Install docker-machine on your Linux machine or Mac.
b] Create AWS account ( If don't have already ) and configure AWS credentials for docker machine. See details here.
c] In AWS EC2 console, create a security group called "docker-machine". Add following inbound policies to the security group. Following ports are required for swarm cluster communication. See this video for the details
b] Create AWS account ( If don't have already ) and configure AWS credentials for docker machine. See details here.
c] In AWS EC2 console, create a security group called "docker-machine". Add following inbound policies to the security group. Following ports are required for swarm cluster communication. See this video for the details
Once we have all the requisites in place, we are ready to deploy Swarm cluster on AWS using our 10 command (ment )s.
1. You shall use 'docker-machine create' to bring up new VMs
This command creates EC2 instance and installs docker 1.12 on it.
docker-machine create --driver amazonec2 swarm-manager
docker-machine create --driver amazonec2 swarm-worker-1
docker-machine create --driver amazonec2 swarm-worker-2
Verify the status of the machines using
docker-machine ls
2. You shall use 'docker swarm init' to configure Swarm manager
This command initializes Swarm manager on the VM. To use this, first get IP address of swarm manager machine followed by swarm init command.
MANAGER_IP=`docker-machine inspect --format '{{ .Driver.PrivateIPAddress}}' swarm-manager`
docker-machine ssh swarm-manager sudo docker swarm init --advertise-addr "$MANAGER_IP":2377
3. You shall use 'docker swarm join' to configure Swarm workers.
Swarm worker nodes (worker-1, worker-2) can join swarm cluster using this command.
First get the token from swarm manager.
Join worker-1
Swarm worker nodes (worker-1, worker-2) can join swarm cluster using this command.
First get the token from swarm manager.
TOKEN=`docker-machine ssh swarm-manager sudo docker swarm join-token -q worker`
Join worker-1
docker-machine ssh swarm-worker-1 sudo docker swarm join --token "$TOKEN" "$MANAGER_IP":2377
Join worker-2
docker-machine ssh swarm-worker-2 sudo docker swarm join --token "$TOKEN" "$MANAGER_IP":2377
4. You shall use 'node ls' & 'node inspect' to verify status of Swarm cluster
5. You shall use 'docker service create' to deploy service on Swarm cluster
Service create command deploys one more containers on the Swarm cluster.
docker-machine ssh swarm-manager sudo docker service create --name myredis --replicas=1 redis:3.0.6
6. You shall use 'service ps' and 'service inspect' to verify status of deployed service
7. You shall use 'docker service scale' to scale up/down the service
8. You shall use 'docker service update' for rolling updates
E.g. update redis image to version 3.0.7.
9. You shall use 'docker service rm' to undeploy service
10. You shall use 'docker-machine stop' and 'docker-machine rm' to delete VMs
Stop/Delete EC2 instances using stop and rm command.
Get status of Swarm cluster by running this command on Swarm Manager.
docker-machine ssh swarm-manager sudo docker node ls
User Inspect command to get status of both manager and worker.
docker-machine ssh swarm-manager sudo docker node inspect swarm-worker-1
docker-machine ssh swarm-manager sudo docker node inspect swarm-manager
Service create command deploys one more containers on the Swarm cluster.
docker-machine ssh swarm-manager sudo docker service create --name myredis --replicas=1 redis:3.0.6
6. You shall use 'service ps' and 'service inspect' to verify status of deployed service
docker-machine ssh swarm-manager sudo docker service ps myredis
docker-machine ssh swarm-manager sudo docker service inspect myredis
docker-machine ssh swarm-manager sudo docker service scale myredis=2
8. You shall use 'docker service update' for rolling updates
E.g. update redis image to version 3.0.7.
docker-machine ssh swarm-manager sudo docker service update --image redis:3.0.7 myredis
9. You shall use 'docker service rm' to undeploy service
docker-machine ssh swarm-manager sudo docker service rm myredis
10. You shall use 'docker-machine stop' and 'docker-machine rm' to delete VMs
Stop/Delete EC2 instances using stop and rm command.
docker-machine stop swarm-worker-1
docker-machine stop swarm-worker-2
docker-machine stop swarm-manager
docker-machine rm swarm-worker-1
docker-machine rm swarm-worker-2
docker-machine rm swarm-manager
References:
1. Docker Swarm Mode : https://docs.docker.com/engine/swarm/
2. Docker machine : https://github.com/docker/machine
Keep Calm, Use COM - A New Approach to Build Next-Gen Cloud Applications
What is COM
COM stands for Container, Open source and Microservices. In late 1990s, .com revolution transformed the way computing was done. Today, containers and microservices are transforming the way cloud computing is done. Microservices architecture is not just an IT buzzword anymore. Many companies have successfully adapted microservices architecture to develop highly scalable cloud applications. Cloud providers like Amazon, Microsoft, Google, etc. are building an ecosystem to facilitate application development using containers and microservices. No wonder open-source community played a significant role in this revolution. If you are wondering how the the next-gen scalable cloud applications will be built, COM approach is one of the most promising alternatives.
The following mind map gives an overview of COM ecosystem. This is definitely not an exhaustive list, however it could be a good starting point for exploring various available options.
To summarize, keep calm and trust COM!
Note : The acronym COM used here is meant only for the purpose of explanation. It is not a standard acronym.
Why COM
- Containers: Build, ship and Run anywhere. A docker container can run on Bare Metal, VM, AWS, Azure, Google, Digital Ocean. Docker1.2 provides native support for Mac and Windows too.
- Open source: It's open source, duh!
- Microservices: Scale, flexibility, system resilience. Monoliths are bad for health!
COM Ecosystem
Building an individual microservice might be easy. Challenge is to integrate all services to build the complete solution. Therefore, for building COM based applications, following aspects need to be considered.
- Containerizing a service: While packaging a service in the container, we must remember that container is NOT a VM. Some of the best practices while building containers are one process per container, ephemeral behavior (can be stopped and a new one can be put in place with an absolute minimum of set-up and configuration).
- Service discovery: COM based cloud application can have hundreds and thousands of containers coming up and going down continuously. Hardcoded IP addresses or traditional DNS based service discovery will not be useful in such scenarios. We need to use a dynamic and scalable service discovery solutions. Open source solutions like zookeper, consule are the suitable for such use cases.
- Service-service communication: For synchronous communication, REST/JSON has been a de-facto standard for past several years. However, other promising alternatives like grpc/protbuf, thrift have emerged. These RPC frameworks take advantage of new HTTP2 (introduced in 2015), binary protocols and bi-direction streaming. For async communication, frameworks such as AMQP, STOMP have been widely used in the industry.
- Monitoring and logging: With hundreds of containers running in the system, we need specialized solutions such as fluetd, datadog, cAdvisor for system monitoring and logging.
- Automation and CI/CD: Building and deploying microservices without proper automation and CI/CD can prove to be the worst nightmare for an engineering team. Apart from traditional deployment techniques, engineering/devops team needs to consider container orchestration and container networking mechanism.
The following mind map gives an overview of COM ecosystem. This is definitely not an exhaustive list, however it could be a good starting point for exploring various available options.
To summarize, keep calm and trust COM!
References
- Linux Foundation : Introduction to cloud infrastructure technologies
- Martin Fowler: http://martinfowler.com/articles/microservices.html
- Docker best practices : https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
Note : The acronym COM used here is meant only for the purpose of explanation. It is not a standard acronym.
Subscribe to:
Posts
(
Atom
)
No comments :
Post a Comment