How to quickly move a microservice infrastructure off AWS in a weekend

Most organizations move to AWS to trim costs and improve scalability. But at least one company found they could reduce costs and improve control while maintaining a robust server infrastructure by moving away from AWS. At the KubeCon Conference in San Francisco, Paolo Kinney, CTO of Vinli explained what they learned in automating the deployment of Kubernetes infrastructure to facilitate this process.

Vinli is a connected car platform that streams the data from connected cars into the cloud. The company provides access to this data to developers via a set of APIs that makes it easy to create new applications for consumers. Developers can work with low-level APIs around telemetry and higher level APIs around safety, trips, automobile diagnostics, and behavioral services.

Kinney said they decided to deploy the entire back-end infrastructure with microservice hosted on Docker containers to make it easy to customize. All of the back end logic and front end building blocks are released on top of containers. As a result the company ends up spinning up and tearing down hundreds of Docker images daily across its service infrastructure.

Moving off AWS

The platform was originally deployed on top of AWS Elastic Beanstalk. It worked well, but can get expensive as the back end grows past 15-20 services that need load balancing said Kinney. Costs started spiraling out of control as Vinli had to load balance close to 200 services.

Kinney decided to experiment with Kubernetes as a way of orchestrating its service infrastructure. Over the course of a weekend he and a small team refactored the entire microservice infrastructure to run on top of Kubernetes. It consisted of of about 45 applications, services, and workers running across 4-software stacks.

A key part of this transition lay in building the last layer of application management based on Kubernetes primitives. One strategy for keeping life simple for Vinli's internal development was to keep the requirements simple. Developers implement their code into Docker containers which can be named and quickly launched. The only internal requirements are for developers and designers to notify the ops team about new services and to enable a health check at the application level.

Thinking like a Farmer

Vinli built a number of simple Kubernetes primitives to optimize cluster manager for their infrastructure. These were named after farm operations like Shepherd, Foreman, Farmer, Burn and Butcher. Kinney said the whole orchestration infrastructure is rather simple. All of these separate processes run in about 200 lines of code. They started by observing their commonality between their services and built the tooling around that.

Shepherd makes it easy to roll existing services over to new updates quickly. Kinney said the standard Kubectl cluster control tool took 30-seconds to a minute to spin up new services. Shepherd was basically optimized for Vinli's infrastructure to execute a service rollover in about 5-seconds. This lets Vinli move containers along without worrying about configuration management from stack to stack.

We throw away pods just to keep new pods coming up, which helps us avoid long running logs or memory issues.

Paolo Kinney, CTO of Vinli

Workers are the microservice processes inside of Kubernetes that help to orchestrate the spinning up and down of application containers. These need to scale up and down at a different pace to coordinate changing application loads. Vinli created another internal tool called Foreman for managing the lifecycle of worker containers that orchestrate other Docker processes. It works in conjunction withy another tool called Farmer to increase and decrease replica count of Docker containers in response to service loads.

Vinli deletes "pods" or groups of Kubernetes containers all of the time. Kinney said, "We throw away pods just to keep new pods coming up, which helps us avoid long running logs or memory issues." A tool called Burn blows away all of the pods in a group on command. It works in conjunction with Butcher to kill one or more pods, handle cron jobs, and post updates to the internal Slack IRC feed used by the developer and operations teams. 

Regularly cull the herd to keep things fresh

Once a day they blow all of the pods away, and within a few seconds the entire infrastructure is back up and running. Kinney said, "With Kubernetes, everything is extremely disposable. We want to be able to allow things to refresh themselves."

Another key to making the application infrastructure as disposable as it is to eliminate persistence in the microservices. All of the persistent data is stored outside the services using Redis and Elasticsearch.

This ensures that no data is lost when new containers or new configurations are pushed into production. Kinney said, "This lets us stay agile."

Have you switched away from any of the big cloud vendors? What was your experience? Let us know.