Spot Instances are too transient for persistent workloads,” or so most think.
Even DevOps pros who are running production workloads on Spot don’t believe it could be a fit for databases or
stateful applications. Or at least not without reworking their architecture to a large degree.
Spotinst’s Elastigroup, however, enables users to run any workload without a single point of failure on Spot Instances with an SLA guaranteeing high-availability.
Here’s how to run 5 workloads that demand consistency and data integrity on Spot:
1. Stateful Applications on Spot
As long as your
stateful application can tolerate a maintenance window (i.e night hours/ weekends), you can run it on Spot Instances using Elastrigroup. In case of a Spot Interruption, Elastigroup will try to restart or launch your Spot Instance on a different
Instance Type or even on a different
pricing model such as On-Demand to maintain your application availability.
2. Running persistent NoSQL DBs
From MongoDB to Cassandra, running databases on Spot Instances is not the most obvious solution. But with Elastigroup, it is possible to run persistent databases on Spot. Today, we have dozens and dozens of customers running persistent databases on Spot Instances.
In the case of Cassandra, for example, it works as follows: If a Cassandra node is replaced, Elastigroup automatically clones the instance and data to reboot it as a new instance before there is a Spot Interruption. At the same time, the cloning of the previous instance ensures that cluster IOPs are not wasted on bringing a new instance up.
For MongoDB, you can utilize their replica sets to run MongoDB workloads with 100% availability and data persistence on Spot. In order to provide fault tolerance, MongoDB allows creating replica sets which are, essentially clusters of MongoDB servers that implement master-slave replication and automated failover. By configuring this feature and running your MongoDB workloads on Elastigroup, you can utilize the “detach + re-attach EBS” feature for Stateful apps to maintain availability and data integrity for your MongoDB workloads while running them on Spot Instances.
Note that you should run at least 1 On-Demand Instance as the primary Instance in your MongoDB replica set while setting its priority for primary election to 1000. This ensures that the replica will always accept requests.
3. Container orchestrators (Kubernetes, Amazon ECS, Docker Swarm, and more…)
Not only does Elastigroup enable you to run Kubernetes or other Container Orchestration platforms (Docker Swarm, ECS, Mesosphere, etc.) on Spot, but it actually does, even more, to efficiently autoscale your workloads on containers.
Elastigroup is constantly calculating your costs for scaling up and down not only to Spot, Reserved, or On-Demand Instances but to differently-sized machines (2 x
m3.large v. 1 x
m3.xlarge, for example).
This helps ensure maximum efficiency not only by running on Spot but by playing the Tetris game of running container workloads on different machines with optimal utilization.
4. CI/CD Workloads
Sometimes, it may be a challenge to run these workloads on Spot Instances. Trying to manage all of your CI/CD automation while dealing with some inevitable Spot Instance interruptions can be highly error-prone and slow you down from getting things done. With Chef, for example, it can cause a ‘
Zombie’ scenario, as Chef will be unaware of Spot Instance interruptions. With Jenkins or CodeDeploy, the complexity of managing updates and configurations across Spot Instances can be a massive headache if there is a Spot interruption.
Elastigroup, however, enables you to easily integrate with all 3. You can pre-define parameters for scale-up and scale-down to run your CI/CD pipeline at minimal cost. This integration ensures that your configurations are set across Spot Instances even during terminations and replacements that are managed by Elastigroup.