Byzantine Reality

Searching for Byzantine failures in the world around us

Using Spot Instances in EC2

AppScale lets you deploy to On-Demand Instances in Amazon EC2, and AppScale 1.6.9 expands this support to also include Spot Instances. Spot Instances cost dramatically less than their On-Demand counterparts (although they often take much longer to acquire), and thus can save you money when deploying AppScale in AWS. This post details how to use this new Spot Instance support within AppScale.

Before continuing, you should be familiar with how to deploy AppScale with On-Demand Instances in EC2.

Spot Instance Support With Automatic Billing

Spot Instances in EC2 require the user to specify what the maximum price they’re willing to pay for the machines is. Choosing a price is a bit of a black art as this YouTube video from AWS describes. Using our “Automatic Bidding” support, the AppScale Tools query the current bid price and adds 10% (to improve your chance of acquiring the machines you need for your AppScale cloud). To use this option, update these two variables in the AppScalefile that is automatically generated for you via the appscale init¬†command.

1
2
instance_type: c1.xlarge
use_spot_instances: True

We recommend using c1.xlarge, m1.xlarge, or m1.large for AppScale instances. The following is example output from an AppScale deployment using these options.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
outer-haven:ec2 cgb$ appscale up
Machine image ami-74fa641d does exist
Starting AppScale 1.6.8 over the ec2 cloud.
Secret key is c330ede601b84f538537c37f6c69ea9d
Verifying that keyname appscalecgb212 is not already registered.
Creating key pair: appscalecgb212
Creating security group: appscalecgb212
Starting 1 machines with machine id ami-74fa641d, with instance type c1.xlarge, keyname appscalecgb212, in security group appscalecgb212
Using spot instances
The average spot instance price for a c1.xlarge machine is 0.0825611285266, and 10% more is 0.0908172413793
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Waiting for your instances to start...
Started 1 spot instances in 321 seconds
Please wait for your instance to boot up.
[Errno 61] Connection refused
Waiting for ec2-184-73-112-3.compute-1.amazonaws.com:22 to open
... same as usual ... 
AppScale successfully started!
View status information about your AppScale deployment at http://ec2-184-73-112-3.compute-1.amazonaws.com/status

You can see that the average price for a c1.xlarge instance when I ran this was $0.08, and that we got our instance with a max bid price of $0.09 – both of these are quite a bit better than the On-Demand price of $0.24! You’ll also note that these instances take quite a bit longer to get than their On-Demand equivalent – 321 seconds for the Spot Instance here versus about 30 seconds for an On-Demand Instance.

Spot Instance Support With Manual Bidding

You can also use manual bidding for spot instances by adding an additional variable to your AppScalefile.

1
2
3
instance_type: m1.large
use_spot_instances: True
max_spot_price: 2.00

You’ll still pay the market price for your instances, but if that increases, you won’t lose your machines (instead, you’ll just pay the increased price). The AWS video details this as being useful if you just need extra compute capacity and aren’t overly concerned with the price (vs AppScale’s Automatic Bidding).

A Caveat

One thing to know about Spot Instances is that when you put in a bid for instances, you’re putting in a request for an instance and you aren’t guaranteed to get it! If you bid significantly lower than the market price, you likely won’t get an instance (until the market price drops down to your bid price, which could be never). Also, if you bid a reasonable price but Amazon doesn’t have the spare capacity available, you won’t get your instances. In AppScale, we wait for a maximum of 30 minutes for your Spot Instances to come up and then abort if they do not. Here’s what it looks like when you run AppScale and we can’t get you Spot Instances:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
outer-haven:ec2 cgb$ appscale up
Machine image ami-74fa641d does exist
Starting AppScale 1.6.8 over the ec2 cloud.
Secret key is 1da5fec42dcc43e7993c213201d6529d
Verifying that keyname appscalecgb210 is not already registered.
Creating key pair: appscalecgb210
Creating security group: appscalecgb210
Starting 1 machines with machine id ami-74fa641d, with instance type m1.large, keyname appscalecgb210, in security group appscalecgb210
Using spot instances
The average spot instance price for a m1.large machine is 0.0836421807747, and 10% more is 0.0920063988522
Waiting for your instances to start...
<message repeated a lot of times>
Waiting for your instances to start...
No public IPs were able to be procured within the time limit

In this case, we log into the AWS Console, click on “EC2”, and then click on “Spot Instances”. That shows the reason why we couldn’t get Spot Instances – in the example above, it was “Capacity not available”. We cancel our Spot Instance Request by clicking “Cancel” so that we don’t get the instance (and thus get billed for it).

Conclusion

We overviewed how you can use AppScale to obtain Spot Instances instead of On-Demand Instances in Amazon EC2. Since AppScale is open source, you can also change how the Automatic Billing scheme works, if you find that bidding 10% more than average doesn’t suit your use case (or you’d like something more complex but still automated). For the adventurous, check out appscale-tools/lib/agents/ec2_agent.py and look at get_optimal_spot_price. In there, you can make it do anything you like, as long as it returns a float (the price to use).