Byzantine Reality

Searching for Byzantine failures in the world around us

Static IPs in AppScale

One feature that’s been heavily requested in AppScale fairly recently is the ability to acquire a static IP address from your favorite cloud provider and have traffic go through it, instead of whatever IP address or fully-qualified domain name they give you. So for AppScale 1.13.0, we’ve added support for Elastic IPs in Amazon Web Services as well as Static IP support in Google Compute Engine! Let’s talk about how you can use it, and what we do behind the scenes to hook it all up for you.

Using Elastic IPs in Amazon Web Services

To use an Elastic IP when running in AWS, begin by acquiring an Elastic IP in the usual fashion:

1
2
outer-haven:ec2 cgb$ ec2-allocate-address
ADDRESS 54.204.29.48            standard

Next, edit your AppScalefile and add a new parameter, static_ip, which tells AppScale what Elastic IP to use in this deployment:

1
static_ip: '54.204.29.48'

Then, just run appscale up as usual! When you do that, we will perform the following steps to get your Elastic IP in place:

  1. Validate that the Elastic IP you entered actually exists and belongs to you
  2. Boot up the first instance in your AppScale deployment
  3. Associate the Elastic IP you gave us with the instance ID of the head node we just spawned

The code for Step 1 is pretty straightforward. Using boto, it looks like this:

1
2
3
4
5
try:
  conn.get_all_addresses(elastic_ip)
  return True
except boto.exception.EC2ResponseError:
  return False

Similarly, associating your Elastic IP with the instance is simple with boto:

1
conn.associate_address(instance_id, elastic_ip)

Using Static IPs in Google Compute Engine

Conceptually, using a static IP in Google Compute Engine is very similar to Amazon Web Services, although a lot of the implementation-level details are different. Begin by acquiring a static IP in the usual fashion (substituting in whatever name you like for ‘cgbtest10’ below):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
outer-haven:gce cgb$ gcutil reserveaddress --region=us-central1 cgbtest10
INFO: Waiting for insert of address cgbtest10. Sleeping for 3s.

Table of resources:

+-----------+-------------+----------+-----------------+
| name      | region      | status   | ip              |
+-----------+-------------+----------+-----------------+
| cgbtest10 | us-central1 | RESERVED | 162.222.183.168 |
+-----------+-------------+----------+-----------------+

Table of operations:

+------------------------------------------------+--------+----------------+
| name                                           | status | operation-type |
+------------------------------------------------+--------+----------------+
| operation-1384185686703-4eae8d82738e0-f315d868 | DONE   | insert         |
+------------------------------------------------+--------+----------------+

Next, add the static_ip parameter to your AppScalefile in the same way we did for AWS:

1
static_ip: '162.222.183.168'

Like before, run appscale up as usual to start up AppScale with your static IP. When you do that, we’ll do the following to use your static IP:

  1. Validate that the static IP you entered actually exists and belongs to you
  2. Boot up the first instance in your AppScale deployment
  3. Remove the public IP address from your instance
  4. Add the static IP address you gave us as the new public IP address for your instance

To make sure that your static IP exists, we use the addresses().list() call in the GCE API. We originally tried to use addresses().get(), with the IP you give us, but that doesn’t work, since it wants the name you chose (which doesn’t get passed in to us). Therefore, the code looks like the following, using the Google API Client:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
gce_service, credentials = self.open_connection(parameters)
http = httplib2.Http()
auth_http = credentials.authorize(http)
request = gce_service.addresses().list(
  project=parameters[self.PARAM_PROJECT],
  filter="address eq {0}".format(parameters[self.PARAM_STATIC_IP]),
  region=parameters[self.PARAM_REGION]
)
response = request.execute(http=auth_http)
AppScaleLogger.verbose(str(response), parameters[self.PARAM_VERBOSE])

if 'items' in response:
  return True
else:
  return False

For Step 3, we remove your public IP address by calling deleteAccessConfig:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
gce_service, credentials = self.open_connection(parameters)
http = httplib2.Http()
auth_http = credentials.authorize(http)
request = gce_service.instances().deleteAccessConfig(
  project=parameters[self.PARAM_PROJECT],
  accessConfig="External NAT",
  instance=instance_id,
  networkInterface="nic0",
  zone=parameters[self.PARAM_ZONE]
)
response = request.execute(http=auth_http)
AppScaleLogger.verbose(str(response), parameters[self.PARAM_VERBOSE])
self.ensure_operation_succeeds(gce_service, auth_http, response,
  parameters[self.PARAM_PROJECT])

Finally, Step 4 adds the static IP address as the new public IP by calling addAccessConfig:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
gce_service, credentials = self.open_connection(parameters)
http = httplib2.Http()
auth_http = credentials.authorize(http)
request = gce_service.instances().addAccessConfig(
  project=parameters[self.PARAM_PROJECT],
  instance=instance_id,
  networkInterface="nic0",
  zone=parameters[self.PARAM_ZONE],
  body={
    "kind": "compute#accessConfig",
    "type" : "ONE_TO_ONE_NAT",
    "name" : "External NAT",
    "natIP" : static_ip
  }
)
response = request.execute(http=auth_http)
AppScaleLogger.verbose(str(response), parameters[self.PARAM_VERBOSE])
self.ensure_operation_succeeds(gce_service, auth_http, response,
  parameters[self.PARAM_PROJECT])

Releasing Your Static IPs

And that’s it! Remember that you get billed for Elastic IPs in Amazon and static IPs in Google even if you’re not using them, since the IPv4 address space is pretty sparse, so be sure to delete them when you no longer need them. In Amazon, it’s as simple as:

1
2
outer-haven:gce cgb$ ec2-release-address 54.204.29.48
ADDRESS 54.204.29.48

And in Google:

1
2
3
4
5
6
7
8
9
outer-haven:gce cgb$ gcutil releaseaddress --region=us-central1 cgbtest10
Release address cgbtest10? [y/n]
>>> y
INFO: Waiting for delete of address cgbtest10. Sleeping for 3s.
+------------------------------------------------+--------+----------------+
| name                                           | status | operation-type |
+------------------------------------------------+--------+----------------+
| operation-1384189195663-4eae9a94e9e40-e64dd5ae | DONE   | delete         |
+------------------------------------------------+--------+----------------+