Byzantine Reality

Searching for Byzantine failures in the world around us

Running App Engine Tasks Automatically

With version 1.2.3 of Google App Engine comes the Task Queue API, which gives the ability to run tasks asynchronously in the background. The only downside is that on the ‘lite’ edition that they give you to test it out on, the tasks don’t run automatically. The rationale is solid: you should control when they run so that you can debug your app and so forth. However, if for some reason you wanted to have them run automatically (which we needed for AppScale), a few changes had to be made. Furthermore, there are a number of efforts to use the lite SDK with real databases, and those maintainers may find this to be a helpful contribution.

The primary change is in google/appengine/api/labs/taskqueue/taskqueue_stub.py (which our code base renames to taskqueue_distributed.py since our AppScale stuff is all distributed). Instead of adding the task onto the queue, we just start up another thread and run the task immediately. The main chunk of the code is thus:

host = os.environ['MY_IP_ADDRESS']
port
= os.environ['MY_PORT']

class AppScaleRunTask(Thread):
 
def __init__(self, request):
   
Thread.__init__(self)
    self
.request = request
 
def run(self):
    queue
= self.request.queue_name()  

    sys
.stderr.write("received a request for queue " + queue + "\n")
    sys
.stderr.write("arg address = " + host + "\n")
    sys
.stderr.write("arg port = " + port + "\n")
   
if queue == "mapreduce":
      sys
.stderr.write("mapreduce queue not implemented yet\n")
   
else:
      url
= self.request.url()

      method
= self.request.method()
     
if (method == taskqueue_service_pb.TaskQueueAddRequest.GET):
        method
= 'GET'
     
elif (method == taskqueue_service_pb.TaskQueueAddRequest.POST):
        method
= 'POST'
     
elif (method == taskqueue_service_pb.TaskQueueAddRequest.HEAD):
        method
= 'HEAD'
     
elif (method == taskqueue_service_pb.TaskQueueAddRequest.PUT):
        method
= 'PUT'
     
elif (method == taskqueue_service_pb.TaskQueueAddRequest.DELETE):
        method
= 'DELETE'

      conn
= httplib.HTTPConnection(host + ":" + port)

      sys
.stderr.write("body is '" + self.request.body() + "'\n")
      keyval
= self.request.body().split("&")
      sys
.stderr.write("keyvals = " + str(keyval) + "\n")

      keyvalhash
= {}
     
for item in keyval:
        temp
= item.split("=")
        key
= temp[0]
        val
= temp[1]
        keyvalhash
[key] = val

      sys
.stderr.write("keyvalhash = " + str(keyvalhash) + "\n")
     
params = urllib.urlencode(keyvalhash)
      conn
.request(method, url, params)
      conn
.close()

Since all a task is is just a HTTP action on a given URL, implementing it turned out to be pretty simple. This way of doing it also has the perk of automatically running your tasks, but now tasks almost always run in the order they are started in. I don’t really see it as a downside, however, and has a much added perk of being very simple to understand and maintain.

So with that said and done, you can find the code here on LaunchPad. Just get a copy of the code by running this:

bzr branch lp:~cgb-cs/+junk/appengine-tasks-mr

Then you can try it out with the sample counter application that I pieced together from Google’s Task Queue page by doing the following:

cd google_appengine
./dev_appserver.py demos/counter --clear_datastore

Navigate your browser to localhost:8080, click on the button, and success! The counter should automatically increment and from there you’re in business!

The more apt of our readers will notice that the project is named appengine-tasks-mr and that we’ve only talked about the Task Queue API and nothing about this MR thing. That’s all coming later, but as you can see from looking at the code, I’m looking at if throwing in MapReduce support into Google AppEngine is viable and how to piece it all out in a coherent manner. Because of that, the queue name “mapreduce” is reserved right now (so don’t name any of your queues that for now), but anything else is fine. Enjoy!