Byzantine Reality

Searching for Byzantine failures in the world around us

JustOnce

By day I’m mostly working on what you’d call “backend programming” in AppScale-land, so every once in a while I like to take on something a little bit different. This time, I wanted to make a web application that solves a problem I periodically face: sometimes I want to send someone a file, but only let it get downloaded once. It’s a fairly trivial app to write with the Google App Engine framework, but for the web interface, I wanted to mix it up a bit. So this time, I checked out AngularJS and threw together JustOnce – an open source app that lets you share files that can only be downloaded a single time.

The RESTful interface

JustOnce exposes a single route, /(.+), that accepts both GET requests and POST requests. Callers can upload something via the POST route, giving it a name in the URL, and its contents in the data itself. Using a tool like curl, all you have to do to store some text like “quux” at the URL “baz” is:

1
curl -X POST -d "quux" http://just-once.appspot.com/baz

Here I’ve pointing curl at the version of JustOnce I’ve got deployed on App Engine, but later I’ll talk about how to deploy it locally, on your own account on App Engine, or on a private cloud using something like AppScale. To download the item we just uploaded, just do a GET on the same URL:

1
curl -X GET http://just-once.appspot.com/baz

Of course, that only works once – if you run it again, you’ll get nothing back.

The web interface

While the RESTful interface is nice, the whole point of this experiment was to learn something new on the frontend side of things. This time around, instead of using something like jQuery, I checked out AngularJS. I actually really like how they have videos for the basic features, and egghead.io looks like it has more than enough videos for advanced features.

The web frontend, exposed on the / route, is pretty much a nice interface to the POST route described earlier. You choose a file to upload and a URL that you want it to be available at, and the form submission will upload it accordingly. I use Angular pretty minimally, so it’s really just to have the link letting you know where you can download the file at dynamically update as you update the form with the name field in it. Regardless, Angular is pretty cool – it definitely is a very natural way to update HTML pages dynamically.

The Backend

Both the RESTful interface and the web interface touch the same, minimalistic App Engine backend code. Let’s begin by defining a database Model with the Next Database (NDB), to save things people upload:

justonce.py
1
2
3
4
5
6
7
8
9
class OneTimeFile(ndb.Model):
  """A OneTimeFile represents a file that can be downloaded one time.
  
  Fields:
    name: The user-provided name of the file. This is the key of the item, to
      avoid having to manually index it.
    contents: A binary blob that holds the actual file the user uploaded.
  """
  contents = ndb.BlobProperty()

For saving data, it’s as simple as creating a new model and saving it to the Datastore:

justonce.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
  def post(self, name):
    """Uploads a new item with the specified name.

    This method doesn't do any validation on the name, so if another item is
    stored with the same name, it will be overwritten.

    Args:
      name: A str that names the item that should be uploaded.
      body: The request body, which will be stored for later retrieval.
    """
    one_time_file = OneTimeFile(id = name)
    body = self.request.body
    if 'body-ui' in self.request.POST.multi:
      one_time_file.contents = self.request.POST.multi['body-ui'].file.getvalue()
    else:
      one_time_file.contents = body

    one_time_file.put()

For retrieving data, we just get the item from the Datastore, give it to the user, and delete it:

justonce.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
  def get(self, name):
    """Downloads the item indexed by the given name, if it exists.

    Note that since the item is deleted after the download is started, if the
    caller fails to download the item, there is no way to download it again.

    Args:
      name: A str that names the item that should be downloaded.

    Returns:
      If the item exists, its contents are printed as the response. Otherwise,
      a 404 is returned to the user.
    """
    one_time_file = OneTimeFile.get_by_id(name)
    if one_time_file:
      self.response.out.write(one_time_file.contents)
      one_time_file.key.delete()
    else:
      self.error(404)

Of course, it has all of the nice perks that App Engine offers: with only that code, I get autoscaling, automatically managed load balancers, application servers, and databases.

Running JustOnce for yourself

What would be the fun of JustOnce if you couldn’t run it yourself? Well, it’s open source, under the extremely permissive MIT License, so begin by grabbing a copy of JustOnce:

1
git clone git://github.com/shatterednirvana/just-once

If you want to run it on your laptop, download the Google App Engine SDK for Python and run it just like you would any other App Engine app:

1
dev_appserver.py .

That will deploy your app on http://localhost:8080. But what if you want to run on your own App Engine account? In that case, just get a unique appid from http://appengine.google.com, put it in the app.yaml file, and deploy your app with:

1
appcfg.py update .

And last, but certainly not least, if you want to deploy JustOnce to an AppScale cloud, start up AppScale and deploy it just like any other app:

1
appscale deploy <path-to-just-once>

Future Work

JustOnce is a nice proof-of-concept that solves the problems I’ve run into trying to share files only one time. But there are some limitations / annoyances I haven’t gotten around to fixing yet:

  • Since this app uses the Datastore API to store and retrieve data, it’s limited to storing 1MB files. It could be refactored to use the Blobstore API, which doesn’t have that restriction. However, since it looks like the only way to read and write files is by getting an upload URL, that could pose problems when letting users choose their own URLs to download the file at.
  • Once a user uploads a file via the web UI, it sends them to a blank page. Presumably the frontend could use an AJAX call to post the file’s contents, so that we could display a success message instead of a blank page.

Of course, feel free to send a pull request and I hope you find JustOnce to be interesting or useful!