2011-05-19
Go + AppScale
I just got back from Google I/O last week and after hearing about the cool new Go support in App Engine, I set out to get Go working over our implementation in of App Engine, namely AppScale. After a bit of hacking I got it working - here's a screenshot of the go-mandlebrot demo:
Living up to their promise as being super cool and open-source, the Go App Engine code is available for people wanting to hack around with it - of course that's me! But I wasn't able to compile the Go App Engine SDK out of the box. Thankfully, it turned out to be pretty straightforward, so here's what to do if you want to do it (or if you're a future version of me who has forgotten how to do it and needs to remember). Enjoy!
Compiling the Go App Engine SDK in Three Easy Steps, each with sub-steps:
Step 1: Install Go
Step 1.1: Install Go's language prerequisites
sudo apt-get install bison ed gawk gcc libc6-dev make sudo apt-get install python-setuptools python-dev build-essential
Step 1.2: Install Mercurial (skip if you can run 'hg')
sudo easy_install mercurial
Step 1.3: Actually install go
hg clone -u release https://go.googlecode.com/hg/ go cd go/src ./all.bash
Step 1.4: Move Go out of your home directory
cd ../.. sudo mv go /usr/local/ export PATH=$PATH:/usr/local/go/bin export GOROOT=/usr/local/go
Step 2: Install Go's Protocol Buffer Bindings
Step 2.1: Install Protocol Buffers
wget http://protobuf.googlecode.com/files/protobuf-2.4.1.tar.gz tar zxvf protobuf-2.4.1.tar.gz cd protobuf-2.4.1 ./configure make make check make install cd .. rm -rfv protobuf-2.4.1 protobuf-2.4.1.tar.gz
Step 2.2: Install the Go Protocol Buffer Library and Protocol Compiler Plugin
goinstall goprotobuf.googlecode.com/hg/proto cd $GOROOT/src/pkg/goprotobuf.googlecode.com/hg/compiler make install
Step 3: Actually compile the Go App Engine SDK
Step 3.1: Get the Go App Engine SDK Source
hg clone https://appengine-go.googlecode.com/hg/ appengine-go
Step 3.2: Compile it
cd appengine-go make
And that's it! It should compile without problems, and now you have the ability to mess around with your very own Go App Engine SDK! I'm not exactly sure what you'd want to do with it, but I sure know what I do! Stay tuned for more updates from the wacky world of the cloud, now with 100% more Go!
2011-05-04
Pipeline API on AppScale
Update (5/25/11): AppScale 1.5 adds transactional task support, so don't you don't need to make the changes listed here to get the Pipeline API up and going if you have that version or anything newer. Otherwise, read on!
Recently I've been hacking away at the super-cool Pipeline API that Google App Engine puts out. It's an easy way to run otherwise complex workflows over large datasets, and of course I wanted to get it working on AppScale as fast as possible. Since we don't have transactional tasks working in AppScale right now, if you want to play with the Pipeline API in AppScale, all you have to do is open up the Pipeline API's pipeline.py file, and replace this:
task.add(queue_name=self.queue_name, transactional=True)
with
task.add(queue_name=self.queue_name, transactional=False)
and you're good to go! Here's a screenshot showing it running on our AppScale deployment:
Of course, we'll have a version of the API with these patches (and eventually, transactional task support so you won't need a separate API in the first place) out with the 1.5 release. Enjoy!
2011-04-25
Talk at LA.rb and SoCal
Just gave a talk last week at LA.rb about the AppScale project (essentially a shortened version of my SBonRails talk) - here are the slides:
I also gave a talk at the SoCal Programming Languages and Systems Workshop on Neptune, our metaprogramming extensions to Ruby that make high performance computing super simple - here are the slides to that:
I also got the video of it, which I'll post a link to when I get it on the YouTube.
Enjoy!
2011-03-10
Presentation at SBonRails
I was at our local SBonRails meetup group last night and gave a talk on AppScale, Google App Engine, and the research we've done with it so far. Here's a link to the slides from my talk. Enjoy!
