Byzantine Reality

After finishing Do Androids Dream of Electric Sheep?, I decided to continue reading what I have heard of as 'the classics' with Slaughterhouse Five. Like Do Androids Dream of Electric Sheep?, Slaughterhouse Five is a short but sweet read, and I definitely recommend it to those looking for a quick read.

Slaugherhouse Five follows the life of Billy Pilgrim, who has become "unstuck in time." He experiences the events of his life out of order, which makes for a particularly interesting read. Kurt Vonnegut does an excellent job of making this readable, as what could become very confusing is instead posed carefully and to maximum effect. The satire and anti-war message are also done nicely - it's told through Vonnegut's own experiences witnessing the bombing of Dresden, and the times when Vonnegut breaks the fourth wall are done superbly. They do so in such a clear fashion that it temporarily broke my immersion, just to pull me in deeper, now that I know that this thing that I've just read really happened. It makes for a great blend of fiction and non-fiction.

The characters are developed nicely, as was the case with Do Androids Dream of Electric Sheep? and Catch-22. No particular character overstays their welcome, and jumping pseudo-randomly through time makes it really easy to speed up situations that would otherwise run long and be boring or slow down situations that need the time. The religious satire is done very nicely as well - I loved the "alternate" interpretation of the Bible as "don't mess with people who are well connected" and the justification behind that.

The book is immersive and I got through it in a single day, so that's a great sign. If you haven't read it, I would definitely say it's a classic, so go check it out!

I've been a long-time lover of the movie Blade Runner and while I've owned Do Androids Dream of Electric Sheep? (the book from which the movie is based), I had never got around to actually reading it until today. It's a great read and I definitely should have read it a long time ago.

The book is just the right length - not too short but certainly does not overstay its welcome. I started reading it this morning and (with several breaks) just finished it. It's incredibly immersive and doesn't suffer from a lot of the pitfalls that I've complained about in other books. The few sex scenes that it has aren't awkward (like the A Song of Fire and Ice series) and the action scenes are lean and to the point, just like the rest of the book. The movie definitely picks up a lot of inspiration from the book, but that's about it. The details vary greatly between the book and movie, and all that really remains the same are the generalities. Both are amazing and have moments that really make them shine, and the book is a little bit deeper than the movie (although this really comes in via the whole Mercer thing).

Do Androids Dream of Electric Sheep? is exactly what a book should be. It has a lot of memorable scenes, despite being relatively short, brings a good amount of depth to its main characters, and there's a bit right before the end that genuinely shocked me in a good way, which I won't ruin too much for you if you haven't read it but involves Rachael's revenge on Deckard. So the book is the epitome of 'short but sweet', and I got a bit of an extra kick out of it having been in San Francisco this summer (the location where the book takes place) and being able to recognize most of the places it talked about. I definitely recommend the book, even more so to those who love the movie Blade Runner!

While I've seen myself as a history enthusiast, I've definitely read a lot more about the West than the East. So I sought to remedy that by reading The Samurai and the Sacred, a contemporary look into Japanese history and culture. It was an informative book but I felt it was aimed more at those well-versed in Japanese culture than at the layperson.

The book starts off strong and covers exactly what you would expect a book on Japanese culture to cover. The thesis of the book seems to be "the relationship between the Japanese and religion is difficult to quantify and definitely is not what you are commonly taught", and uses the metaphor of a "religious supermarket" to describe the pick-and-choose attitude undertaken by the Japanese with respect to religion. The idea seems to be that instead of having a single religion with a set of beliefs and standards, the Japanese have historically taken many religions and meshed them as needed. The process was definitely not uniform: everybody seems to get a unique blend of religions based on local and temporal influences.

What is confusing in this book is that despite this blend, there seem to be parts in Japan's history where a single, well-established religion (let's take Buddhism as one example) is seen as problematic and somehow de-integrated from the existing culture. Since the book goes out of its way to tell us how well Buddhism and Shinto are in Japan, it's hard to grasp how this happens, and is a point of confusion in the book. The history and interplay involving Christianity is a lot more clear and a lot of this seems to be because Christianity was not integrated as well into Japanese culture as Buddhism was (although over time it appeared to have gained good traction there).

The book is unique in the large number of visuals it displays: nearly every other page has an image on it, and goes a long way towards making the book readable. The iterative style of the book goes a bit against this, however. What I mean is that the book is broken down into themes, so each chapter focuses on a particular theme, and can run from the beginning of Japan's history to the present day. This means that once we've gotten to the present, the very next chapter can jump all the way back to the beginning again. This would be ok but in this book a lot of important events or people were not thoroughly explained the first time through. I had no idea what the Meiji Restoration was until the third time I saw it, when it was finally explained. I get that it wasn't that important the first two times, but since the actual explanation was really only a sentence long, it just created a lot more confusion than necessary.

I've loved math for quite a long time, and have stumbled through the math section at my local bookstores whenever I got a chance. As such, it was pretty hard to miss The Drunkard's Walk, a short but sweet introduction to probability and statistics (with a little bit of history dabbled in), and was entranced by its tagline: "How Randomness Rules Our Lives". I finally got around to buying (and reading) it, and overall it was pretty good.

For my birthday back in June I got the first four books in George R.R. Martin's famous (by now) series A Song of Fire and Ice, and just finished the first two books, A Game of Thrones and A Clash of Kings. Overall I thought they were good but not as amazing as I had been led to believe. Let me tell you why they are good but not great. THIS REVIEW IS SPOILER-FREE!

Why it's good:

Despite their impressive length (>800 pages each with what looks like 8pt font), the books are definitely not boring. There are lots of great characters, and lots of characters in general. This gives the feeling of a big open world and although it sometimes is hard to keep track of who is related to who, this really only crops up in the first book and once characters are incrementally introduced by the second book, it's not that bad. Tyrion is my favorite character so far, which is probably not a surprise because others seem to like him a lot as well. As the series' only vertically-challenged member, he's often mocked by others but far exceeds them as far as intelligence goes. That definitely comes in handy, because there's a lot of political maneuvering going on (plans within plans) and friends can become enemies very quickly (and vice versa).

Martin knows that he has a ton of characters and realizes that many of them need to die through natural or other means throughout this series, and doesn't flinch at killing main characters. This is a big of a mixed bag, but in general is a positive move: killing beloved main characters evokes a lot of emotion and in general, they leave behind enough family/friends to continue the battle after they've left. Thus far really only a single main character has died that I really didn't want to die (everyone who has read the first book likely knows who this is) and reading the second book really brings out a lot of emotion as to the legacy he left behind.

Why it's not great:

There is a lot of maneuvering going on in this series so far, with no sign of it letting up in the next few books. What I mean by maneuvering is 'interactions / showdowns between various characters', and I guess this is to be expected, with the series at five books so far and not yet complete. However, this presents two problems:

1) Since the maneuvering between different characters often doesn't result in any of the parties being killed or expelled from wherever they're operating from, it's hard to get really excited when two of the big characters have a showdown, and when something interesting actually does happen, it completely blindsided me and I had to keep re-reading those sections to make sure I didn't miss anything. As I go to the next chapter I would say something like "wow let's see what trouble Tyrion gets himself into now" but at the same time realize that there's a high probability that it will be nothing decisive. The chapters are all interesting, so I wasn't bored at any part of the books (again, amazing for ~2000 pages), but it's hard to commit to even the next chapter when you know that there will really only be one or two big plot twists in the whole book.

I am aware that this is often referred to as "character development", but for me it just meant that not much of any real value goes on for a lot of the book. So far, it hasn't been the case that any character has developed so much that their decisions couldn't have been guessed from knowing who they were when they were first described in the first book - that is, their development really hasn't changed them to any noticeable degree.

2) The layout of at least the first two books seems to be designed to prevent being immersed in it. Each chapter typically focuses on a different character than before, and in some cases the characters are on opposite ends of the world with nothing to do with one another, so the second I've gotten into the story and really want to know what happens next, I have to put down everything I know about, let's say Tyrion, and remember everything I know about Bran. And this is a massive pain in the ass - I really want to see more of Tyrion and don't really care much about Bran (don't worry about the names too much). I get that the story has to progress on all sides, but this completely kills my drive to read more because now I have to care about Bran and the second I'm interested in his story, BAM! now we're talking about Davos, who is nowhere physically close to the other two characters, and while I like Davos, now we're back to step 1 and I have to remember his story...

Each of the first two books have had climaxes where they stop switching between faraway characters and either stick with one character or switch between characters very close to one another, and these are easily the best parts of the books. These are the parts where Martin really shines - he's a great writer and this is where he gets to show us it. Unfortunately, the rest of the time it suffers from what I will call True Blood Syndrome, in which I have a minority of characters that have really interesting stories but I have to waste a lot of time seeing the other characters dick around to fill time. Note - I've only seen that TV show, so the books could be different. If you've seen the TV show you likely can sympathize - the main plot focuses around this whole vampire thing but the second-tier characters steal a surprising amount of screentime with their adventures that I couldn't care any less about, and the series as a whole would be a lot better if they bit the dust.

So how does that apply to A Song of Fire and Ice? Well, let me give you a little bit of spoiler-free info. In the second book, A Clash of Kings, there are many kings who are clashing to be the one king who rules over all (see! no spoilers!). But while we want to be reading about these kings and their clashing, about half of the time is consumed by second-tier character maneuvering. Granted, the second-tier characters here are a lot more interesting than in True Blood, but by the end of the book they're in pretty much the same shitty situation as when they started. And to top it off, the clashing is the most interesting part but doesn't come up that often - it's more of a "what happens to the world when kings decide to clash" as opposed to "hey some kings are about to clash, do you want to see that?"

A Possible Saving Grace

So in the second book a lot of the tier-one characters (the interesting ones) are conspicuously missing in action. I'm not saying they've been captured and thus don't get screentime, I'm saying you hear about things they've done after it's happened from the second-tier characters. A natural response I had was to say "well what the hell happened to them?", and in typical fashion for a multi-book series, the next book hints at the answer - it runs in parallel to the second book and will have the missing points of view. The book, A Storm of Swords, hints at many battles and tells us right at the beginning that it will not keep the timeline in sync anymore. What that means is spelled out pretty clearly - while everything so far has alternated between characters with events happening more or less at the same time, this book will jump wildly around in time and space to retcon something epic into the story.

Retconning is a very dangerous plot device - I have been on record for hating it as a cheap way for authors to add drama to the story that they forgot to talk about when they wrote the story the first time, and a whole post will be devoted to this later, but it presents a big risk: the next book (for me) has the potential to make or break the series. A particularly stupid retcon could easily kill the series, but if it turns out that the first two books have set it up nicely then it could put a lot of new life into the series and keep it going strong for me.

Although I've spent more time talking about the negatives than the positives, I do recommend at least the first two books, especially if you're into fantasy books. it also showed me how Dan Carlin has killed my love of fantasy and made me a lover of history - he can talk about the fall of the Roman Republic for TEN HOURS and have it be amazing and exciting throughout. There are a crapload of characters there but he knows how to give each of them the screentime that they deserve (which I think is a lot harder since the figures are all real people) and make a seriously engaging story out of a ton of source material. That's also best kept for another post though, so let me reiterate what I said near the beginning: I've read about 2000 pages of A Song of Fire and Ice without getting bored, and that's a great sign.

On one of my random adventures through San Francisco's comic book stores I stopped at Isotope Comics (my favorite comic book store). I happened to be in on the day that Grant Morrison was there promoting his new book Supergods. While it was unfortunate that I was there an hour early and completely unaware of him being there in the first place (thus unable to actually buy a ticket to see him), I did buy the book. I read through it pretty quickly and liked it a lot - if you're a comic book reader, this book is a must-buy.

Supergods tells us what comics mean about us as humans. This naturally entails a discussion on the history of comics, which takes up a majority of the book, and even though I had a good familiarity with it beforehand, Morrison does a great job of filling in lots of areas I knew with more information and bringing me up to speed on areas I knew nothing about. There's lots of fun little trivia-ish things in there that are great for the alpha-nerd to pick up on (like all the old-school Batman movies/actors), and lots of discussion on famous and obscure comics of the day. Morrison definitely has his history down, and ties in why certain comics were popular with social events of the times, as well as a good discussion of how the characters have evolved with them.

The rest of the book (told mostly in parallel with the main history lesson) focuses on Morrison's life itself, how he's changed, and how his comics have reflected it. Morrison clearly has had a rockstar life, with all that entails, and it makes for fun reading as well. It breaks up the history parts just often enough to keep both parts interesting, and his life plays like this psychedelic mind-trip of wacky experiences. 

Morrison also chats a bit about the comics he's been involved in, including DC's controversial Final Crisis. I've gone on record before as completely hating it, to the point that I put down all comics altogether until extremely recently, largely because I felt that the use of a deus ex machina is rarely justifiable, but he explains (a lot better than I'm going to right now) that it was all in pursuit of making the comic serve a bigger picture. Specifically, he's talking about issues of hope and giving up, and that hope could triumph over all at the end of the day. At the time, I saw it as changing the fundamental motivations of one of DC's awesome baddies (Darkseid) and just making him into a boring villain, as he switched from being a fascist seeking order at all costs to just being a bastard (which was one-dimensional and boring). Now I see how it served the big picture he was going for but I still agree with my original assessment. Regardless, it taught me to keep my eyes out for this more often, and I saw a bit of it going on in Marvel's Vengeance - more on that in a future post.

Supergods includes a great discussion of the comics of the last few decades, so it also has provided me with a list of a few greats I've missed out on. It answers its thesis well and kept my interest throughout, so I recommend it highly!

Recently I took on a redesign of an old project of mine: Active Cloud DB. It's a Python App Engine app that exposes a REST API to the Datastore, allowing clients of any programming language to access Google's scalable key-value datastore. However, the web frontend didn't look too hot, and when I saw what Bootstrap could do, I knew I could use it to do justice for Active Cloud DB. So I did just that, and made a new Active Cloud DB with Go App Engine and Bootstrap, and boy does it look a lot nicer now. Of course it's all open source, so feel free to grab the code we'll be talking about today and follow along at home.

Active Cloud DB provides a minimalist REST API for the Datastore, exposing four of the Datastore API's operations: get, put, delete, and query. To speed up these operations, we also throw in two types of caching via the Memcache API: generational caching and write-through caching. The latter is the more familiar, so let's go over that one first. With write-through caching, write operations (put and delete) hit both Memcache and the Datastore, while read operations (get) read the Memcache version, and only hit the Datastore if the data isn't found in Memcache. Here's what the code for a put looks like:

func put(w http.ResponseWriter, r *http.Request) {
 keyName := r.FormValue("key")
 value := r.FormValue("val")

 c := appengine.NewContext(r)

 key := datastore.NewKey("Entity", keyName, 0, nil)
 entity := new(Entity)
 entity.Value = value

 result := map[string] string {
  "error":"",
 }
 if _, err := datastore.Put(c, key, entity); err != nil {
  result["error"] = fmt.Sprintf("%s", err)
 }

 // Set the value to speed up future reads - errors here aren't
 // that bad, so don't worry about them
 item := &memcache.Item{
  Key: keyName,
  Value: []byte(value),
 }
 memcache.Set(c, item)
 bumpGeneration(c)

 fmt.Fprintf(w, "%s", mapToJson(result))
}

And here's what the code for a get looks like:

func get(w http.ResponseWriter, r *http.Request) {
 keyName := r.FormValue("key")

 c := appengine.NewContext(r)

 result := map[string] string {
  keyName:"",
  "error":"",
 }

 if item, err := memcache.Get(c, keyName); err == nil {
  result[keyName] = fmt.Sprintf("%q", item.Value)
  fmt.Fprintf(w, "%s", mapToJson(result))
  return
 }

 key := datastore.NewKey("Entity", keyName, 0, nil)
 entity := new(Entity)

 if err := datastore.Get(c, key, entity); err == nil {
  result[keyName] = entity.Value

  // Set the value to speed up future reads - errors here aren't
  // that bad, so don't worry about them
  item := &memcache.Item{
   Key: keyName,
   Value: []byte(entity.Value),
  }
  memcache.Set(c, item)
 } else {
  result["error"] = fmt.Sprintf("%s", err)
 }

 fmt.Fprintf(w, "%s", mapToJson(result))
}

func query(w http.ResponseWriter, r *http.Request) {
 c := appengine.NewContext(r)

 cacheKey := getCacheKey(c)
 if item, err := memcache.Get(c, cacheKey); err != memcache.ErrCacheMiss {
  fmt.Fprintf(w, "%s", item.Value)
  return
 }

 q := datastore.NewQuery("Entity")
 result := map[string] string {}
 for t := q.Run(c); ; {
  var entity Entity
  key, err := t.Next(&entity)
  if err == datastore.Done {
   break
  }
  if err != nil {
   result["error"] = fmt.Sprintf("%s", err)
  }
  keyString := fmt.Sprintf("%s", key)
  result[keyString] = entity.Value
 }

 jsonResult := mapToJson(result)
 item := &memcache.Item{
  Key: cacheKey,
  Value: jsonResult,
 }
 memcache.Set(c, item)

 fmt.Fprintf(w, "%s", jsonResult)
}

With that, you now know how to use Memcache to cache Datastore accesses in Go. Of course, see our CloudComp paper for more details on the Python implementation and an evaluation. I hope that piqued your interest in Go and App Engine, so get coding!

One of the big problems I have as a programmer is that I don't have great UI skills, so whenever I have to make a UI for an app, it tends to look like crap. Lo and behold, to my amazement, that Twitter has taken a huge step towards fixing this problem for me with Bootstrap. Who would have guessed that all I needed was a single CSS file to make my web apps look pretty?

Having seen what Bootstrap can do (and if you haven't looked at that link, go check it out), I decided to take on a small project to give it a test drive. With that, let's talk about Active Cloud DB - Go Edition!

Last year I put together Active Cloud DB, a REST interface to any database that supports the Google App Engine Datastore API. That means that your client (in whatever language you want) can make REST calls to save and retrieve data without having to know how the underlying database works or how to optimize it. Naturally you may assume that the Google App Engine Datastore API is only supported by Google App Engine, and therefore you'd only be able to utilize a single database. But you'd be wrong! Since AppScale implements the Datastore API for all its databases, you can now save and retrieve data to any of the nearly dozen databases it supports. For example, you can save data from your Haskell application to Active Cloud DB and have Cassandra running via AppScale, which I think is pretty cool.

Active Cloud DB is written in Python, and I've been itching for a project to undertake with the sexy new Go App Engine runtime. It couldn't be better reflected in the name - Active Cloud DB - Go Edition! Here's how it all meshes together. All the Datastore communication is done in Go, so querying is done in the usual manner:

func query(w http.ResponseWriter, r *http.Request) {
 c := appengine.NewContext(r)
 q := datastore.NewQuery("Entity")

 result := map[string] string {}
 for t := q.Run(c); ; {
 var entity Entity
 key, err := t.Next(&entity)
 if err == datastore.Done {
 break
 }
 if err != nil {
 result["error"] = fmt.Sprintf("%s", err)
 }
 keyString := fmt.Sprintf("%s", key)
 result[keyString] = entity.Value
 }

 fmt.Fprintf(w, "%s", mapToJson(result))
}

As you can see from the above snippet, all the Go methods that we have output JSON that AJAX requests can easily consume. We have a small set of Go methods (put/get/delete/query) to keep the interface nice and simple. The app is simple enough to need only a single HTML page, and Bootstrap makes it look damn nice (if I do say so myself). It's got some nice pretty buttons, flash messages, and other things you've seen for years now but look just flawless with Bootstrap. To wrap it all up, there's AJAX that intercepts most of the button clicks and talks to the Go backend. Here's a screenshot of Active Cloud DB in action:

The Python version is still a bit ahead as far as feature support goes - it has caching optimizations from our CloudComp paper that speed up most of the operations via the Memcache API. We'll get to implementing those soon, and that will necessitate a follow-up post. In the meanwhile, go get coding with Go, App Engine, and Bootstrap!