Byzantine Reality

Searching for Byzantine failures in the world around us

Python's Ctypes Package

Last time we looked at an inherent problem in the book POSA 2: sometimes programmers want to use new fun languages to solve problems but have some legacy C code they need to use. POSA 2 claimed that the Wrapper Facade Design Pattern solves this problem, and it does if you’re programming in C++. But what about other languages?

We ended up with two technologies that the POSA 2 guys can now recommend to alleviate this problem. Python’s ctypes and SWIG. The former works only for Python  (as you could gather from the name), and the latter works for roughly a dozen languages but bears caveats. Of course many languages offer some way to use C code (e.g. the Java Native Interface for Java), but Python’s ctypes is so much easier to use than anything else I’ve currently seen that I’d put it as the model for others to follow thus far.

Let’s look at how to use ctypes to alleviate a problem in the same way that POSA 2 did. Suppose we have some legacy C code that fetches data blocks from a server and commits them after the user has made some change to it. However, the code is not thread-safe, and we need to make this change. The C code is really complicated (suppose it is although it isn’t, because it could be for your real-world example) and thus we don’t want to re-write it in Python, which we’ve chosen for this task because we heard about some library that makes this simple. That being said, here’s our “complicated” legacy C code (note I had to put spaces between the < and > to get stdio.h to show up, which you should remove when you try to compile this):

#include < stdio.h >

int* get_data(int block_number) {
   
// do some input validation here
    printf
("Retrieving block %d\n", block_number);
   
int* dummy_data;
   
return dummy_data;
}

void send_data(int block_number, int* data) {
   
// do some input validation here
    printf
("Committing block %d\n", block_number);
}

So our goal is to take this code and wrap it in some Python code to make it thread-safe. We begin by compiling our C code to a dynamic library as follows (this is for Mac OS X):

gcc -dynamiclib c-code.c -o c-code.A.dylib

If you’re on a Linux box, compiling your C code looks like this:

gcc -fPIC -c test.c
ld
-shared -soname libtest.so.1 -o libtest.so.1.0 -lc test.o

Now we can import the “ctypes” library and write the following code:

from ctypes import *
import thread

class RealWrapperFacade:
 
def __init__(self, libHandle, theLock):
   
# load our C library
    self
.cLibHandle = libHandle
    self
.lock = theLock
   
print "C library loaded successfully."

 
def getData(self, blockNumber):
    self
.lock.acquire()
    data
= self.cLibHandle.get_data(blockNumber)
    self
.lock.release()
   
return data

 
def sendData(self, blockNumber, data):
    self
.lock.acquire()
    self
.cLibHandle.send_data(blockNumber, data)
    self
.lock.release()

Even if you don’t know Python or ctypes, looking at the code shows what’s going on. Note that if you are on Linux, you need to replace the instance of “c-code.A.dylib” with “libtest.so.1.0″. So just like this tutorialshowed me, we will call “LoadLibrary” (done later) to load your C functions which returns a Python object whose methods are the C file’s functions. Then it’s as simple as calling those methods with the right parameters (ctypes takes care of a lot of this for you)! Note that our lock and the C library object reference are shared so that no matter how many instances of the Wrapper are made we’re still ok (I wanted to encapsulate the lock and reference in the class but Python made this difficult, let me know if you know how this is done). So now to use the Wrapper we do the following:

lock = thread.allocate_lock()
cLibHandle
= cdll.LoadLibrary('./c-code.A.dylib')

myWrap
= RealWrapperFacade(cLibHandle, lock)
myData
= myWrap.getData(10)
myWrap
.sendData(10, myData)

And now we’re done. This example is easily extensible to the one used in POSA 2 or whatever real-world example you’ve got, and makes the Wrapper Facade usable in Python in a simple fashion. Next time we’ll look at SWIG, which is more complicated but offers so many more languages that it may be what the Wrapper Facade truly needs.