Personally I would not consider Python as my first preference to build a complex system. The most important reason is the lack of type checking, so it likely causes some inter-component problems not to manifest until the system is actually running. In addition, if you make some syntactic mistakes such as mistyping variable names, then good luck… You definitely know how annoying it is to debug those problems.

On the other hand, if there is a system written in Python, and you need to do some deep analysis on it, you will probably appreciate those developers who decided to use Python in the first place.

Recently I have been doing some analysis for OpenStack, which is a very popular open-source cloud system, and yes, it is written in Python. So why does Python become so adorable when you analyze a complex system? My answer is Decorators.

Python Decorators really make tasks of instrumentation and interception way easier than in other programming languages. Now I can just insert a simple annotation to some target places in the OpenStack source code via sed, and do a lot of amazing stuff without changing the original source code too much. For instance, it is quite easy to extract communication flows among OpenStack components by decorating communication modules.

The idea of Decorators sourced from the aspect-oriented programming (AOP), and with the dynamism of Python, it becomes easier to implement. Traditionally, if you want to implement Decorators in C# or Java, you will be facing a quite heavy task – which involves writing a bunch of proxy classes, interfaces, and everything else you need to implement the classic factory pattern and the decorator pattern. These patterns are pretty elegant, but too heavy for most scenarios. In contrast, with the ability of meta-programming provided by Python, you can directly modify classes and functions at runtime, and do the magic behind those simple annotations without defining additional classes and interfaces.

There are multiple ways to implement Decorators in Python. Let’s use the following simple and classic example to show some typical implementations.

@class_dec
class Dummy(object):
    def __init__(self, name='World'):
        self.name = name
    @func_dec
    def hello(self):
        print "Hello %s!" % self.name

There are two Decorators here: @class_dec for the class Dummy, and @func_dec for the function hello. Let’s start from implementing @func_dec.

We can do @func_dec in either a class or a function, depending on personal favorites:

""" Implement @func_dec using a class. """
class func_dec(object):
    def __init__(self, fn):
        self.fn = fn
    def __call__(self, inst):
        # Do whatever you want before and after the real function call.
        self.fn(inst)
    def __get__(self, inst, insttype):
        def function(*args, **kwargs):
            self.fn(inst, *args, **kwargs)
        return function

Some people may prefer the above implementation simply because it looks more understandable in an object-oriented fashion. The decoration is mostly done by overriding __call__ and __get__. However, the complexity here is about the “self” object, which is actually referred to different objects in the contexts of Dummy and func_dec. That is why we need __get__ to ensure the method hello receives the correct object.

If you feel dizzy on the above implementation, just try understand the one below, which is purely functional, nice and simple.

""" Implement @func_dec using a function. """
def func_dec(fn):
    def function(self):
        """ Do whatever you want. """
        fn(self)
    return function

To implement @class_func, a nice solution would be the one below:

""" Implement @class_dec using class inheritance. """
def class_dec(cls):
    class NewClass(cls):
        def hook(self):
            """ Do whatever you want. """
            self.name = "Lester"
        def hello(self):
            self.hook()
            super(NewClass, self).hello()
    return NewClass

Looks interesting, right? Actually there could be more ways to implement Decorators. In future posts, I plan to demonstrate some solutions on instrumenting OpenStack.