Out Of What Box?

As a fairly new Python developer, my first attempt at decorators hit a snag: my simple class-based decorator failed when decorating a method. I got around the immediate problem by rewriting the decorator as a function. Yet the episode left me wondering if there were some way to fix the class-based decorator to work when applied to methods. I’ve found what seems like an elegant solution, and picked up a better understanding of decorators and descriptors in the process.

Here’s an example that illustrates the original problem. DebugTrace is the decorator class:

class DebugTrace(object):
    def __init__(self, f):
        print("Tracing: {0}".format(f.__name__))
        self.f = f

    def __call__(self, *args, **kwargs):
        print("Calling: {0}".format(self.f.__name__))
        return self.f(*args, **kwargs)


class Greeter(object):
    instances = 0

    def __init__(self):
        Greeter.instances += 1
        self._inst = Greeter.instances

    @DebugTrace
    def hello(self):
        print("*** Greeter {0} says hello!".format(self._inst))


@DebugTrace
def greet():
    g = Greeter()
    g2 = Greeter()
    g.hello()
    g2.hello()


greet()

Running this with Python 2.6 or 3.1 results in an error:

Tracing: hello
Tracing: greet
Calling greet
Calling hello
Traceback (most recent call last):
  File "./DecoratorExample.py", line 31, in <module>
    greet()
  File "./DecoratorExample.py", line 8, in __call__
    return self.f(*args, **kwargs)
  File "./DecoratorExample.py", line 27, in greet
    g.hello()
  File "./DecoratorExample.py", line 8, in __call__
    return self.f(*args, **kwargs)
TypeError: hello() takes exactly 1 argument (0 given)

The output explains the problem. DebugTrace was instantiated only twice: once for Greeter.hello, and once for greet. That’s a reminder that decoration occurs during compile time, not run time. Accordingly, DebugTrace‘s reference to Greeter.hello represents an unbound function—it doesn’t reference any Greeter instances. So no ‘self’ argument was passed into the call to Greeter.hello; hence the TypeError.

All object-oriented languages that are worth knowing (and many that aren’t) allow container objects to redirect access to their contained objects. But of the languages that I’ve used, Python is unique in allowing object attributes to redirect calls that attempt to access them. Objects that implement this capability are called descriptors¹. As we’ll soon see, function objects, which Python uses to implement methods, are descriptors.

To see where descriptors come into play, let’s look at the calling sequences for different versions of Greeter.hello. Here’s the rough sequence before Greeter.hello was decorated:

• The interpreter searches for an attribute named hello on the Greeter instance, finding it in the Greeter class object.
• The value of the Greeter.hello attribute (a function object) has a __get__ method, making it a descriptor, so that __get__ method is invoked. It’s passed a reference to the Greeter instance (obj) through which Greeter.hello was called.
• The function object’s __get__ method creates and returns another callable object, which we’ll refer to as a (bound) method object. The method object references both the Greeter.hello function and obj.
• The interpreter invokes the method object, passing it the arguments from the call to Greeter.hello (an empty list.) The method object then calls Greeter.hello, passing obj as the first argument, followed by the (empty) argument list.

When Greeter.hello is decorated with the DebugTracer class as shown above, a call to Greeter.hello runs more or less like this:

• The interpreter searches for an attribute named hello on the Greeter instance, finding it in the Greeter class object.
• The value of the Greeter.hello attribute is an instance of DebugTrace. This isn’t a function object, and it doesn’t have a __get__ method, but it does have a __call__ method. That __call__ method is invoked with the empty argument list.
• DebugTrace.__call__ then calls Greeter.hello with the empty argument list.
• Since Greeter.hello was looking for a single argument (self), rather than an empty argument list, a TypeError is raised.

To fix DebugTrace, I turned it into a descriptor class, adding a __get__ method that fills the same role as a function object’s __get__ method. However, this method binds the Greeter instance to the callable DebugTrace object.

import types
# ...
    def __get__(self, obj, ownerClass=None):
        # Return a wrapper that binds self as a method of obj (!)
        return types.MethodType(self, obj)

Compare the new calling sequence for Greeter.hello to the sequence prior to decoration:

• The interpreter finds an attribute named hello in the Greeter class object.
• The value of the Greeter.hello attribute is an instance of DebugTrace, which is now a descriptor. DebugTrace.__get__ is called, with obj (the Greeter instance) passed as one of the arguments.
• DebugTrace.__get__ creates and returns a method object. The method object references both the DebugTrace instance and obj.
• The interpreter invokes the method object, passing it the arguments from the call to Greeter.hello (an empty list.) The method object then calls DebugTrace.__call__. That in turn calls Greeter.hello, passing obj as the first argument, followed by the (empty) argument list.

It’s worth noting that DebugTrace.__get__ is only invoked when accessing a DebugTrace object through an object’s class dictionary. Hence its presence has no effect on functions that aren’t methods, such as greet.

You can see the full, working example here (click on the “show source” link to view) :

import types

class DebugTrace(object):
    def __init__(self, f):
        print("Tracing: {0}".format(f.__name__))
        self.f = f

    def __get__(self, obj, ownerClass=None):
        # Return a wrapper that binds self as a method of obj (!)
        return types.MethodType(self, obj)

    def __call__(self, *args, **kwargs):
        print("Calling: {0}".format(self.f.__name__))
        return self.f(*args, **kwargs)


class Greeter(object):
    instances = 0

    def __init__(self):
        Greeter.instances += 1
        self._inst = Greeter.instances

    @DebugTrace
    def hello(self):
        print("*** Greeter {0} says hello!".format(self._inst))


@DebugTrace
def greet():
    g = Greeter()
    g2 = Greeter()
    g.hello()
    g2.hello()


greet()

Executing the new version gives the desired output:

Tracing: hello
Tracing: greet
Calling greet
Calling hello
*** Greeter 1 says hello!
Calling hello
*** Greeter 2 says hello!

¹ I found the term “descriptor” to be somewhat confusing at first. A descriptor doesn’t really describe anything other than itself. To be fair, I don’t have any better suggestions. (“Redirector” ?) Naming is often one of the hardest challenges in software design, at least when it’s done right.