{"id":650,"date":"2010-07-07T16:42:52","date_gmt":"2010-07-07T20:42:52","guid":{"rendered":"http:\/\/www.outofwhatbox.com\/blog\/?p=650"},"modified":"2010-07-09T16:50:05","modified_gmt":"2010-07-09T20:50:05","slug":"python-decorator-classes-on-the-edge","status":"publish","type":"post","link":"https:\/\/www.outofwhatbox.com\/blog\/2010\/07\/python-decorator-classes-on-the-edge\/","title":{"rendered":"Python: Decorator Classes On The Edge"},"content":{"rendered":"<p>OK, I cheated.<\/p>\n<p>In yesterday&#8217;s post on writing <a href=\"http:\/\/www.outofwhatbox.com\/blog\/2010\/07\/python-decorating-with-class\/\">decorator classes that decorate methods<\/a>, I left out two edge cases that can&#8217;t be completely ignored: static methods and class methods.<\/p>\n<p>To illustrate, I&#8217;ll start <a href=\"http:\/\/www.outofwhatbox.com\/blog\/2010\/07\/python-decorating-with-class\/#WhereILeftOff\">where I left off<\/a> yesterday, adding a decorated class method and a decorated static method to the example:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nimport types\r\n\r\nclass DebugTrace(object):\r\n    def __init__(self, f):\r\n        print(&quot;Tracing: {0}&quot;.format(f.__name__))\r\n        self.f = f\r\n\r\n    def __get__(self, obj, ownerClass=None):\r\n        # Return a wrapper that binds self as a method of obj (!)\r\n        return types.MethodType(self, obj)\r\n\r\n    def __call__(self, *args, **kwargs):\r\n        print(&quot;Calling: {0}&quot;.format(self.f.__name__))\r\n        return self.f(*args, **kwargs)\r\n\r\n\r\nclass Greeter(object):\r\n    instances = 0\r\n\r\n    def __init__(self):\r\n        Greeter.instances += 1\r\n        self._inst = Greeter.instances\r\n\r\n    @DebugTrace\r\n    def hello(self):\r\n        print(&quot;*** Greeter {0} says hello!&quot;.format(self._inst))\r\n\r\n    @DebugTrace\r\n    @classmethod\r\n    def classHello(cls, to):\r\n        print(&quot;*** The {0} class says hello to {1}&quot;.format(cls.__name__, to))\r\n\r\n    @DebugTrace\r\n    @staticmethod\r\n    def staticHello(to):\r\n        print(&quot;*** Something says hello to &quot; + to)\r\n\r\n\r\n@DebugTrace\r\ndef greet():\r\n    g = Greeter()\r\n    g2 = Greeter()\r\n    g.hello()\r\n    g2.hello()\r\n    Greeter.staticHello(&quot;you&quot;)\r\n    Greeter.classHello(&quot;everyone&quot;)\r\n\r\ngreet()\r\n<\/pre>\n<p>Running this gives an error:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\n\r\nTracing: hello\r\nTraceback (most recent call last):\r\n  File &quot;DecoratorExample.py&quot;, line 17, in &lt;module&gt;\r\n    class Greeter(object):\r\n  File &quot;DecoratorExample.py&quot;, line 29, in Greeter\r\n    @classmethod\r\n  File &quot;DecoratorExample.py&quot;, line 5, in __init__\r\n    print(&quot;Tracing: {0}&quot;.format(f.__name__))\r\nAttributeError: 'classmethod' object has no attribute '__name__'\r\n<\/pre>\n<p>Just for this example, I&#8217;ll try removing the &#8220;<strong>Tracing<\/strong>&#8221; print call; but still no joy:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nCalling: greet\r\nCalling: hello\r\n*** Greeter 1 says hello!\r\nCalling: hello\r\n*** Greeter 2 says hello!\r\nTraceback (most recent call last):\r\n  File &quot;DecoratorExample.py&quot;, line 48, in &lt;module&gt;\r\n    greet()\r\n  File &quot;DecoratorExample.py&quot;, line 14, in __call__\r\n    return self.f(*args, **kwargs)\r\n  File &quot;DecoratorExample.py&quot;, line 45, in greet\r\n    Greeter.staticHello(&quot;you&quot;)\r\n  File &quot;DecoratorExample.py&quot;, line 10, in __get__\r\n    return types.MethodType(self, obj)\r\nTypeError: self must not be None\r\n<\/pre>\n<p>The essential problem is that class methods and static methods are not callable.<a href=\"#DecoratorEdge1\" name=\"DecoratorEdge1Ref\"><sup>1<\/sup><\/a> There&#8217;s an easy enough workaround: always use <code>@staticmethod<\/code> or <code>@classmethod<\/code> as the <em>outermost<\/em> (i.e., last) decorator in a sequence, as in:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n    @classmethod\r\n    @DebugTrace\r\n    def classHello(cls, to):\r\n        print(&quot;*** The Greeter class says hello to &quot; + to)\r\n\r\n    @staticmethod\r\n    @DebugTrace\r\n    def staticHello(to):\r\n        print(&quot;*** Something says hello to &quot; + to)\r\n<\/pre>\n<p>That produces the desired result:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nTracing: hello\r\nTracing: classHello\r\nTracing: staticHello\r\nTracing: greet\r\nCalling: greet\r\nCalling: hello\r\n*** Greeter 1 says hello!\r\nCalling: hello\r\n*** Greeter 2 says hello!\r\nCalling: staticHello\r\n*** Something says hello to you\r\nCalling: classHello\r\n*** The Greeter class says hello to everyone\r\n<\/pre>\n<p>But suppose we really, really <em>need<\/em> to decorate an already-decorated classmethod or staticmethod. The key lies again in the <a href=\"http:\/\/docs.python.org\/reference\/datamodel.html#invoking-descriptors\">descriptor protocol<\/a>.<\/p>\n<p>First, we need to modify the decorator&#8217;s <code>__init__<\/code> method. (Note that the <em>only<\/em> reason that we need to modify <code>__init__<\/code> is to find the name of the classmethod or staticmethod that&#8217;s being decorated. If we didn&#8217;t produce the &#8220;<strong>Tracing:<\/strong>&#8221; output, we could leave <code>__init__ <\/code>alone.)<\/p>\n<p>The new <code>__init__<\/code> method detects whether the passed &#8220;function&#8221; has a <code>__call__<\/code> method. If it doesn&#8217;t, then it&#8217;s reasonable to assume that it&#8217;s a classmethod or a staticmethod. Calling the object&#8217;s <code>__get__<\/code> method returns a function object, from which we can get the function name:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n    def __init__(self, f):\r\n        self.f = f\r\n        if hasattr(f, &quot;__call__&quot;):\r\n            name = self.f.__name__\r\n        else:\r\n            # f is a class or static method.\r\n            tmp = f.__get__(None, f.__class__)\r\n            name = tmp.__name__\r\n        print(&quot;Tracing: {0}&quot;.format(name))\r\n<\/pre>\n<p>In the decorator&#8217;s <code>__get__<\/code> method, we&#8217;ll know that we&#8217;re dealing with a staticmethod or classmethod if the passed <code>obj<\/code> has the value <code>None<\/code>. If that&#8217;s the case, then we make a one-time adjustment to <code>self.f<\/code>, ensuring that it points to the underlying function.<\/p>\n<p><em>Wait\u00e2\u20ac\u201dwhy didn&#8217;t we do this in <code>DebugTrace.__init__<\/code>?<\/em> It may seem redundant, but the call to <code>f.__get__<\/code> that we made in <code>DebugTrace.__init__<\/code> doesn&#8217;t count: that call didn&#8217;t specify the class that <code>f<\/code> actually belongs to. (<em>Any<\/em> class works for the purpose of getting the function&#8217;s name.) Now that we&#8217;re in <code>DebugTrace.__get__<\/code>, we know via the <code>ownerClass<\/code> parameter the class that <code>self.f<\/code> is associated with. This class may make its way into a classmethod call (e.g., the call to <code>Greeter.classHello<\/code>), so it matters that we get it right.<\/p>\n<p>Note that we return <code>self<\/code> in this case. We don&#8217;t want to create a new method object for classmethods or staticmethods; just calling <code>self.__call__<\/code> will call the method appropriately.<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n    def __get__(self, obj, ownerClass=None):\r\n        if obj is None:\r\n            f = self.f\r\n            if not hasattr(f, &quot;__call__&quot;):\r\n                self.f = f.__get__(None, ownerClass)\r\n            return self\r\n        else:\r\n            # Return a wrapper that binds self as a method of obj (!)\r\n            return types.MethodType(self, obj)\r\n<\/pre>\n<div class=\"oowbnotice\">Setting <code>self.f<\/code> as above might raise thread-safety issues, especially if you don&#8217;t want to rely on the <a href=\"http:\/\/effbot.org\/zone\/thread-synchronization.htm\">atomicity of modifying a dict<\/a> in-place. Borrowing from <a href=\"http:\/\/blog.ianbicking.org\/2008\/10\/24\/decorators-and-descriptors\/\">Ian Bicking&#8217;s solution<\/a>, which returns a copy of the decorator for each call to <code>__get__<\/code>, can help us dodge the concurrency bullet. We&#8217;d replace <\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n            return self\r\n<\/pre>\n<p>with<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n            return self.__class__(self.f)\r\n<\/pre>\n<p>However, this results in any side effects in the decorator&#8217;s __init__ method being re-executed for every call to the decorated method. Note the additional &#8220;<strong>Tracing:<\/strong>&#8221; lines in the output here:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nTracing: hello\r\nTracing: classHello\r\nTracing: staticHello\r\nTracing: greet\r\nCalling: greet\r\nCalling: hello\r\n*** Greeter 1 says hello!\r\nCalling: hello\r\n*** Greeter 2 says hello!\r\nTracing: staticHello\r\nCalling: staticHello\r\n*** Something says hello to you\r\nTracing: classHello\r\nCalling: classHello\r\n*** The Greeter class says hello to everyone\r\n<\/pre>\n<p>Another option, of course, is to use a mutex around the statement that modifies <code>self.f<\/code>.\n<\/div>\n<p>The decorator&#8217;s <code>__call__<\/code> method is unchanged from yesterday&#8217;s example. As before, it simply prints out the desired trace message, then invokes <code>self.f<\/code>.<\/p>\n<p>Here&#8217;s the entire decorator, as revised:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nimport types\r\n\r\nclass DebugTrace(object):\r\n    def __init__(self, f):\r\n        self.f = f\r\n        if hasattr(f, &quot;__call__&quot;):\r\n            name = self.f.__name__\r\n        else:\r\n            # f is a class or static method\r\n            tmp = f.__get__(None, f.__class__)\r\n            name = tmp.__name__\r\n        print(&quot;Tracing: {0}&quot;.format(name))\r\n\r\n    def __get__(self, obj, ownerClass=None):\r\n        if obj is None:\r\n            f = self.f\r\n            if not hasattr(f, &quot;__call__&quot;):\r\n                self.f = f.__get__(None, ownerClass)\r\n            return self\r\n        else:\r\n            # Return a wrapper that binds self as a method of obj (!)\r\n            return types.MethodType(self, obj)\r\n\r\n    def __call__(self, *args, **kwargs):\r\n        print(&quot;Calling: {0}&quot;.format(self.f.__name__))\r\n        return self.f(*args, **kwargs)\r\n\r\n\r\nclass Greeter(object):\r\n    instances = 0\r\n\r\n    def __init__(self):\r\n        Greeter.instances += 1\r\n        self._inst = Greeter.instances\r\n\r\n    @DebugTrace\r\n    def hello(self):\r\n        print(&quot;*** Greeter {0} says hello!&quot;.format(self._inst))\r\n\r\n    @DebugTrace\r\n    @classmethod\r\n    def classHello(cls, to):\r\n        print(&quot;*** The {0} class says hello to {1}&quot;.format(cls.__name__, to))\r\n\r\n    @DebugTrace\r\n    @staticmethod\r\n    def staticHello(to):\r\n        print(&quot;*** Something says hello to &quot; + to)\r\n\r\n\r\n@DebugTrace\r\ndef greet():\r\n    g = Greeter()\r\n    g2 = Greeter()\r\n    g.hello()\r\n    g2.hello()\r\n    Greeter.staticHello(&quot;you&quot;)\r\n    Greeter.classHello(&quot;everyone&quot;)\r\n\r\ngreet()\r\n<\/pre>\n<p>I&#8217;ve tested this with Python 2.6, 2.7, and 3.1.<\/p>\n<hr \/>\n<p><a href=\"#DecoratorEdge1Ref\" name=\"DecoratorEdge1\"><sup>1<\/sup><\/a> Without taking a deep dive into Python&#8217;s history, I couldn&#8217;t say <em>why<\/em> they&#8217;re not callable. But it does seem that class methods and static methods <a href=\"http:\/\/mail.python.org\/pipermail\/python-dev\/2006-March\/062014.html\">were never intended to be used frequently<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yesterday&#8217;s entry on Python decorator classes left out two related edge cases: classmethod objects and staticmethod objects.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[30],"tags":[50,31,32],"_links":{"self":[{"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/posts\/650"}],"collection":[{"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/comments?post=650"}],"version-history":[{"count":4,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/posts\/650\/revisions"}],"predecessor-version":[{"id":658,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/posts\/650\/revisions\/658"}],"wp:attachment":[{"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/media?parent=650"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/categories?post=650"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.outofwhatbox.com\/blog\/wp-json\/wp\/v2\/tags?post=650"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}