{"id":217,"date":"2014-05-26T00:34:07","date_gmt":"2014-05-26T00:34:07","guid":{"rendered":"http:\/\/96.126.106.214\/?p=217"},"modified":"2023-11-26T01:00:18","modified_gmt":"2023-11-26T01:00:18","slug":"interface-driven-programming-in-python","status":"publish","type":"post","link":"https:\/\/codingismycraft.blog\/index.php\/2014\/05\/26\/interface-driven-programming-in-python\/","title":{"rendered":"Interface Driven Programming In Python"},"content":{"rendered":"

If it looks like a duck, quacks like a duck and walks like a duck, it’s a duck<\/strong><\/p>\n

Probably the greatest feature of python is its dynamic <\/em>nature.\u00a0 By this we mean that variable names are not bound to types and they also can be assigned at run time, this concept is also know as duck typing<\/em> and an example of it is the following:<\/p>\n

\r\n#!\/usr\/bin\/python\r\nclass Person(object):\r\n        pass\r\n\r\n\r\ndef print_name(obj):\r\n    ''' Expects obj to expose a property name'''\r\n    print obj.name\r\n\r\np = Person()\r\n# we add name dynamically\r\np.name = 'John'\r\n\r\n# print name\r\nprint_name(p)\r\n\r\n# we can change name from a string to anyother type\r\np.name = 123\r\n\r\n# print name continues to work\r\nprint_name(p)\r\n\r\n<\/pre>\n
At this point, you should notice that the print_name<\/em> function, silently is making the assumption that obj<\/em> is supporting the attribute name. Of course this will not always be the case since we can pass any kind of a variable to the function causing an exception to be thrown.<\/p>\n
It’s easier to ask forgiveness than permission<\/strong><\/p>\n
A very common solution for this kind of problem in python is the following:<\/p>\n
\r\ndef print_name(obj):\r\n    try:\r\n        print obj.name\r\n    except TypeError:\r\n        pass\r\n<\/pre>\n
This pattern is commonly known as It’s easier to ask forgiveness than permission<\/em>. In other words we assume that the passed in object indeed supports name<\/em> and we try to use it, if it turns out that it not supported an TypeError exception will be raised and caught allowing our program to continue to function.<\/p>\n
Look before you leap<\/strong><\/p>\n
Another way to achieve similar behaviour, is to apply a technique known as Look before you leap<\/em> meaning that, we verify that an object is supporting a specific operation before we try to use it. The following code is an example of this technique:<\/p>\n
\r\ndef print_name(obj):\r\n    if isinstance(obj, Person):\r\n        print obj.name\r\n<\/pre>\n
Another way to achieve a similar effect is the following:<\/p>\n
\r\ndef print_name(obj):\r\n    if hasattr(obj, "name"):\r\n        print obj.name\r\n<\/pre>\n
Possible Problematic Behaviour<\/strong><\/p>\n
Although the solutions we have seen so far, can allow a program to continue its execution without breaking in case of a wrong argument, we still can see some problematic behaviour that might cause side effects, possibly leading to bogus behaviour.<\/p>\n
For example, checking for the type of the passed object will ignore any other object than Person<\/em> defining name which might not be our intention. Also using the hasattr<\/em> again might lead to cases like the following:<\/p>\n
\r\nclass Person(object):\r\n    name = 'To be defined'\r\n\r\ndef print_name(obj):\r\n    if hasattr(obj, "name"):\r\n        print obj.name\r\n\r\nprint_name(Person)\r\n<\/pre>\n
The output of this snippet will be:<\/p>\n
To be defined<\/p><\/blockquote>\n
while what would have seen more reasonable in this case would have been something like this:<\/p>\n
\r\n\r\ndef print_name(obj):\r\n    if hasattr(obj, "__name__"):\r\n        print obj. __name__\r\n\r\n# do your stuff ...\r\n<\/pre>\n
With the following output:<\/p>\n
Person<\/p><\/blockquote>\n
Another example of undesired behaviour can be demonstrated in the following example:<\/p>\n
\r\nclass Person:\r\n    def __init__(self, name):\r\n        self._name = name\r\n\r\n    def get_name(self):\r\n        return self._name;\r\n\r\ndef print_name(obj):\r\n    if isinstance(obj, Person):\r\n        print obj.get_name()\r\n\r\n\r\nperson = Person('john')\r\n\r\n# print_name will work as expected\r\nprint_name(person) \r\n\r\n# we delete name from person\r\ndel person._name    \r\n\r\n# now we will have an unhandled exception!\r\nprint_name(person) \r\n\r\n<\/pre>\n
Documentation Issues<\/strong><\/p>\n
More than the problems I discussed so far as far as duck typing<\/em> is going, there are also issues with proper documentation. Given the flexibility of dynamic typing, chances are that users of specific module will have to delve in directly to the source code trying to understand its details and how it is working. A developer who just started using a system, is very possible to misuse it and even start use attributes consisting implementation details breaking encapsulation, making the maintenance of the code difficult and error prone.<\/p>\n
The problem became obvious since python’s early stages<\/strong><\/p>\n
As early as 2001, we can find proposals like: PEP 245<\/a> and PEP 246<\/a> trying to address the problem we are discussing here.<\/p>\n
The discussion was very long and controversial since it was about a change that seemed not pythonic<\/em> at all!<\/p>\n
A few years latter Guido van van Rossum came with his notorious blog posting:Interfaces or Abstract Base Classes?<\/a> which led to PEP 3119<\/a> introducing the concept of Abstract Base Classes<\/em> as we use it today.<\/p>\n
In this posting we will not deal with historical aspects of ABCs (Abstract Base Classes<\/em>) nor with their non pythonic nature but we will focus in understanding their implementation mechanics and details, so keep on reading to the next page!<\/p>\n

\nA simple example leading to the use of an Abstract Base Class<\/strong>
\nTo better understand the need of an Abstract Base Class, let’s start with a simple example consisting of a base class called Shape which defines a single method called get_area and a couple of its descendants called Rectangle and Circle:<\/p>\n
\r\n\r\nclass Shape(object):\r\n    def get_area(self):\r\n        pass\r\n\r\nclass Rectangle(Shape):\r\n    def __init__(self, width, height):\r\n        self.width = width\r\n        self.height = height\r\n\r\n    def get_area(self):\r\n        return self.width * self.height\r\n\r\nclass Circle(Shape):\r\n    def __init__(self, radius):\r\n        self.radius = radius\r\n\r\n    def get_area(self):\r\n        return self.radius **2 * 3.14\r\n\r\n<\/pre>\n
This code looks good from the first glance, but as we inspect it closer we can see some possible pitfalls. For example lets assume that we need a function with the name find_total_area receiving a collection of shapes and returning the total area covered by all of them, something like the following:<\/p>\n
\r\ndef find_total_area(shapes):\r\n    return sum(shape.get_area() for shape in shapes )\r\n<\/pre>\n
A possible use of this function could be the following:<\/p>\n
\r\nshapes = [Rectangle(10.2, 8.3), Circle(6.2)]\r\nprint find_total_area(shapes)\r\n<\/pre>\n
which will correctly output the following:<\/p>\n
205.3616\r\n<\/pre>\n
Problems with this implementation<\/strong><\/p>\n
A problem we can immediately see with the implementation of find_total_area has to do with the user passing a list of objects not supporting the get_area functionality. For example there might exist an Ellipse class implemented as follows:<\/p>\n
\r\nclass Ellipse:\r\n    def __init__(self, a, b):\r\n        self.a, self.b = a,b\r\n\r\n    def calculate_area(self):\r\n        return self.a * self.b * 3.14\r\n<\/pre>\n
and our user now might do something like the following:<\/p>\n
\r\nshapes = [Rectangle(10.2, 8.3), Circle(6.2), Ellipse(3,8) ]\r\nprint find_total_area(shapes)\r\n<\/pre>\n
As expected, in this case the find_total_area will throw an exception since the Ellipse object does not support the get_area<\/em> function…<\/p>\n
One (not so good though!) way of fixing this problem, would had been to re-implement the find_total_area function checking for each shape to see if it supports the get_area functionality, as can be seen here:<\/p>\n
\r\ndef find_total_area(shapes):\r\n    def find_total_area(shapes):\r\n    return sum(shape.get_area() for shape in shapes \r\n                                if hasattr(shape, 'get_area')\r\n              )\r\n\r\n<\/pre>\n
Now our code will run happily without throwing any exceptions, but we still have some problems. First of all we silently ignore the Ellipse in our calculations and secondly we are making a silent assumption that the get_area returns a numeric value that can be accumulated by sum. This can become a problem for our implementation, as can be seen in the following user of our code:<\/p>\n
\r\nshapes = [Rectangle(10.2, 8.3), Circle(6.2), Shape() ]\r\nprint find_total_area(shapes)\r\n<\/pre>\n
In this case, our modified version of find_total_area, will try to call the get_area as it is implemented in Shape and of course throw an exception since it is returning a None value that cannot be added to the areas of the other shapes.<\/p>\n
A better solution to our problem can be implemented by using an Abstract Base Class, which we will see in the next page…<\/p>\n
<\/p>\n
Defining an Abstract Base Class<\/h2>\n
The definition of a ABC is straightforward besides the fact that is using the not so widely concept of metaclasses<\/em>. The developer does not need to understand exactly what is happening in the background although in one of my next posts I will talk more about metaclasses<\/em> and how they can be used.<\/p>\n
For now you can simply follow some implementation guidelines and understand what ABCs are, strictly from the user<\/em> scope of view.<\/p><\/blockquote>\n
Let’s now go back to our example and see how we can improve its functionality..<\/p>\n
Our original base class looked like this:<\/p>\n
\r\nclass Shape(object):\r\n    def get_area(self):\r\n        pass\r\n<\/pre>\n
Assuming we are using python 2.7, to convert Shape to an ABC is as easy as follows:<\/p>\n
\r\nfrom abc import ABCMeta, abstractmethod\r\n\r\nclass Shape(object):\r\n    __metaclass__= ABCMeta\r\n\r\n    @abstractmethod\r\n    def get_area(self):\r\n        pass\r\n<\/pre>\n
As you can see the necessary steps to follow are the following:<\/p>\n
    \n
      \n
        \n
      • Import ABCMeta and abstractmethod from abc<\/li>\n
      • Specify the metaclass of the class as ABCMeta<\/li>\n
      • Decorate the functions that we need to become abstract with the @abstractmethod decorator.<\/li>\n<\/ul>\n<\/ul>\n<\/ul>\n
        That’s it! Shape has now become an abstract class!<\/p>\n
        How an ABC differs from any other class<\/h2>\n
        An ABC cannot be instantiated directly<\/strong><\/p>\n
        After defining Shape as an ABC, code like the following:<\/p>\n
        \r\nsome_shape = Shape()\r\n<\/pre>\n
        will fail to execute throwing the following TypeError:<\/p>\n
        TypeError: Can't instantiate abstract class Shape\r\nwith abstract methods get_area\r\n<\/pre>\n
        An ABC is meant to serve only as a base class leaving the implementation of its abstract members to its descendants. This does not mean that an ABC cannot contain implementation details, in other words we can define members of the ABC as in any other class, assuming that they are not decorated<\/em> by the abstractmethod (or absractproperty that we will see later) decorators.<\/p>\n
        An ABC is forcing its descendants to implement its abstract parts<\/strong><\/p>\n
        A descendant of an ABC is forced to implement its abstract parts. Code like the following:<\/p>\n
        \r\nclass BogusShape(Shape):\r\n    pass\r\n\r\nb = BogusShape()\r\n\r\n<\/pre>\n
        Will fail with the following exception:<\/p>\n
        TypeError: Can't instantiate abstract class BogusShape with \r\nabstract methods get_area\r\n<\/pre>\n
        Note that the TypeError exception will be thrown at the time we will try to instantiate BogusShape. In other languages like C++ or Java the error condition would had been caught at compile time.<\/p><\/blockquote>\n
        Defining descendants of an ABC<\/h2>\n
        A descendant of an ABC needs to implement all of its abstract parts. Failure to do so will cause an exception to be thrown at run time. For example the following two classes are valid implementers of the ABC Shape<\/em>:<\/p>\n
        \r\n\r\nclass Rectangle(Shape):\r\n    def __init__(self, width, height):\r\n        self.width = width\r\n        self.height = height\r\n\r\n    def get_area(self):\r\n        return self.width * self.height\r\n\r\nclass Circle(Shape):\r\n    def __init__(self, radius):\r\n        self.radius = radius\r\n\r\n    def get_area(self):\r\n        return self.radius **2 * 3.14\r\n\r\n<\/pre>\n
        As you can see, there is nothing specific to the fact that the base class in an ABC.<\/p>\n
        Using this approach our original code can be written as follows:<\/p>\n
        \r\n#!\/usr\/bin\/python\r\n\r\nfrom abc import ABCMeta, abstractmethod\r\n\r\nclass Shape(object):\r\n    __metaclass__= ABCMeta\r\n\r\n    @abstractmethod\r\n    def get_area(self):\r\n        pass\r\n\r\nclass Rectangle(Shape):\r\n    def __init__(self, width, height):\r\n        self.width = width\r\n        self.height = height\r\n\r\n    def get_area(self):\r\n        return self.width * self.height\r\n\r\nclass Circle(Shape):\r\n    def __init__(self, radius):\r\n        self.radius = radius\r\n\r\n    def get_area(self):\r\n        return self.radius **2 * 3.14\r\n\r\n\r\nclass Ellipse:\r\n    def __init__(self, a, b):\r\n        self.a, self.b = a,b\r\n\r\n    def get_area(self):\r\n        return self.a * self.b * 3.14\r\n\r\n\r\ndef find_total_area(shapes):\r\n    return sum(shape.get_area() for shape in shapes \r\n                                if isinstance(shape, Shape))\r\n\r\nshapes = [Rectangle(10.2, 8.3), Circle(6.2), Ellipse(8,12)]\r\nprint find_total_area(shapes)\r\n\r\n<\/pre>\n
        An interesting feature of ABCs is that if we want Ellipse to be considered as a Shape as well, it is possible to do even without deriving from Shape, as long it implements get_area. This can be accomplished by adding the following line of code:<\/p>\n
        \r\nShape.register(Ellipse)\r\n<\/pre>\n
        Now our program becomes:<\/p>\n
        \r\n#!\/usr\/bin\/python\r\n\r\nfrom abc import ABCMeta, abstractmethod\r\n\r\nclass Shape(object):\r\n    __metaclass__= ABCMeta\r\n\r\n    @abstractmethod\r\n    def get_area(self):\r\n        pass\r\n\r\nclass Rectangle(Shape):\r\n    def __init__(self, width, height):\r\n        self.width = width\r\n        self.height = height\r\n\r\n    def get_area(self):\r\n        return self.width * self.height\r\n\r\nclass Circle(Shape):\r\n    def __init__(self, radius):\r\n        self.radius = radius\r\n\r\n    def get_area(self):\r\n        return self.radius **2 * 3.14\r\n\r\n\r\nclass Ellipse:\r\n    def __init__(self, a, b):\r\n        self.a, self.b = a,b\r\n\r\n    def get_area(self):\r\n        print 'Ellipse get_area was called!'\r\n        return self.a * self.b * 3.14\r\n\r\ndef find_total_area(shapes):\r\n    return sum(shape.get_area() for shape in shapes \r\n                                if isinstance(shape, Shape))\r\n\r\nshapes = [Rectangle(10.2, 8.3), Circle(6.2), Ellipse(8,12)]\r\nprint find_total_area(shapes)\r\n\r\nShape.register(Ellipse)\r\n\r\nprint find_total_area(shapes)\r\n<\/pre>\n
        The output of our program now, will be the following:<\/p>\n
        205.3616\r\nEllipse get_area was called!\r\n506.8016\r\n<\/pre>\n
        As you can see, the Ellipse.get_area was now called even if Ellipse does not derives from Shape, since we called the register<\/em> function of the ABC.<\/p>\n
        Conclusion<\/h2>\n
        Abstract base classes can be used to customize type checking whenever something like this is preferable over the classical pythonic<\/em> duck typing.<\/p>\n
        We can use this technique in cases where we want to favour code extensibility based in loose coupled components, that will be developed by many developers who do not necessary have a deep understanding of the code details and prefer to view specific components as black boxes that they interact with using well defined interface-based contracts.<\/p>\n
        Another argument in favour of ABCs based design has to do with the improved documentation which can become higher level, encapsulating as many implementation details as possible from the user of the component allowing for easier code changes and testing.<\/p>\n","protected":false},"excerpt":{"rendered":"
        If it looks like a duck, quacks like a duck and walks like a duck, it’s a duck Probably the greatest feature of python is…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"footnotes":""},"categories":[6],"tags":[],"class_list":["post-217","post","type-post","status-publish","format-standard","hentry","category-programming"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":213,"url":"https:\/\/codingismycraft.blog\/index.php\/2017\/10\/04\/adding-descriptors-to-your-python-arsenal\/","url_meta":{"origin":217,"position":0},"title":"Adding descriptors to your python arsenal","author":"john","date":"October 4, 2017","format":false,"excerpt":"Python\u00a0provides a\u00a0 rich programming paradigm providing the ability to the programmer to express his solutions in a very concise and elegant way.\u00a0 One of the python features that is not in wide use although it gives the opportunity to simplify certain pieces of code,\u00a0is the descriptor protocol which is the\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/codingismycraft.blog\/index.php\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":219,"url":"https:\/\/codingismycraft.blog\/index.php\/2014\/05\/23\/python-decorators-basics\/","url_meta":{"origin":217,"position":1},"title":"Python Decorators Basics","author":"john","date":"May 23, 2014","format":false,"excerpt":"Introduction to decorators Decorators introduce a programming paradigm that is not found in statically linked languages, like C++ or Java. Taking advantage of closures and the fact that functions are first class citizens of python, we can easily change the behaviour of them adding a single line of code. If\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/codingismycraft.blog\/index.php\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":279,"url":"https:\/\/codingismycraft.blog\/index.php\/2024\/10\/03\/the-challenges-of-python-migration-lessons-from-c-and-beyond\/","url_meta":{"origin":217,"position":2},"title":"The Challenges of Python Migration: Lessons from C++ and Beyond","author":"john","date":"October 3, 2024","format":false,"excerpt":"One project that confirmed the need for caution and conservatism when estimating deadlines involved migrating a massive codebase of over 4,000 Python files and 250+ open-source libraries from Python 3.6 to 3.10. What was initially seen as a straightforward task, expected to take just a few weeks, ended up consuming\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":70,"url":"https:\/\/codingismycraft.blog\/index.php\/2017\/04\/08\/python-and-neural-networks-nyu-lecture\/","url_meta":{"origin":217,"position":3},"title":"Python And Neural Networks NYU Lecture","author":"john","date":"April 8, 2017","format":false,"excerpt":"Last week I presented a lecture as part of the Business Intelligence Analytics and Data Science course in the School of Professional Studies of NYU here in New York that is taught from Stavros Zervoudakis. The objective of the lecture was to introduce students to python and provide them with\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/codingismycraft.blog\/index.php\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":30,"url":"https:\/\/codingismycraft.blog\/index.php\/2013\/03\/11\/python-is-great\/","url_meta":{"origin":217,"position":4},"title":"Python is great","author":"john","date":"March 11, 2013","format":false,"excerpt":"I have fallen in love with several programming languages in the past, with C++ being the most dominate of them. During the last years though, I have become a Python fan boy, trying to use it as much as I can, something that translates to use it almost everywhere except\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/codingismycraft.blog\/index.php\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":266,"url":"https:\/\/codingismycraft.blog\/index.php\/2024\/07\/03\/arch-installing-vim-9-from-source-code\/","url_meta":{"origin":217,"position":5},"title":"Arch: Installing vim 9 from source code","author":"john","date":"July 3, 2024","format":false,"excerpt":"Summary The vim that is installed by default when installing arch does not support neither clipboard nor python. The same applies to the vim version that can be install from the remote repositories using pacman. The objective of the posting is to install vim 9 complied with clipboard and python\u2026","rel":"","context":"In "arch"","block_context":{"text":"arch","link":"https:\/\/codingismycraft.blog\/index.php\/category\/arch\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/posts\/217","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/comments?post=217"}],"version-history":[{"count":1,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/posts\/217\/revisions"}],"predecessor-version":[{"id":218,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/posts\/217\/revisions\/218"}],"wp:attachment":[{"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/media?parent=217"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/categories?post=217"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codingismycraft.blog\/index.php\/wp-json\/wp\/v2\/tags?post=217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}