Why Declare Virtual Subclasses?

To understand why you would want to do this, recall the example at the beginning of the chapter where you wanted to read from a list-like object. It needs to be iterable like list or tuple, and it needs to have a __getitem__ method that takes integers. On the other hand, you do not necessarily want to have a restriction of only accepting list or tuple.

Abstract base classes provide a very good, extensible mechanism for that. A previous example showed that you can use isinstance to check against a tuple of classes.

>>> isinstance([], (list, tuple))
True

This is not really extensible, however. If you are checking against list or tuple in your implementation, and someone using your library wants to send something else that acts list-like but does not subclass list or tuple, that person is up a creek.

Abstract base classes provide the solution to this problem. First, define an abstract base class and register list and tuple to it, as shown here:

>>> import abc
>>> class MySequence(metaclass=abc.ABCMeta):
...     pass
...
>>> MySequence.register(list)
<class 'list'>
>>> MySequence.register(tuple)
<class 'tuple'>

Now, alter the isinstance check to check against MySequence instead of against (list, tuple). It will still return True when a list or tuple is checked, and False for other objects.

>>> isinstance([], MySequence)
True
>>> isinstance((), MySequence)
True
>>> isinstance(object(), MySequence)
False

Thus far, you have the same situation as before. But, there is one crucial difference. Consider the case where another developer is using a library that expects a MySequence object, and, therefore, expects a list or tuple.

When (list, tuple) is hard-coded in the library, there is nothing that the developer can do. However, MySequence is an abstract base class that the library is defining. That means that the developer can import it.

Once the developer is able to import it, the custom class that is sufficiently list-like can simply be registered with MySequence:

>>> class CustomListLikeClass(object):
...     pass
...
>>> MySequence.register(CustomListLikeClass)
<class '__main__.CustomListLikeClass'>
>>> issubclass(CustomListLikeClass, MySequence)
True

The developer is able to pass the CustomListLikeClass instance into the library that expects a MySequence. Now, when the library does its isinstance checks, the check passes, and the object is allowed.

Using register as a Decorator

As of Python 3.3, the register method provided by classes using the ABCMeta metaclass can also be used as a decorator.

If you are creating a new class that should be registered as a subclass of an ABCMeta, you normally register it like this (using the MySequence abstract base class defined in the previous example):

>>> class CustomListLikeClass(object):
...     pass
...
>>> MySequence.register(CustomListLikeClass)
<class '__main__.CustomListLikeClass'>

Note, however, that the register method returns the class that is passed to it. It works this way so that register can also be used as a decorator. It is accepting a callable and returning a callable (in this case, the exact same callable).

The following code will have an identical effect:

>>> @MySequence.register
... class CustomListLikeClass(object):
...     pass
...
>>> 

You can confirm this by doing the same issubclass check as you did before.

>>> issubclass(CustomListLikeClass, MySequence)
True

It is worth noting that this decorator behavior was added in Python 3.3. In Python 2, as well as in Python 3.2 and below, the register method on abstract base classes returned None, rather than returning the class that was passed to it.

This means that it is unable to be used as a decorator in these versions. If you are writing code that is intended to be cross-compatible with Python 2 and Python 3, or if you are writing code that may run on an older version of Python 3, you should avoid using register as a decorator.

__subclasshook__

For most purposes, using a class with the ABCMeta metaclass and then using the register method that ABCMeta provides is an entirely sufficient way to get what you need. However, you may have a case where manual registration of every intended subclass is not tenable.

Classes created with the ABCMeta metaclass may optionally define a special magic method called __subclasshook__.

The __subclasshook__ method must be defined as a class method (using the @classmethod decorator) and takes a single additional positional argument, which is the class being tested. It can return three values: True, False, or NotImplemented.

The case for True and False is salient enough. The __subclasshook__ method returns True if the tested class should be considered a subclass, and False if it should not be considered a subclass.

Consider the traditional duck typing paradigm. The fundamental concern in the duck-typing paradigm is whether an object has certain methods or attributes (whether it “quacks like a duck”), rather than whether it subclasses this or that class. An abstract base class could implement this concept with __subclasshook__, as shown here:

import abc


class AbstractDuck(metaclass=abc.ABCMeta):
    @classmethod
    def __subclasshook__(cls, other):
        quack = getattr(other, 'quack', None)
        return callable(quack)

This abstract base class is declaring that any class with a quack method (but not a non-callable quack attribute) should be considered its subclass, and nothing else should be.

>>> class Duck(object):
...     def quack(self):
...         pass
...
>>>
>>> class NotDuck(object):
...     quack = 'foo'
...
>>> issubclass(Duck, AbstractDuck)
True
>>> issubclass(NotDuck, AbstractDuck)
False

An important thing to note here is that when the __subclasshook__ method is defined, it takes ­precedence over the register method.

>>> AbstractDuck.register(NotDuck)
<class '__main__.NotDuck'>
>>> issubclass(NotDuck, AbstractDuck)
False

This is where NotImplemented comes in. If the __subclasshook__ method returns NotImplemented, then (and only then) the traditional route of checking to see if a class has been registered is checked.

Consider the following modified AbstractDuck class:

import abc


class AbstractDuck(metaclass=abc.ABCMeta):
    @classmethod
    def __subclasshook__(cls, other):
        quack = getattr(other, 'quack', None)
        if callable(quack):
            return True
        return NotImplemented

The only change made here is that if there is not a quack method, the __subclasshook__ method returns NotImplemented instead of False. Now, the registry is checked, and a class that has been previously registered will come back as a subclass.

>>> issubclass(NotDuck, AbstractDuck)
False
>>> AbstractDuck.register(NotDuck)
<class '__main__.NotDuck'>
>>> issubclass(NotDuck, AbstractDuck)
True

Essentially, the first example says, “It is an AbstractDuck if it quacks like a duck,” and the second example says, “It is an AbstractDuck if it quacks like a duck … or if it just says flat out that it is an AbstractDuck.”

Of course, note that if you do this, you must be able to handle anything that you receive. It does you no good to make the quack method optional if you rely on being able to call it!

So, what is the value of doing this? It would be easy enough simply to do a hasattr or callable check on the methods you need.

In a relatively straightforward case, it is probably actually a hindrance to use an abstract base class. For example, it would simply add unnecessary complexity to use one as a stand-in to check for the presence of a single method.

For non-trivial cases, there is some value. First, there is value in compartmentalization. The abstract base class defines a single place for the overall test to live. Any code using a subclass of the abstract base class simply uses the issubclass or isinstance function. This ensures that as needs evolve, there is a single place for the conformity-checking code to live.

Also, the availability of NotImplemented as a return value for __subclasshook__ adds some power. It provides a mechanism to say that while there are ways to definitively pass or definitively fail to match the given protocol, there is also the way for a custom class author to explicitly opt in.

Other Existing Approaches

You can approach this fundamental problem without using abstract base classes. Because abstract base classes are a relatively new language feature, several of these approaches are quite common.

from datetime import datetime


class Task(object):
    """An abstract class representing a task that must run, and
    which should track individual runs and results.
    """
    def __init__(self):
        self.runs = []

    def run(self):
        start = datetime.now()
        result = self._run()
        end = datetime.now()
        self.runs.append({
            'start': start,
            'end': end,
            'result': result,
        })
        return result

    def _run(self):
        raise NotImplementedError('Task subclasses must define '
                                  'a _run method.')

>>> t = Task()
>>> t.run()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 10, in run
  File "<stdin>", line 20, in _run
NotImplementedError: Task subclasses must define a _run method.

from datetime import datetime, timezone


class TaskMeta(type):
    """A metaclass that ensures the presence of a _run method
    on any non-abstract classes it creates.
    """
    def __new__(cls, name, bases, attrs):
        # If this is an abstract class, do not check for a _run method.
        if attrs.pop('abstract', False):
            return super(TaskMeta, cls).__new__(cls, name, bases, attrs)

        # Create the resulting class.
        new_class = super(TaskMeta, cls).__new__(cls, name, bases, attrs)

        # Verify that a _run method is present and raise
        # TypeError otherwise.
        if not hasattr(new__class, '__run') or not callable(new__class.__run):
            raise TypeError('Task subclasses must define a _run method.')

        # Return the new class object.
        return new_class


class Task(metaclass=TaskMeta):
    """An abstract class representing a task that must run, and
    which should track individual runs and results.
    """
    abstract = True

    def __init__(self):
        self.runs = []

    def run(self):
        start = datetime.now(tz=timezone.utc)
        result = self._run()
        end = datetime.now(tz=timezone.utc)
        self.runs.append({
            'start': start,
            'end': end,
            'result': result,
        })
        return result

>>> t = Task()
>>> t.run()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 12, in run
AttributeError: 'Task' object has no attribute '_run'

>>> class TaskSubclass(Task):
...     pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 16, in __new__
NotImplementedError: Task subclasses must define a _run method.

The Value of Abstract Base Classes

Both of these approaches are valuable, but it is also fair to criticize them for being somewhat ad hoc.

Abstract base classes provide a more formal way to present the same pattern. They provide a mechanism to declare a protocol using an abstract class, and subclasses must provide an implementation that conforms to that protocol.

The abc module provides a decorator called @abstractmethod, which designates that a given method must be overridden by all subclasses. The method body may be empty (pass), or may contain an implementation that the subclass methods may choose to call using super.

Consider a Task class that uses the @abstractmethod decorator in lieu of a custom metaclass.

import abc
from datetime import datetime, timezone


class Task(metaclass=abc.ABCMeta):
    """An abstract class representing a task that must run, and
    which should track individual runs and results.
    """
    def __init__(self):
        self.runs = []

    def run(self):
        start = datetime.now(tz=timezone.utc)
        result = self._run()
        end = datetime.now(tz=timezone.utc)
        self.runs.append({
            'start': start,
            'end': end,
            'result': result,
        })
        return result

    @abc.abstractmethod
    def _run(self):
        pass

Again, this is mostly identical to the previous two examples, but ever so slightly different from both. First, note that the Task class itself is unable to be instantiated.

>>> t = Task()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Task with abstract methods _run

This is distinct from the NotImplementedError approach, which would have allowed the base Task class to be instantiated.

Similarly, it is distinct from both of the previous approaches in that the error case for a subclass that does not properly override the _run method is slightly different. In the first example, using NotImplementedError, you end up having NotImplementedError raised at the point where the _run method is called. In the second example, using a custom TaskMeta metaclass, TypeError is raised when the offending subclass is created.

When using an abstract base class, the interpreter is perfectly happy to create a subclass that does not implement all (or even any) of the abstract methods in the base class.

>>> class Subtask(Task):
...   pass
...
>>>

What the interpreter is not willing to do, however, is instantiate it. In fact, it gives the exact same error as the Task class gives, which is logically exactly what you expect.

>>> st = Subtask()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Subtask with abstract methods _run

However, once you define a subclass that overrides the abstract methods, it works just fine, and you are able to instantiate your subclass.

>>> class OtherSubtask(Task):
...   def _run(self):
...     return 2 + 2
...
>>>
>>> ost = OtherSubtask()
>>> ost.run()
4

And, if you inspect the runs attribute, you will see that information about the run has been saved, as shown here:

>>> ost.runs
[{'result': 4, 'end': datetime.datetime(…), 'start': datetime.datetime(…)}]

This is actually a very useful approach to this problem, for several reasons. First (and probably most important), this approach is formalized rather than ad hoc. Abstract base classes were specifically proposed as a solution to fill this particular need, pursuant to the notion that, ideally, there should be one and only one “correct” way to do it.

Second, the @abstractmethod decorator is very simple, and avoids a lot of potential errors that can crop up if you're attempting to write boilerplate code. As an example, what if, in your TaskMeta metaclass, you accidentally only check for the presence of _run in the attrs dictionary, but do not allow for the presence of _run in the superclass? This is an easy mistake to make, and it would result in Task subclasses that are not themselves subclassable unless you manually override _run every time. With the @abstractmethod decorator, you get the right behavior without having to put too much thought into it.

Finally, this approach makes it very easy to have intermediate implementations. Consider an abstract base class that has 10 abstract methods instead of one. It is entirely reasonable to have an entire subclass tree, where higher subclasses on the chain implement some common methods, but leave other methods in their abstract state for their subclasses to implement. In fairness, you can do this with the custom metaclass approach also (by declaring every intermediate class abstract = True in the TaskMeta example). However, when using @abstractmethod, you basically get exactly the behavior you want intuitively.

Of course, there is one big reason not to use an abstract base class if you need this type of functionality, which is if you need to support versions of Python that do not yet have abc. This is becoming more rare, though, because abc was added in Python 2.6, and many Python packages do not support versions of Python older than 2.6.

Abstract Properties

It is also possible for properties (that is, methods that use the @property decorator) to be declared as abstract. However, the correct approach to this depends slightly on what versions of Python you are supporting.

In Python 2.6 through 3.2 (including any code that must be cross-compatible with these versions), the correct approach is to use the @abstractproperty decorator, which is provided by the abc module.

import abc


class AbstractClass(metaclass=abc.ABCMeta):
    @abc.abstractproperty
    def foo(self):
        pass

In Python 3.3, this approach is deprecated, because @abstractmethod has been updated to be able to work alongside @property. Therefore, having a special decorator to provide both is now redundant. Thus, the following example is identical to the previous one, but only in Python 3.3 and up:

import abc


class AbstractClass(metaclass=abc.ABCMeta):
    @property
    @abc.abstractmethod
    def foo(self):
        pass

Attempting to instantiate a subclass of AbstractClass that does not override the foo method will raise an error.

>>> class InvalidChild(AbstractClass):
...     pass
...
>>> ic = InvalidChild()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class InvalidChild with abstract methods foo

However, a subclass that overrides the abstract method is able to be instantiated.

>>> class ValidChild(AbstractClass):
...     @property
...     def foo(self):
...         return 'bar'
...
>>>
>>> vc = ValidChild()
>>> vc.foo
'bar'

Abstract Class or Static Methods

As with properties, you may want to combine the @abstractmethod decorator with either a class method or static method (that is, a method decorated with @classmethod or @staticmethod).

This is a little bit trickier. Python 2.6 through 3.1 simply do not provide a way to do this at all. Python 3.2 does provide a way, using the @abstractclassmethod or @abstractstaticmethod decorators. These work similarly to the previous abstract properties example.

Python 3.3 then alters this by changing @abstractmethod to be compatible with the @classmethod and @staticmethod decorators, and deprecates the Python 3.2 approach.

In this case, because most code written for Python 3 usually is only written to be compatible with Python 3.3 and up (you learn more about this in Chapter 10, “Python 2 Versus Python 3”), most likely what you want to do is simply use the two decorators separately. However, if you need compatibility with Python 3.2, and do not need compatibility with any previous versions of Python (including any versions of Python 2), then those decorators are available to you.

Consider the following abstract class using the Python 3.3 syntax:

class AbstractClass(metaclass=abc.ABCMeta):
    @classmethod
    @abc.abstractmethod
    def foo(cls):
        return 42

Subclassing this class without overriding the method will work as usual, but the subclass is unable to be instantiated.

>>> class InvalidChild(AbstractClass):
...     pass
...
>>> ic = InvalidChild()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class InvalidChild with abstract methods foo

The abstract method itself can actually be called directly without error, though.

>>> InvalidChild.foo()
42

Once the abstract method is overridden in a subclass, that subclass is able to be instantiated.

>>> class ValidChild(AbstractClass):
...     @classmethod
...     def foo(cls):
...         return 'bar'
...
>>> ValidChild.foo()
'bar'
>>> vc = ValidChild()
>>> vc.foo()
'bar'

Single-Method ABCs

Python provides five abstract base classes that contain one abstract method each, and whose _­subclasscheck_ methods simply check for the presence of that method. They are as follows:

• Callable (__call__)

• Container (__contains__)

• Hashable (__hash__)

• Iterable (__iter__)

• Sized (__len__)

Any class that contains the appropriate method is automatically considered to be a subclass of the relevant abstract base class.

>>> from collections.abc import Sized
>>>
>>> class Foo(object):
...     def __len__(self):
...         return 42
...
>>> issubclass(Foo, Sized)
True

Similarly, classes may subclass the abstract base classes directly, and are expected to override the relevant method.

>>> class Bar(Sized):
...     pass
...
>>> b = Bar()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Bar with abstract methods __len__

In addition to these five classes, there is one more. Iterator is slightly special. It inherits from Iterable, provides an implementation for __iter__ (which just returns itself and can be overridden), and adds an abstract method called __next__.

Alternative-Collection ABCs

Another major type of built-in abstract base classes in Python 3 are those that serve to identify ­subclasses that serve a similar role as the major Python collection classes: list, dict, and set.

There are six of these classes, divided into three categories with two in each category (one immutable class and one mutable one).

The first category is Sequence and MutableSequence. These abstract base classes are intended for collections that generally act like Python tuples or lists, respectively. The Sequence abstract base class requires __getitem__ and __len__. However, it also provides implementations for a lot of other common methods you use with list and tuples, such as __contains__ and __iter__ (among others). The idea here is that you can subclass Sequence and define just the things you need, and Python provides you with the other common functionality of sequences. Of course, list, tuple, and set are considered to be subclasses of Sequence.

MutableSequence is similar, but adds the notion of modifying the sequence in-place. Therefore, it adds __setitem__, __delitem__, and insert as abstract methods, and provides functionality for append, pop, and the like. The principle is still the same—you must define just the things you fundamentally need to have a mutable sequence, and Python provides list-like methods for the rest. As you probably expect, list and set are already considered to be subclasses of MutableSequence out of the box.

The other two categories are Mapping and Set, which come with MutableMapping and MutableSet, as you would expect. Mappings are intended for dictionary-like objects (similar to dict, and dict is considered a subclass), whereas Sets are intended for unordered collections (similar to set, and set is considered a subclass). In both cases, they specify some key methods (with names corresponding to those of dict and set) as abstract, and provide implementations for the remainder.

Additional ABCs

There are other abstract base classes in the standard library not covered in detail here. In particular, the numbers module contains abstract base classes for implementing many different kinds of numbers.