A Simple Type Check

Here is a simple decorator that ensures that every argument the function receives is an integer, and complains otherwise:

def requires_ints(decorated):
    def inner(*args, **kwargs):
        # Get any values that may have been sent as keyword arguments.
        kwarg_values = [i for i in kwargs.values()]

        # Iterate over every value sent to the decorated method, and
        # ensure that each one is an integer; raise TypeError if not.
        for arg in list(args) + kwarg_values:
            if not isinstance(arg, int):
                raise TypeError('%s only accepts integers as arguments.’ %
                                decorated.__name__)

        # Run the decorated method, and return the result.
        return decorated(*args, **kwargs)
    return inner

What is happening here?

The decorator itself is requires_ints. It accepts one argument, decorated, which is the decorated callable. The only thing that this decorator does is return a new callable, the local function inner. This function replaces the decorated method.

You can see this in action by declaring a function and decorating it with requires_ints:

@requires_ints
def foo(x, y):
    """Return the sum of x and y."""
    return x + y

Notice what you get if you run help(foo):

Help on function inner in module __main__:

inner(*args, **kwargs)
(END)

The inner function has been assigned to the name foo instead of the original, defined function. If you run foo(3, 5), the inner function runs with those arguments. The inner function performs the type check, and then it runs the decorated method simply because the inner function calls it using return decorated(*args, **kwargs), returning 8. Absent this call, the decorated method would have been ignored.

Preserving the help

It often is not particularly desirable to have a decorator steamroll your function's docstring or hijack the output of help. Because decorators are tools for adding generic and reusable functionality, they are necessarily going to be more vague. And, generally, if someone using a function is trying to run help on it, he or she wants information about the guts of the function, not the shell.

The solution to this problem is actually … a decorator. Python implements a decorator called @functools.wraps that copies the important introspection elements of one function onto another function.

Here is the same @requires_ints decorator, but it adds in the use of @functools.wraps:

import functools


def requires_ints(decorated):
    @functools.wraps(decorated)
    def inner(*args, **kwargs):
        # Get any values that may have been sent as keyword arguments.
        kwarg_values = [i for i in kwargs.values()]

        # Iterate over every value sent to the decorated method, and
        # ensure that each one is an integer; raise TypeError if not.
        for arg in args + kwarg_values:
            if not isinstance(i, int):
                raise TypeError('%s only accepts integers as arguments.' %
                                decorated.__name__)

        # Run the decorated method, and return the result.
        return decorated(*args, **kwargs)
    return inner

The decorator itself is almost entirely unchanged, except for the addition of the second line, which applies the @functools.wraps decorator to the inner function. You must also import functools now (which is in the standard library). You will also notice some additional syntax. This decorator actually takes an argument (more on that later).

Now you apply the decorator to the same function, as shown here:

@requires_ints
def foo(x, y):
    """Return the sum of x and y."""
    return x + y

Here is what happens when you run help(foo) now:

Help on function foo in module __main__:

foo(x, y)
    Return the sum of x and y.
(END)

You see that the docstring for foo, as well as its method signature, is what is read out when you look at help. Underneath the hood, however, the @requires_ints decorator is still applied, and the inner function is still what runs.

Depending on which version of Python you are running, you will get a slightly different result from running help on foo, specifically regarding the function signature. The previous paste represents the output from Python 3.4. However, in Python 2, the function signature provided will still be that of inner (so, *args and **kwargs rather than x and y).

User Verification

A common use case for this pattern (that is, performing some kind of sanity check before running the decorated method) is user verification. Consider a method that is expected to take a user as its first argument.

The user should be an instance of this User and AnonymousUser class, as shown here:

class User(object):
    """A representation of a user in our application."""

    def __init__(self, username, email):
        self.username = username
        self.email = email


class AnonymousUser(User):
    """An anonymous user; a stand-in for an actual user that nonetheless
    is not an actual user.
    """
    def __init__(self):
        self.username = None
        self.email = None

    def __nonzero__(self):
        return False

A decorator is a powerful tool here for isolating the boilerplate code of user verification. A @requires_user decorator can easily verify that you got a User object and that it is not an anonymous user.

import functools


def requires_user(func):
    @functools.wraps(func)
    def inner(user, *args, **kwargs):
        """Verify that the user is truthy; if so, run the decorated method,
        and if not, raise ValueError.
        """
        # Ensure that user is truthy, and of the correct type.
        # The "truthy"check will fail on anonymous users, since the
        # AnonymousUser subclass has a ‘__nonzero__‘ method that
        # returns False.
        if user and isinstance(user, User):
            return func(user, *args, **kwargs)
        else:
            raise ValueError('A valid user is required to run this.')
    return inner

This decorator applies a common, boilerplate need—the verification that a user is logged in to the application. When you implement this as a decorator, it is reusable and more easily maintainable, and its application to functions is clear and explicit.

Note that this decorator will only correctly wrap a function or static method, and will fail if wrapping a bound method to a class. This is because the decorator ignores the expectation to send self as the first argument to a bound method.

Output Formatting

In addition to sanitizing input to a function, another use for decorators can be sanitizing output from a function.

When you're working in Python, it is normally desirable to use native Python objects when possible. Often, however, you want a serialized output format (for example, JSON). It is cumbersome to manually convert to JSON at the end of every relevant function, and (and it's not a good idea, either). Ideally, you should be using the Python structures right up until serialization is necessary, and there may be other boilerplate that happens just before serialization (such as or the like).

Decorators provide an excellent, portable solution to this problem. Consider the following decorator that takes Python output and serializes the result to JSON:

import functools
import json


def json_output(decorated):
    """Run the decorated function, serialize the result of that function
    to JSON, and return the JSON string.
    """
    @functools.wraps(decorated)
    def inner(*args, **kwargs):
        result = decorated(*args, **kwargs)
        return json.dumps(result)
    return inner

Apply the @json_output decorator to a trivial function, as shown here:

@json_output
def do_nothing():
    return {'status': 'done'}

Run the function in the Python shell, and you get the following:

>>> do_nothing()
'{"status": "done"}'

Notice that you got back a string that contains valid JSON. You did not get back a dictionary.

The beauty of this decorator is in its simplicity. Apply it to a function, and suddenly a function that did return a Python dictionary, list, or other object now returns its JSON-serialized version.

You might ask, “Why is this valuable?” After all, you are adding a one-line decorator that essentially removes a single line of code—a call to json.dumps. However, consider the value of having this decorator as the application's needs expand.

For example, what if certain exceptions should be trapped and output specifically formatted JSON, rather than having the exception bubble up and traceback? Because you have a decorator, that functionality is very easy to add.

import functools
import json


class JSONOutputError(Exception):
    def __init__(self, message):
        self._message = message

    def __str__(self):
        return self._message


def json_output(decorated):
    """Run the decorated function, serialize the result of that function
    to JSON, and return the JSON string.
    """
    @functools.wraps(decorated)
    def inner(*args, **kwargs):
        try:
            result = decorated(*args, **kwargs)
        except JSONOutputError as ex:
            result = {
                'status': 'error',
                'message': str(ex),
            }
        return json.dumps(result)
    return inner

By augmenting the @json_output decorator with this error handling, you have added it to any function where the decorator was already applied. This is part of what makes decorators so valuable. They are very useful tools for code portability and reusability.

Now, if a function decorated with @json_output raises a JSONOutputError, you will get this ­special error handling. Here is a function that does:

@json_output
def error():
    raise JSONOutputError('This function is erratic.')

Running the error function in the Python interpreter gives you the following:

>>> error()
'{"status": "error", "message": "This function is erratic."}'

Note that only the JSONOutputError exception class (and any subclasses) receives this special ­handling. Any other exception is passed through normally, and generates a traceback. Consider this function:

@json_output
def other_error():
    raise ValueError('The grass is always greener...')

When you run it, you will get the traceback you expect, as shown here:

>>> other_error()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 8, in inner
  File "<stdin>", line 3, in other_error
ValueError: The grass is always greener...

This reusability and maintainability is part of what makes decorators valuable. Because a decorator is being used for a reusable, generally applicable concept throughout the application (in this case, JSON serialization), the decorator becomes the place for housing that functionality as needs arise that are applicable whenever that concept is used.

Essentially, decorators are tools to avoid repeating yourself, and part of their value is in providing hooks for future maintenance.

This can be accomplished without the use of decorators. Consider the example of requiring a logged-in user. It is not difficult to write a function that does this and simply place it near the top of functions that require that functionality. The decorator is primarily syntactic sugar. The syntactic sugar has value, though. Code is read more often than it is written, after all, and it is easy to locate decorators at a glance.

Logging

One final example of execution-time wrapping of code is a general-use logging function. Consider the following decorator that causes the function call, timings, and result to be logged:

import functools
import logging
import time


def logged(method):
    """Cause the decorated method to be run and its results logged, along
    with some other diagnostic information.
    """
    @functools.wraps(method)
    def inner(*args, **kwargs):
        # Record our start time.
        start = time.time()

        # Run the decorated method.
        return_value = method(*args, **kwargs)

        # Record our completion time, and calculate the delta.
        end = time.time()
        delta = end - start

        # Log the method call and the result.
        logger = logging.getLogger('decorator.logged')
        logger.warn('Called method %s at %.02f; execution time %.02f ‘
                    'seconds; result %r.' %
                    (method.__name__, start, delta, return_value))

        # Return the method's original return value.
        return return_value
    return inner

When applied to a function, this decorator runs that function normally, but uses the Python logging module to log out information about the function call after it completes. Now, suddenly, you have (extremely rudimentary) logging of any function where this decorator is applied.

>>> import time
>>> @logged
... def sleep_and_return(return_value):
...      time.sleep(2)
...      return return_value
...
>>>
>>> sleep_and_return(42)
Called method sleep_and_return at 1424462194.70;
     execution time 2.00 seconds; result 42.
42

Unlike the previous examples, this decorator does not alter the function call in an obvious way. No cases exist where you apply this decorator and get a different result from the decorated function than you did from the undecorated function. The previous examples raised exceptions or modified the result if this or that check did not pass. This decorator is more invisible. It does some under-the-hood work, but in no situation should it change the actual result.

Variable Arguments

It is worth noting that the @json_output and @logged decorators both provide inner functions that simply take, and pass on with minimal investigation, variable arguments and keyword arguments.

This is an important pattern. One way that it is particularly important is that many decorators may be used to decorate plain functions as well as methods of classes. Remember that in Python, methods declared in classes receive an additional positional argument, conventionally known as self. This does not change when decorators are in use. (This is why the requires_user decorator shown earlier does not work on bound methods within classes.)

For example, if @json_result is used to decorate a method of a class, the inner function is called and it receives the instance of the class as the first argument. In fact, this is fine. In this case, that argument is simply args[0], and it is passed to the decorated method unmolested.

How Does This Work?

But wait. If json_output is not a decorator, but a function that returns a decorator, why does it look like it is being applied as a decorator? What is the Python interpreter doing here that makes this work?

More explanation is in order. The key here is in the order of operations. Specifically, the function call (json_output(indent=4)) precedes the decorator application syntax (@). Thus, the result of the function call is used to apply the decorator.

The first thing that is happening is that the interpreter is seeing the function call for json_output and resolving that call (note that the boldface does not include the @):

@json_output(indent=4)
def do_nothing():
    return {'status': 'done'}

All the json_output function does is define another function, actual_decorator, and return it. As the result of that function, it is then provided to @, as shown here:

@actual_decorator
def do_nothing():
    return {'status': 'done'}

Now, actual_decorator is being run. It declares another local function, inner, and returns it. As previously discussed, that function is then assigned to the name do_nothing, the name of the decorated method. When do_nothing is called, the inner function is called, runs the decorated method, and JSON dumps the result with the appropriate indentation.

The Call Signature Matters

It is critical to realize that when you introduced your new, altered json_output function, you ­actually introduced a backward-incompatible change.

Why? Because now there is this extra function call that is expected. If you want the old json_output behavior, and do not need values for any of the arguments available, you still must call the method.

In other words, you must do the following:

@json_output()
def do_nothing():
    return {'status': 'done'}

Note the parentheses. They matter, because they indicate that the function is being called (even with no arguments), and then the result is applied to the @.

The previous code is not—repeat, not—equivalent to the following:

@json_output
def do_nothing():
    return {'status': 'done'}

This presents two problems. It is inherently confusing, because if you are accustomed to seeing decorators applied without a signature, a requirement to supply an empty signature is counterintuitive. Secondly, if the old decorator already exists in your application, you must go back and edit all of its existing calls. You should avoid backward-incompatible changes if possible.

In a perfect world, this decorator would work for three different types of applications:

• @json_output
• @json_output()
• @json_output(indent=4)

As it turns out, this is possible, by having a decorator that modifies its behavior based on the arguments that it receives. Remember, a decorator is just a function and has all the flexibility of any other function to do what it needs to do to respond to the inputs it gets.

Consider this more flexible iteration of json_output:

import functools
import json


class JSONOutputError(Exception):
    def __init__(self, message):
        self._message = message

    def __str__(self):
        return self._message


def json_output(decorated_=None, indent=None, sort_keys=False):
    """Run the decorated function, serialize the result of that function
    to JSON, and return the JSON string.
    """
    # Did we get both a decorated method and keyword arguments?
    # That should not happen.
    if decorated_ and (indent or sort_keys):
        raise RuntimeError('Unexpected arguments.')

    # Define the actual decorator function.
    def actual_decorator(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            try:
                result = func(*args, **kwargs)
            except JSONOutputError as ex:
                result = {
                    'status': 'error',
                    'message': str(ex),
                }
            return json.dumps(result, indent=indent, sort_keys=sort_keys)
        return inner

    # Return either the actual decorator, or the result of applying
    # the actual decorator, depending on what arguments we got.
    if decorated_:
        return actual_decorator(decorated_)
    else:
        return actual_decorator

This function is endeavoring to be intelligent about whether or not it is currently being used as a decorator.

First, it makes sure it is not being called in an unexpected way. You never expect to receive both a method to be decorated and the keyword arguments, because a decorator is always called with the decorated method as the only argument.

Second, it defines the actual_decorator function, which (as its name suggests) is the actual decorator to be either returned or applied. It defines the inner function that is the ultimate function to be returned from the decorator.

Finally, it returns the appropriate result based on how it was called:

• If decorated_ is set, it was called as a plain decorator, without a method signature, and its responsibility is to apply the ultimate decorator and return the inner function. Here again, observe how decorators that take arguments are actually working. First, actual_decorator(decorated_) is called and resolved, then its result (which must be a callable, because this is a decorator) is called with inner provided as its only argument.

• If decorated_ is not set, then this was called with keyword arguments instead, and the function must return an actual decorator, which receives the decorated method and returns inner. Therefore, the function returns actual_decorator outright. This is then applied by the Python interpreter as the actual decorator (which ultimately returns inner).

Why is this technique valuable? It enables you to maintain your decorator's functionality as previously used. This means that you do not have to update each case where the decorator has been applied. But you still get the additional flexibility of being able to add arguments in the cases where you need them.