Читать книгу Professional Python - Luke Sneeringer - Страница 8

Decorators are simply functions that (usually) accept the decorated callable as their only argument, and that return a callable (such as in the previous trivial example).

It is important to note that the decorator code itself runs when the decorator is applied to the decorated function, rather than when the decorated function is called. Understanding this is critical, and will become very clear over the course of the next several examples.

An Initial Example: A Function Registry

Consider the following simple registry of functions:

The register method is a simple decorator. It appends the positional argument, decorated to the registry variable, and then returns the decorated method unchanged. Any method that receives the register decorator will have itself appended to registry.

If you have access to the registry, you can easily iterate over it and execute the functions inside.

The answers list at this point would now contain [3, 5]. This is because the functions are executed in order, and their return values are appended to answers.

Several less-trivial uses for function registries exist, such as adding “hooks” into code so that custom functionality can be run before or after critical events. Here is a Registry class that can handle just such a case:

One thing worth noting about this class is that the register method – the decorator – still works the same way as before. It is perfectly fine to have a bound method as a decorator. It receives self as the first argument (just as any other bound method), and expects one additional positional argument, which is the decorated method.

By making several different registry instances, you can have entirely separate registries. It is even possible to take the same function and register it with more than one registry, as shown here:

Running the code from either registry's run_all method gives the following results:

Notice that the run_all method is able to take arguments, which it then passes to the underlying functions when they are run.

Execution-Time Wrapping Code

These decorators are very simple because the decorated function is passed through unmodified. However, sometimes you want additional functionality to run when the decorated method is executed. You do this by returning a different callable that adds the appropriate functionality and (usually) calls the decorated method in the course of its execution.

A Simple Type Check

Here is a simple decorator that ensures that every argument the function receives is an integer, and complains otherwise:

What is happening here?

The decorator itself is requires_ints. It accepts one argument, decorated, which is the decorated callable. The only thing that this decorator does is return a new callable, the local function inner. This function replaces the decorated method.

You can see this in action by declaring a function and decorating it with requires_ints:

Notice what you get if you run help(foo):

The inner function has been assigned to the name foo instead of the original, defined function. If you run foo(3, 5), the inner function runs with those arguments. The inner function performs the type check, and then it runs the decorated method simply because the inner function calls it using return decorated(*args, **kwargs), returning 8. Absent this call, the decorated method would have been ignored.

Preserving the help

It often is not particularly desirable to have a decorator steamroll your function's docstring or hijack the output of help. Because decorators are tools for adding generic and reusable functionality, they are necessarily going to be more vague. And, generally, if someone using a function is trying to run help on it, he or she wants information about the guts of the function, not the shell.

The solution to this problem is actually … a decorator. Python implements a decorator called @functools.wraps that copies the important introspection elements of one function onto another function.

Here is the same @requires_ints decorator, but it adds in the use of @functools.wraps:

The decorator itself is almost entirely unchanged, except for the addition of the second line, which applies the @functools.wraps decorator to the inner function. You must also import functools now (which is in the standard library). You will also notice some additional syntax. This decorator actually takes an argument (more on that later).

Now you apply the decorator to the same function, as shown here:

Here is what happens when you run help(foo) now:

You see that the docstring for foo, as well as its method signature, is what is read out when you look at help. Underneath the hood, however, the @requires_ints decorator is still applied, and the inner function is still what runs.

Depending on which version of Python you are running, you will get a slightly different result from running help on foo, specifically regarding the function signature. The previous paste represents the output from Python 3.4. However, in Python 2, the function signature provided will still be that of inner (so, *args and **kwargs rather than x and y).

User Verification

A common use case for this pattern (that is, performing some kind of sanity check before running the decorated method) is user verification. Consider a method that is expected to take a user as its first argument.

The user should be an instance of this User and AnonymousUser class, as shown here:

A decorator is a powerful tool here for isolating the boilerplate code of user verification. A @requires_user decorator can easily verify that you got a User object and that it is not an anonymous user.

This decorator applies a common, boilerplate need – the verification that a user is logged in to the application. When you implement this as a decorator, it is reusable and more easily maintainable, and its application to functions is clear and explicit.

Note that this decorator will only correctly wrap a function or static method, and will fail if wrapping a bound method to a class. This is because the decorator ignores the expectation to send self as the first argument to a bound method.

Output Formatting

In addition to sanitizing input to a function, another use for decorators can be sanitizing output from a function.

When you're working in Python, it is normally desirable to use native Python objects when possible. Often, however, you want a serialized output format (for example, JSON). It is cumbersome to manually convert to JSON at the end of every relevant function, and (and it's not a good idea, either). Ideally, you should be using the Python structures right up until serialization is necessary, and there may be other boilerplate that happens just before serialization (such as or the like).

Decorators provide an excellent, portable solution to this problem. Consider the following decorator that takes Python output and serializes the result to JSON:

Apply the @json_output decorator to a trivial function, as shown here:

Run the function in the Python shell, and you get the following:

Notice that you got back a string that contains valid JSON. You did not get back a dictionary.

The beauty of this decorator is in its simplicity. Apply it to a function, and suddenly a function that did return a Python dictionary, list, or other object now returns its JSON-serialized version.

You might ask, “Why is this valuable?” After all, you are adding a one-line decorator that essentially removes a single line of code – a call to json.dumps. However, consider the value of having this decorator as the application's needs expand.

For example, what if certain exceptions should be trapped and output specifically formatted JSON, rather than having the exception bubble up and traceback? Because you have a decorator, that functionality is very easy to add.

By augmenting the @json_output decorator with this error handling, you have added it to any function where the decorator was already applied. This is part of what makes decorators so valuable. They are very useful tools for code portability and reusability.

Now, if a function decorated with @json_output raises a JSONOutputError, you will get this special error handling. Here is a function that does:

Running the error function in the Python interpreter gives you the following:

Note that only the JSONOutputError exception class (and any subclasses) receives this special handling. Any other exception is passed through normally, and generates a traceback. Consider this function:

When you run it, you will get the traceback you expect, as shown here:

This reusability and maintainability is part of what makes decorators valuable. Because a decorator is being used for a reusable, generally applicable concept throughout the application (in this case, JSON serialization), the decorator becomes the place for housing that functionality as needs arise that are applicable whenever that concept is used.

Essentially, decorators are tools to avoid repeating yourself, and part of their value is in providing hooks for future maintenance.

This can be accomplished without the use of decorators. Consider the example of requiring a logged-in user. It is not difficult to write a function that does this and simply place it near the top of functions that require that functionality. The decorator is primarily syntactic sugar. The syntactic sugar has value, though. Code is read more often than it is written, after all, and it is easy to locate decorators at a glance.

Logging

One final example of execution-time wrapping of code is a general-use logging function. Consider the following decorator that causes the function call, timings, and result to be logged:

When applied to a function, this decorator runs that function normally, but uses the Python logging module to log out information about the function call after it completes. Now, suddenly, you have (extremely rudimentary) logging of any function where this decorator is applied.

Unlike the previous examples, this decorator does not alter the function call in an obvious way. No cases exist where you apply this decorator and get a different result from the decorated function than you did from the undecorated function. The previous examples raised exceptions or modified the result if this or that check did not pass. This decorator is more invisible. It does some under-the-hood work, but in no situation should it change the actual result.

Variable Arguments

It is worth noting that the @json_output and @logged decorators both provide inner functions that simply take, and pass on with minimal investigation, variable arguments and keyword arguments.

This is an important pattern. One way that it is particularly important is that many decorators may be used to decorate plain functions as well as methods of classes. Remember that in Python, methods declared in classes receive an additional positional argument, conventionally known as self. This does not change when decorators are in use. (This is why the requires_user decorator shown earlier does not work on bound methods within classes.)

For example, if @json_result is used to decorate a method of a class, the inner function is called and it receives the instance of the class as the first argument. In fact, this is fine. In this case, that argument is simply args[0], and it is passed to the decorated method unmolested.

Decorator Arguments

One thing that has been consistent about all the decorators enumerated thus far is that the decorators themselves appear not to take any arguments. As discussed, there is an implied argument – the method that is being decorated.

However, sometimes it is useful to have the decorator itself take some information that it needs to decorate the method appropriately. The difference between an argument passed to the decorator and an argument passed to the function at call time is precisely that. An argument to the decorator is processed once, when the function is declared and decorated. By contrast, arguments to the function are processed when that function is called.

You have already seen an example of an argument sent to a decorator with the repeated use of @functools.wraps. It takes an argument – the method being wrapped, whose help and docstring and the like should be preserved.

However, decorators have implied call signatures. They take one positional argument – the method being decorated. So, how does this work?

The answer is that it is complicated. Recall the basic decorators that have execution-time wrapping of code. They declare an inner method in local scope that they then return. This is the callable returned by the decorator. It is what is assigned to the function name. Decorators that take arguments add one more wrapping layer to this dance. This is because the decorator that takes the argument is not actually the decorator. Rather, it is a function that returns the decorator, which is a function that takes one argument (the decorated method), which then decorates the function and returns a callable.

That sounds confusing. Consider the following example where a @json_output decorator is augmented to ask about indentation and key sorting:

So, what has happened here, and why does this work?

This is a function, json_output, which accepts two arguments (indent and sort_keys). It returns another function, called actual_decorator, which is (as its name suggests) intended to be used as a decorator. That is a classic decorator – a callable that accepts a single callable (decorated) as an argument and returns a callable (inner).

Note that the inner function has changed slightly to accommodate the indent and sort_keys arguments. These arguments mirror similar arguments accepted by json.dumps, so the call to json.dumps accepts the values provided to indent and sort_keys in the decorator's signature and provides them to json.dumps in the antepenultimate line.

The inner function is what ultimately makes use of the indent and sort_keys arguments. This is fine, because Python's block scoping rules allow for this. It also is not a problem that this might be called with different values for inner and sort_keys, because inner is a local function (a different copy is returned each time the decorator is used).

Applying the json_output function looks like this:

And if you run the do_nothing function now, you get a JSON block back with indentation and newlines added, as shown here:

How Does This Work?

But wait. If json_output is not a decorator, but a function that returns a decorator, why does it look like it is being applied as a decorator? What is the Python interpreter doing here that makes this work?

More explanation is in order. The key here is in the order of operations. Specifically, the function call (json_output(indent=4)) precedes the decorator application syntax (@). Thus, the result of the function call is used to apply the decorator.

The first thing that is happening is that the interpreter is seeing the function call for json_output and resolving that call (note that the boldface does not include the @):

All the json_output function does is define another function, actual_decorator, and return it. As the result of that function, it is then provided to @, as shown here:

Now, actual_decorator is being run. It declares another local function, inner, and returns it. As previously discussed, that function is then assigned to the name do_nothing, the name of the decorated method. When do_nothing is called, the inner function is called, runs the decorated method, and JSON dumps the result with the appropriate indentation.

The Call Signature Matters

It is critical to realize that when you introduced your new, altered json_output function, you actually introduced a backward-incompatible change.

Why? Because now there is this extra function call that is expected. If you want the old json_output behavior, and do not need values for any of the arguments available, you still must call the method.

In other words, you must do the following:

Note the parentheses. They matter, because they indicate that the function is being called (even with no arguments), and then the result is applied to the @.

The previous code is not —repeat, not —equivalent to the following:

This presents two problems. It is inherently confusing, because if you are accustomed to seeing decorators applied without a signature, a requirement to supply an empty signature is counterintuitive. Secondly, if the old decorator already exists in your application, you must go back and edit all of its existing calls. You should avoid backward-incompatible changes if possible.

In a perfect world, this decorator would work for three different types of applications:

● @json_output

● @json_output()

● @json_output(indent=4)

As it turns out, this is possible, by having a decorator that modifies its behavior based on the arguments that it receives. Remember, a decorator is just a function and has all the flexibility of any other function to do what it needs to do to respond to the inputs it gets.

Consider this more flexible iteration of json_output:

This function is endeavoring to be intelligent about whether or not it is currently being used as a decorator.

First, it makes sure it is not being called in an unexpected way. You never expect to receive both a method to be decorated and the keyword arguments, because a decorator is always called with the decorated method as the only argument.

Second, it defines the actual_decorator function, which (as its name suggests) is the actual decorator to be either returned or applied. It defines the inner function that is the ultimate function to be returned from the decorator.

Finally, it returns the appropriate result based on how it was called:

● If decorated_ is set, it was called as a plain decorator, without a method signature, and its responsibility is to apply the ultimate decorator and return the inner function. Here again, observe how decorators that take arguments are actually working. First, actual_decorator(decorated_) is called and resolved, then its result (which must be a callable, because this is a decorator) is called with inner provided as its only argument.

● If decorated_ is not set, then this was called with keyword arguments instead, and the function must return an actual decorator, which receives the decorated method and returns inner. Therefore, the function returns actual_decorator outright. This is then applied by the Python interpreter as the actual decorator (which ultimately returns inner).

Why is this technique valuable? It enables you to maintain your decorator's functionality as previously used. This means that you do not have to update each case where the decorator has been applied. But you still get the additional flexibility of being able to add arguments in the cases where you need them.