Python教程:Python函数和Python闭包
原文链接:https://www.codevoila.com/post/51/python-tutorial-python-function-and-python-closure
Python functions beyond basics and a deep look at closure in Python.
All examples in this article use Python 3.3+.
1 Python functions beyond basics
Let’s introduce several important features of Python functions first.
1.1 Python function as a variable
Python function, essentially is also object, like a common Python variable, can be assigned to a variable.
Let’s see a simple demo. First of all, define a random string. Then call print
and len
function and pass the defined string.
>>> slogan = "Life is short, I use Python"
>>> print
>>> <built-in function print>
>>> print(slogan)
>>> Life is short, I use Python
>>> len
>>> <built-in function len>
>>> len(slogan)
>>> 27
These work fine, nothing special.
Next step:
- assign
print
to a new variableoriginal_print
- then assign
len
toprint
.
>>> original_print = print # assign print to original_print
>>> original_print # original_print now becomes print
>>> <built-in function print>
>>> original_print(slogan)
>>> Life is short, I use Python
>>> print = len # assign len to print
>>> print # print now becomes len
>>> <built-in function len>
>>> print(slogan)
>>> 27
The conclusion here is straightforward: Python function can be assigned to a variable.
1.2 Python function as function argument
A Python function can be passed as a argument to a function.
1.2.1 Custom example
Let’s give a smart_add
function as an example. It takes three arguments and the third argument is a function.
def smart_add(x, y, f):
return f(x) + f(y)
Try to call smart_add
and pass abs
function as the third argument to it.
>>> smart_add(-3, 7, abs) # abs(-3) + abs(7)
>>> 10
Again, try to pass another one: math.sqrt
.
>>> import math
>>> smart_add(4, 9, math.sqrt) # math.sqrt(4) + math.sqrt(9)
>>> 5.0
1.2.2 Built-in example: map
>>> help(map)
# ...
map(func, *iterables) --> map object
# ...
According to the document: map
function will make an iterator that computes the function using arguments from each of the Iterables. Stops when the shortest Iterable is exhausted.
In short, map
will take each item x
in iterables
and map it to func(x)
.
x in iterables |--map to--> func(x)
The first parameter of map
is a function, and the second one is an Iterable collection. For example, pass len
function as the first argument and map string to its length.
>>> names = ["Tom", "Jerry", "Bugs Bunny"]
>>> mapped_obj = map(len, names)
>>> mapped_obj
>>> <map object at 0x102629320>
>>> print(list(mapped_obj))
[3, 5, 10]
Many other functions in Python are similar to map
, which takes a function as argument, such as reduce
, filter
.
You may have noticed that map
function can be replaced with list comprehensions.
# same effect as the map function
[func(item) for item in iterables]
In fact, the reduce
function was demoted from built-in in Python 2.x to the functools
module in Python 3 on that account. But the map
and filter
functions are still built-ins in Python 3.
Anyway, what we learned from this part is that Python function can be passed as an argument to a function.
1.3 Return a function in a Python function
Let’s define an inner
function within an outer
function and then return this inner
function from outer
.
def outer():
print('call outer() ...')
# define an inner function within the outer function
def inner():
print('call inner() ...')
# return the inner function
return inner
Call the outer
function and notice that the returned result is a function.
>>> r = outer() # call outer()
call outer() ...
>>> r # Returned result by calling outer() is a function
>>> <function outer.<locals>.inner at 0x1089c6d08>
>>> r() # Call the returned function
call inner() ...
One important thing to remember is not to confuse “return a function” with “return a data value”.
import math
def demo_one():
return math.sqrt # return a function
def demo_two(x):
return math.sqrt(x) # return a data value
Look at another example below.
# pow_later.py
def pow_later(x):
y = 2
def lazy_pow():
print('calculate pow({}, {})...'.format(x, y))
return pow(x, y) # Use Python built-in function: pow
return lazy_pow
Try it in Python shell.
>>> from pow_later import pow_later
>>> my_pow = pow_later(3)
>>> my_pow
>>> <function pow_later.<locals>.lazy_pow at 0x10a043d08>
pow_later
returns a function that will actually calculate the result of pow(3, 2)
in the future.
So call it when you need, and you will get the real calculated result:
>>> my_pow()
calculate pow(3, 2)...
9
1.4 Bonus: higher-order function and first-class function
A function that meet at least one of the following criteria is called a higher-order function.
- takes one or more functions as arguments
- returns a function as its result
In fact, A Python function is not only a higher-order function, but also a first-class function, which satisfies following four criteria:
- can be created at runtime
- can be assigned to a variable
- can be passed as a argument to a function
- can be returned as the result of a function
2 Python closure
Now take a deeper look at the latest example mentioned above.
def pow_later(x):
y = 2
def lazy_pow():
print('calculate pow({}, {})...'.format(x, y))
return pow(x, y)
return lazy_pow
We called pow_later(3)
and it returned a function object.
>>> my_pow = pow_later(3)
>>> my_pow
>>> <function pow_later.<locals>.lazy_pow at 0x10a043d08>
then we invoked the returned function object.
>>> my_pow()
calculate pow(3, 2)...
9
Obviously, the variable y
and the parameter x
are local variables of pow_later
function. So when my_pow()
was called, the pow_later
function had already returned, and its local variables also had gone. But in fact my_pow()
still remembered the values of x
and y
even the outer scope pow_later
was long gone. How did this happen?
2.1 Free variable
If a variable in a function is neither a local variable nor a parameter of that function, this variable is called a free variable of that function.
In short, free variables are variables that are used locally, but defined in an enclosing scope.
In our case, x
is a parameter of pow_later
and y
is a local variable of pow_later
. But within lazy_pow
, x
and y
are free variables.
2.2 Closure
2.2.1 What is closure
Specifically speaking, my_pow
, actually the function object returned by calling pow_later(x)
, is a closure.
Note that the closure for lazy_pow
extends the scope of lazy_pow
function to include the binding for the free variables: x
and y
.
Generally speaking, a closure is a structure (code blocks, function object, callable object, etc.) storing a function together with an environment. The environment here means information about free variables that function bounded, especially values or storage locations of free variables.
For example, a closure is created, returned and assigned to my_pow
after following function call.
>>> my_pow = pow_later(3)
Essentially, this closure is the codes of function lazy_pow
together with free variables x
and y
.
2.2.2 Inspect closure
You can see that the closure keeps names of free variables by inspecting __code__
attribute of my_pow
function which represents the compiled body of the function.
>>> my_pow.__code__.co_freevars
>>> ('x', 'y')
Meanwhile, pow_later
will also keep names of local variables that are referenced by its nested functions in co_cellvars
attribute of its code object.
>>> pow_later.__code__.co_cellvars
>>> ('x', 'y')
However, where is the values of free variables?
>>> dir(my_pow)
>>> my_pow.__closure__
>>> (<cell at 0x10a428348: int object at 0x109e06b60>, <cell at 0x10a428378: int object at 0x109e06b40>)
Note that my_pow
has an attribute named __closure__
and it’s a tuple with two elements.
>>> dir(my_pow.__closure__[0])
>>> ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'cell_contents']
>>> my_pow.__closure__[0].cell_contents
>>> 3
>>> my_pow.__closure__[1].cell_contents
>>> 2
So __closure__
is a tuple of cells
that contain bounded values of free variables.
If your Python version is 3.3+, you can use inspect
module to inspect. The nonlocals
dictionary in inspecting result is exactly the bounded free variables and their values.
>>> import inspect
>>> inspect.getclosurevars(my_pow)
ClosureVars(nonlocals={'x': 3, 'y': 2}, globals={}, builtins={'print': <built-in function print>, 'pow': <built-in function pow>, 'format': <built-in function format>}, unbound=set())
2.2.3 closure
Functions without free variables are not closures.
def f(x):
def g():
pass
return g
Note that returned function g
has no free variable. And its __closure__
is None
.
>>> h=f(1)
>>> h
>>> <function f.<locals>.g at 0x10f650158>
>>> h.__code__.co_freevars
>>> ()
>>> print(h.__closure__)
>>> None
Global variables are not free variables in Python. So global functions are not closures.
>>> data=200 # global
>>> def d(): # global
>>> print(data)
...
...
>>> d()
>>> 200
>>> d.__code__.co_freevars
>>> ()
>>> print(d.__closure__)
>>> None
__closure__
attribute of global functions is None
.
2.2.4 nonlocal declaration
Let’s review our pow_later(x)
function.
- pass a number
x
to functionpow_later
; pow_later
will return a function object;- the returned function object
my_pow
will calculatex**2
(y=2
) each time it is called.
Now I’d like to change above behavior, let y
increase 1 automatically each time my_pow
is called. That is:
- the first time call, calculate
x**2
; - the second time call, calculate
x**3
; - the third time call, calculate
x**4
; - ….
The updated source codes are as follows.
# pow_later.py
def pow_later(x):
y = 2
def lazy_pow():
print('calculate pow({}, {})...'.format(x, y))
result = pow(x, y)
y = y + 1 # increase y
return result
return lazy_pow
Try it in Python shell.
>>> from pow_later import pow_later
>>> my_pow = pow_later(3)
>>> my_pow
>>> <function pow_later.<locals>.lazy_pow at 0x108e020d0>
So far so good, let’s call my_pow
to see result.
>>> my_pow()
>>> Traceback (most recent call last):
...
UnboundLocalError: local variable 'y' referenced before assignment
The error message is clear enough.
- It’s a
UnboundLocalError
y
is a local variable- local variable
y
referenced before assignment
The problem happens in this line: y = y + 1
.
We are actually assigning to y
in lazy_pow
scope, and that makes y
becomes local to lazy_pow
scope. So Python considers y
a local variable of lazy_pow
. Before assigning to that local variable, Python will first read the local variable y
. But y
is a free variable as mentioned earlier and there is no local variable named y
in lazy_pow
scope at all.
You may think, OK, we don’t assign! How about use y += 1
instead of y = y + 1
? The +=
operation is performed in-place, meaning that rather than creating and assigning a new value to the variable, the old variable is modified instead.
The answer is: no change here. Because y
is a number, which is an immutable type. +=
will also create a new number object with new value behind the scene and assign the reference of the new object to y
.
To deal with this situation, a nonlocal
declaration was introduced in Python 3. It marks a variable as a free variable even though it is assigned a new value within the function.
# pow_later.py
def pow_later(x):
y = 2
def lazy_pow():
nonlocal y # nonlocal declaration
print('calculate pow({}, {})...'.format(x, y))
result = pow(x, y)
y = y + 1
return result
return lazy_pow
Now the closure works well.
>>> from pow_later import pow_later
>>> my_pow = pow_later(3)
>>> my_pow()
>>> calculate pow(3, 2)...
9
>>> my_pow()
>>> calculate pow(3, 3)...
27
>>> my_pow()
>>> calculate pow(3, 4)...
81
3 Summary
Two topics were discussed in this article.
First, Python functions are first-class functions.
Second, what is closure and how it works in Python.