Python function features

Functions are first-class objects in Python. They can be assigned to variables, stored in data structures, passed as arguments to other functions, and even returned as return values from other functions.

A solid understanding of these concepts will not only help you understand advanced Python features like lambdas and decorators, but will also expose you to functional programming techniques.
The following pages will provide examples to help you build an intuitive understanding of these concepts. These examples are presented in a step-by-step manner, so it’s important to read them sequentially and practice them repeatedly in your Python interpreter sessions.

It may take a while to understand these concepts. Don’t worry; this is perfectly normal; I’ve been there too. You might feel lost at first, but as time goes on, things will become clearer.

In this section, we’ll use the following function, yell, to demonstrate its functionality. This is a simple example with minimal output.

def yell(text):
return text.upper() + '!'

>>> yell('hello')
'HELLO!'

Python Function Characteristics: Functions are Objects

All data in a Python program is represented by objects or relationships between objects. Strings, lists, modules, and so on are all objects. Python functions are no exception; they are also objects.

Since the yell function is a Python object, it can be assigned to another variable like any other object:

>>> bark = yell

This line doesn’t call the function. Instead, it gets the function object referenced by yell and creates a name named bark that points to that object. Now calling bark executes the same underlying function object:

>>> bark('woof')
'WOOF!'

The function object and its name are independent entities. Let’s verify this. First, delete the original function name (yell). Since the other name (bark) still refers to the underlying function, we can still call the function through bark:

>>> del yell

>>> yell('hello?')
NameError: "name 'yell' is not defined"

>>> bark('hey')
'HEY!'

Incidentally, Python attaches a string identifier to each function when it is created for debugging purposes. This internal identifier can be accessed using the __name__ attribute: Starting with Python 3.3, a similar feature, __qualname__, was added, returning a qualified name string to disambiguate function and class names (see PEP 3155).

>>> bark.__name__
'yell'

Although the function’s __name__ is still yell, it is no longer accessible from source code using this name. Name identifiers are only used to aid debugging; the variable pointing to the function and the function itself are actually independent of each other.

Python Function Features: Functions Can Be Stored in Data Structures

Because functions are first-class objects, they can be stored in data structures like other objects. For example, you can add functions to a list:

>>> funcs = [bark, str.lower, str.capitalize]
>>> funcs
[<function yell at 0x10ff96510>,
<method 'lower' of 'str' objects>,
<method 'capitalize' of 'str' objects>]

Accessing function objects stored in a list is the same as accessing any other type of object:

>>> for f in funcs:
... print(f, f('hey there'))
<function yell at 0x10ff96510> 'HEY THERE!'
<method 'lower' of 'str' objects> 'hey there'
<method 'capitalize' of 'str' objects> 'Hey there'

Function objects stored in a list can be called directly without first assigning them to a variable. For example, you can look up a function in a single expression and then immediately call the “empty” function object.

>>> funcs[0]('heyho')
'HEYHO!'

Python Function Features: Functions can be passed to other functions

Since functions are objects, they can be passed as arguments to other functions. The following greet function takes another function object as an argument, uses that function to format a greeting string, and then prints the result:

def greet(func):
greeting = func('Hi, I am a Python program')
print(greeting)

Passing different functions will produce different results. Passing the bark function to the greet function yields the following result:

>>> greet(bark)
'HI, I AM A PYTHON PROGRAM!'

Of course, you can also define new functions to generate different greetings. For example, if you don’t want this Python program to sound like Optimus Prime when greeting, you can use the following whisper function:

def whisper(text):
return text.lower() + '...'

>>> greet(whisper)
'Hi, I am a Python program...'

Passing function objects as arguments to other functions is a powerful way to abstract and distribute behavior within a program. In this example, the greet function remains unchanged, but passing different greeting behaviors produces different results.

Functions that accept other functions as arguments are called higher-order functions. Higher-order functions are an essential part of the functional programming style.

A representative higher-order function in Python is the built-in map function. map takes a function object and an iterable object, then applies the function to each element in the iterable to produce the result.

The following formats a string by mapping the bark function onto multiple greetings:

>>> list(map(bark, ['hello', 'hey', 'hi']))
['HELLO!', 'HEY!', 'HI!']

As you can see above, map iterates over the entire list and applies the bark function to each element. As a result, we now have a new list object containing the modified greeting string.

Python Function Features: Functions Can Be Nested

It might be a little surprising, but Python allows you to define functions within functions, often called nested functions or inner functions. Consider the following example:

def speak(text):
def whisper(t):
return t.lower() + '...'
return whisper(text)

>>> speak('Hello, World')
'hello, world...'

What’s happening here? Every time you call speak, a new inner function whisper is defined and immediately called. This is where I get a little lost, but it’s relatively straightforward.

But there’s a problem. whisper only exists inside speak:

>>> whisper('Yo')
NameError:
"name 'whisper' is not defined"

>>> speak.whisper
AttributeError:
"'function' object has no attribute 'whisper'"

So how can we access the nested whisper function from outside speak? Since functions are objects, we can return inner functions to the parent function’s caller.

For example, the following function defines two inner functions. The top-level function returns the corresponding internal function to the caller based on the passed arguments:

def get_speak_func(volume):

def whisper(text):

return text.lower() + '...'

def yell(text):

return text.upper() + '!'

if volume > 0.5:

return yell

else:

return whisper

Note that get_speak_func does not actually call any internal functions; it simply selects the appropriate internal function based on the volume argument and then returns the function object:

>>> get_speak_func(0.3)
<function get_speak_func.<locals>.whisper at 0x10ae18>

>>> get_speak_func(0.7)
<function get_speak_func.<locals>.yell at 0x1008c8>

The returned function can be called directly or by assigning a variable name:

>>> speak_func = get_speak_func(0.7)
>>> speak_func('Hello')
'HELLO!'

This concept needs to be understood for a moment. It means that functions can not only accept behavior through arguments, but can also return behavior. Pretty cool, right?
That’s a lot of information. I’m going to take a coffee break before continuing (and I suggest you do too).

Python Function Features: Functions Capture Local State

As mentioned earlier, functions can contain inner functions, and even return inner functions (which are invisible by default) from parent functions.

Now get ready, we’re about to dive into the deeper realms of functional programming. (You took a break, didn’t you?)

Inner functions can not only return from their parent functions, but they can also capture and carry some of the parent function’s state. What does this mean?

Let’s make some minor changes to the previous get_speak_func example to illustrate this step-by-step. The new version uses the volume and text parameters internally, so the returned function can be called directly:

def get_speak_func(text, volume):
def whisper():
return text.lower() + '...'
def yell():
return text.upper() + '!'
if volume > 0.5:
return yell
else:
return whisper

>>> get_speak_func('Hello, World', 0.7)()
'HELLO, WORLD!'

Look closely at the inner functions whisper and yell. Notice that they don’t have a text parameter. However, somehow, the inner functions still have access to the text parameter defined in the parent function. They seem to capture and “remember” the value of this parameter.

Functions with this behavior are called lexical closures, or closures for short. A closure remembers the values of the enclosing scope even when program flow is outside the closure’s scope.

Practically, this means that functions can not only return behavior but also pre-configure that behavior. Let’s use another example to illustrate:

def make_adder(n):
def add(x):
return x + n
return add

>>> plus_3 = make_adder(3)
>>> plus_5 = make_adder(5)

>>> plus_3(4)
7
>>> plus_5(4)
9

In this example, make_adder acts as a factory function to create and configure various adder functions. Note that these adder functions can still access the parameter n in the enclosing scope of the make_adder function.

Python Function Features: Objects can also be used as functions

Although all functions in Python are objects, the reverse is not true. Some objects are not functions, but they are still callable, so in many cases they can be treated as functions.

If an object is callable, it means that you can use the parenthesized function call syntax and even pass in arguments. This is done by the double-underscore method __call__. The following class defines a callable object:

class Adder:
def __init__(self, n):
self.n = n

def __call__(self, x):
return self.n + x

>>> plus_3 = Adder(3)
>>> plus_3(4)
7

Behind the scenes, “calling” an object instance like a function is actually trying to execute the object’s __call__ method.

Of course, not all objects are callable, so Python has a built-in function called callable to check whether an object is callable.

>>> callable(plus_3)
True
>>> callable(yell)
True
>>> callable('hello')
False

Key Points about Python Functions

• Everything in Python is an object, including functions. Functions can be assigned to variables or stored in data structures. As first-class objects, functions can be passed to other functions or returned from them.

• The first-class nature of functions can be used to abstract and communicate program behavior.

• Functions can be nested and can capture and carry around some of the state of their parent function. Functions that exhibit this behavior are called closures.

• Objects can be made callable, so they can be treated as functions in many cases.