Building higher-order functions in Python

Python Building Higher-Order Functions. You can define higher-order functions by creating a Callable class object. Similar to writing generator functions, creating a callable object allows you to use Python statements. In addition to using statements, you can also statically configure the function when creating it.

Defining a Callable class object using the class declaration actually defines a function that returns a function. Callable objects are often used to combine two functions to form a complex function.

As shown in the following class:

from typing import Callable, Optional, Any

class NullAware:
def __init__(
self, some_func: Callable[[Any], Any]) -> None:
self.some_func = some_func
def __call__(self, arg: Optional[Any]) -> Optional[Any]:
return None if arg is None else self.some_func(arg)

This class is used to create new null-aware functions. When creating an instance of this class, you need to provide a function (some_func) as an argument. The constraints of this function are defined by Callable[[Any], Any]: that is, it takes a single value as input and returns a single value. The result is a callable that accepts an optional argument. The __call__() method handles the case where the argument is None. This method defines the return type as Callable[[Optional[Any]], Optional[Any]].

Evaluates the expression NullAware(math.log), creating a new function that acts on the argument. The __init__ method stores the user-defined function in the result object, which is a function that contains the data processing logic.

You typically assign a name to the newly created function so it can be used later, as shown below:

null_log_scale = NullAware(math.log)

Here, we assign the newly created function to the variable null_log_scale. We can then use the function in a new context, as shown below:

>>> some_data = [10, 100, None, 50, 60]
>>> scaled = map(null_log_scale, some_data)
>>> list(scaled)
[2.302585092994046, 4.605170185988092, None, 3.912023005428146,
4.0943445622221]

Or, as follows, create and use the function in the same expression:

>>> scaled = map(NullAware(math.log), some_data)
>>> list(scaled)
[2.302585092994046, 4.605170185988092, None, 3.912023005428146,
4.0943445622221]

Evaluating NullAware(math.log) returns an anonymous function, which is used in the map() function to process the iterable object some_data.

The __call__() method in the above example is based entirely on expression evaluation, providing a concise and easy way to create composite functions from underlying component functions. When using scalar functions, there are few design constraints to consider, but when iterable collections are involved, more careful consideration is required.

Ensuring Correct Functional Design

When using stateless functional Python code, be mindful of using objects, as objects are stateful. In fact, object-oriented programming aims to encapsulate state changes within class definitions. This way, when working with collections using Python class definitions, you’ll find that functional programming diverges from imperative programming.

Using callable objects to create composite functions allows us to use these composite functions with a relatively simple syntax. When using iterables for mapping or reducing, it’s important to be mindful of why and how stateful objects are introduced.

Returning to the previous sum_filter_f() composite function. The implementation based on the Callable class definition is as follows:

from typing import Callable, Iterable

class Sum_Filter:
__slots__ = ["filter", "function"]

def __init__(self,
filter: Callable[[Any], bool],
func: Callable[[Any], float]) -> None:

self.filter = filter

self.function = func

def __call__(self, iterable: Iterable) -> float:

return sum(
self.function(x)
for x in iterable
if self.filter(x)
)

Each object of this class can have only two attributes to reduce the possibility of subsequent use of the function as a stateful object. This restriction does not completely prevent modifications to the returned result object, but limiting the object’s attributes to two ensures that adding additional attributes will cause an exception.

The initialization method __init__() loads two functions into the object instance: filter and func. The return value of the __call__() method is a generator expression that uses the first two functions. The self.filter() method is used to select and discard elements in the collection, while the self.function() function is used to transform elements that pass the filter() function.

Class instances are functions that contain two policy functions. Creating a class instance is as follows:

count_not_none = Sum_Filter(
lambda x: x is not None,
lambda x: 1)

This function counts the number of non-None elements in a sequence. It uses two anonymous functions: one that filters out non-None elements in a sequence, and another that converts all non-None elements to 1.

Overall, this count_not_none() function is like any other Python function, but its usage is a bit simpler than the previous sum_filter_f() function.

Use the count_not_none() function as follows:

N = count_not_none(data)

Use the sum_filter_f() function as follows:

N = sum_filter_f(valid, count_, data)

As you can see, the count_not_none() function, based on a callable object, has fewer parameters than a regular function, making it relatively simple to use. However, the code defining the function’s behavior is split between two places (the definition in the callable class and the parameters specified when using the function), which makes this approach less clear.