Python high-order mapping and filtering functions

Python High-order mapping and filtering functions. Python’s two built-in high-order functions, map() and filter(), can handle virtually any data. Generally speaking, optimizing these two functions for higher performance is difficult. Later, we will introduce several functions of Python 3.4, including imap(), ifilter(), and ifilterfalse().

There are three ways to express a mapping relationship. Suppose there is a function f(x) and a collection object C. The mapping relationship can be expressed in the following three ways:

• map() function: map(f, C)
• Generator expression: (f(x) for x in C)
• Generator function containing a yield statement:

def mymap(f, C):
for x in C:
yield f(x)
mymap(f, C)

Similarly, the filter() function can be applied to a collection in the following three ways.

• filter() function: filter(f, C).
• Generator expression: (x for x in C if f(x)).
• Generator function containing a yield statement:

def myfilter(f, C):
for x in C:
if f(x):
yield x
myfilter(f, C)

Performance varies somewhat between approaches, with map() and filter() being the fastest. More importantly, the map and filter methods above can be extended in various ways, as shown below.

• You can create a more complex function g(x) that applies to each element, or apply a function to the collection before processing. These approaches apply to all three design patterns above and are areas where functional design excels.
• Modify the for loop in a generator expression or generator function. A common approach is to add an if clause to the generator expression, combining the mapping and filtering operations into a single operation. Alternatively, you can combine the mymap() and myfilter() functions to combine the mapping and filtering operations.

As software continues to improve and mature, it’s common to modify loop bodies to adjust data structures. There are several design patterns for this, including packing, unpacking (or spreading), flattening, and structuring, several of which have been discussed previously.

When designing mappings, be wary of functions that contain too many transformations. Avoid creating functions whose primary purpose is unclear. Because Python lacks an optimizing compiler, you may have to manually optimize poorly performing applications by combining functions. However, use these optimizations only as a last resort—that is, only when performance analysis indicates that the application requires them.