4. Functional style applied

In this lesson we learn to solve problems by chaining transformations instead of accumulating state. All from practice, without paradigm theory. You will see that many things you used to do with loops and temporary variables come out cleaner this way.

Transforming data with functions

Immutability. The idea is simple: instead of modifying the data you have, you create new data from it.

wave_heights_m = [0.8, 3.1, 1.2]

# Mutate: you change the original list, and whoever shared it gets the surprise
wave_heights_m.append(4.5)

# Immutable: you don't touch the original, you create a new one with the change
with_new_reading = [*wave_heights_m, 4.5]

Why prefer the second form? Because if nobody mutates a value, nobody changes it on you by surprise behind your back. Many bugs are "this list was this and now it's something else, who touched it?". With immutability, that question doesn't exist.

Higher-order functions. They are functions that receive or return other functions. It sounds abstract, but it is exactly what apply_to_all does here:

def apply_to_all(function, wave_heights_m):
    """Apply a function to each height in the list."""
    return [function(height) for height in wave_heights_m]


def to_feet(height_m):
    return height_m * 3.281


apply_to_all(to_feet, [0.8, 3.1, 1.2])  # [2.62, 10.17, 3.94]

apply_to_all receives another function as an argument and uses it. map, filter and sorted(key=...), which you see just below, are exactly this. Once you get the hang of it, you use it all the time.

map, filter and comprehensions. Transforming and filtering a sequence without writing a loop by hand:

wave_heights_m = [0.8, 3.1, 1.2, 4.5, 0.5]

# Convert meters to feet
in_feet = [h * 3.281 for h in wave_heights_m]

# Keep only the operable waves (below 2.5 m)
operable = [h for h in wave_heights_m if h < 2.5]

You also have map(function, iterable) and filter(function, iterable), which do the same. In Python, when the transformation is simple, a list comprehension is usually clearer than a map or a filter. Use whichever reads better.

Lambda functions. A lambda is an anonymous single-expression function, useful for short transformations you pass to another function. They are great when they are brief, and horrible when someone stretches them. If a lambda doesn't fit comfortably on one line, give it a name and make it a normal function.

sorted, min, max with the right key. The key argument is what gives all the power. You pass it a function that says "sort by this":

readings = [
    {"buoy": "A", "wave_height_m": 3.1},
    {"buoy": "B", "wave_height_m": 1.2},
    {"buoy": "C", "wave_height_m": 4.5},
]

# The buoy with the roughest sea
roughest = max(readings, key=lambda r: r["wave_height_m"])

# Sort from lowest to highest sea
by_wave = sorted(readings, key=lambda r: r["wave_height_m"])

Here the lambda is in its element: short, clear and at the service of max and sorted.

Closures, decorators and functools

Closures: functions that remember their context. A closure is a function defined inside another that "remembers" the outer function's variables even after it has finished:

def make_wave_limit_checker(limit_m: float):
    """Creates a function that checks whether a wave exceeds a given limit."""

    def is_within_limit(wave_height_m: float) -> bool:
        return wave_height_m < limit_m

    return is_within_limit


is_safe_for_diving = make_wave_limit_checker(1.5)
is_safe_for_crane = make_wave_limit_checker(2.5)

is_safe_for_diving(1.2)  # True
is_safe_for_crane(1.2)  # True

is_within_limit remembers the limit_m it was created with. We have manufactured two different checkers without repeating the logic or using global variables.

Decorators: adding behavior without touching the function. A decorator wraps a function to add something to it (measure times, cache, log) without modifying its code:

import functools
import time


def measure_time(function):
    """Measures and displays how long the decorated function takes."""

    @functools.wraps(function)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = function(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{function.__name__} took {elapsed:.4f} s")
        return result

    return wrapper


@measure_time
def process_large_dataset(readings: list[float]) -> float:
    return average_wave_height(readings)

The @functools.wraps(function) preserves the name and docstring of the original function, so the decorator doesn't disguise it. This is, by the way, one of the few places where *args, **kwargs is justified: a decorator has to accept any signature.

functools.partial: fixing arguments and reusing. It takes a function and "freezes" some of its arguments, returning a shorter version:

from functools import partial


def is_operational(wave_limit_m: float, wave_height_m: float) -> bool:
    return wave_height_m < wave_limit_m


is_operational_for_diving = partial(is_operational, 1.5)
is_operational_for_diving(1.2)  # True

It is the first cousin of the closure, and often shorter to write. We will come back to it later, when we talk about constants and configuration.

functools.lru_cache: memoization. If a function is expensive and you call it many times with the same arguments, cache the result and don't recompute it:

from functools import lru_cache


@lru_cache(maxsize=128)
def expensive_forecast(wind_speed_kn: float, hours_ahead: int) -> float:
    """Expensive forecast that depends only on its arguments."""
    ...

The second time you call it with the same arguments, the answer is instant. That said, lru_cache only works with pure functions: if it depended on the time or on a file, it would return you stale data. Another reason to write pure functions.

Iterators and generators for large data

This is where the functional style gets serious with real marine data, those sensor files that don't fit in memory.

Iterables and iterators. An iterable is something you can go through (a list, a file). An iterator is the object that delivers the elements one by one, on demand. With iter you get the iterator out of an iterable, and with next you ask it for the next value:

wave_heights_m = [0.8, 3.1, 1.2]  # an iterable

readings = iter(wave_heights_m)  # its iterator
next(readings)  # 0.8
next(readings)  # 3.1
next(readings)  # 1.2
next(readings)  # StopIteration: it's over

When the iterator runs out of elements, it raises StopIteration. A for loop does exactly this under the hood: it calls iter once and next until that signal fires. The key is that the iterator doesn't need to have everything loaded at once, and the generators coming up next are iterators you build yourself.

Generators with yield. A generator is a function that, instead of returning everything at once with return, delivers values one by one with yield. Perfect for processing a huge file record by record without loading it whole:

def read_wave_heights(path: str):
    """Reads wave heights from the buoy log, line by line.

    It does not load the whole file into memory: it delivers each value on demand.

    Args:
        path: File path, with the wave height in the second column.

    Yields:
        The wave height of each reading, in meters.
    """
    with open(path) as file:
        next(file)  # Skip the header
        for line in file:
            fields = line.strip().split(",")
            yield float(fields[1])

Even if the file has millions of buoy readings, only one line is in memory at a time.

Lazy evaluation. Generators compute only what is used. See it with an endless generator: if it weren't lazy, it would hang the program forever.

def wave_heights_forever():
    """Generate wave readings without end, one at a time."""
    height = 0.0
    while True:
        print(f"computing {height}")
        yield height
        height += 0.5


heights = wave_heights_forever()
first_three = [next(heights) for _ in range(3)]
# computing 0.0
# computing 0.5
# computing 1.0

The loop is infinite, but we only ask for three values, so the generator computes just those three and stops. If you chain transformations and at the end only look at the first ten results, the generator processes those ten and not one more. It doesn't do extra work.

The itertools module. The toolbox for chaining and grouping sequences lazily. Some you will use:

import itertools

# The first 100 records of a long stream of readings
first_hundred = itertools.islice(read_wave_heights("buoy.csv"), 100)

# Chain several files as if they were one
all_readings = itertools.chain(
    read_wave_heights("day1.csv"),
    read_wave_heights("day2.csv"),
)

Constants and configuration without breaking purity

Here there is a real tension worth understanding well, because it affects how you structure all of your code.

Under the functional style, functions should be pure. But a pure function should not read a global constant, because then it depends on something external that doesn't appear in its signature. And constants, even if they don't change, complicate the tests: you have to import them or simulate them.

The problem, with an example:

WAVE_LIMIT_M = 2.5


def is_operational(wave_height_m: float) -> bool:
    return wave_height_m < WAVE_LIMIT_M

It looks harmless, but what happens if the operational limit changes by zone? And if where you call this function you already have your own limit? The function has stopped being portable, and to test it with a different limit you have to touch the global constant. It has become fragile.

Solution 1: inject the constant and fix it with partial. The base function receives everything as parameters, and then we create the concrete versions by fixing the constants:

from functools import partial


def is_operational(wave_limit_m: float, wave_height_m: float) -> bool:
    return wave_height_m < wave_limit_m


# Versions by zone, fixing the limit of each one
is_operational_north_sea = partial(is_operational, 2.0)
is_operational_mediterranean = partial(is_operational, 2.8)

is_operational_north_sea(1.8)  # True
is_operational_mediterranean(2.5)  # True

The base function is one hundred percent reusable and trivial to test: you pass it different configurations as arguments and that's it. We gain maximum purity. The cost is some visual complexity if your team isn't used to partial.

Solution 2: encapsulate the constants in a closure (factory pattern). You group the functions that share context inside a constructor function, without resorting to classes or mutable state:

def create_operational_rules(wave_limit_m: float, wind_limit_kn: float):
    """Creates the operational rules for a specific zone."""

    def is_operational(wave_height_m: float, wind_speed_kn: float) -> bool:
        return wave_height_m < wave_limit_m and wind_speed_kn < wind_limit_kn

    def margin_to_limit(wave_height_m: float) -> float:
        return wave_limit_m - wave_height_m

    return is_operational, margin_to_limit


north_sea_operational, north_sea_margin = create_operational_rules(2.0, 18.0)

The constants stay trapped in the closure and we logically group the functions that share context. The drawback is that it can be hard to read if the inner functions grow too much.

The trade-off. There is no single answer, and this is the important thing you take away from this section: the more a function looks outward, the easier it is to write, but the harder it is to test in isolation. Python is multi-paradigm, so you are not forced to marry one approach. What you must do is be aware of the trade-off and choose on purpose, not out of inertia.