5. Functional style applied

In this lesson we learn to solve problems by chaining transformations instead of accumulating state. All from practice, without paradigm theory. You will see that many things you used to do with loops and temporary variables come out cleaner this way.

Transforming data with functions

Immutability. The idea is simple: instead of modifying the data you have, you create new data from it. Why? Because if nobody mutates a value, nobody changes it on you by surprise behind your back. Many bugs are "this list was this and now it's something else, who touched it?". With immutability, that question doesn't exist.

Higher-order functions. They are functions that receive or return other functions. It sounds abstract, but you use it all the time once you get the hang of it. The basis for what comes next.

map, filter and comprehensions. Transforming and filtering a sequence without writing a loop by hand:

wave_heights_m = [0.8, 3.1, 1.2, 4.5, 0.5]

# Convert meters to feet
in_feet = [h * 3.281 for h in wave_heights_m]

# Keep only the operable waves (below 2.5 m)
operable = [h for h in wave_heights_m if h < 2.5]

You also have map(function, iterable) and filter(function, iterable), which do the same. In Python, when the transformation is simple, a list comprehension is usually clearer than a map or a filter. Use whichever reads better.

functools.reduce: accumulating results. When you need to condense a sequence into a single value while dragging along an accumulator:

from functools import reduce

wave_heights_m = [0.8, 3.1, 1.2, 4.5, 0.5]

max_wave = reduce(lambda highest, h: h if h > highest else highest, wave_heights_m)

That said, use it with judgment: to sum you have sum, for the maximum you have max. reduce shines when the accumulation doesn't have a ready-made library function.

Lambda functions. A lambda is an anonymous single-expression function, useful for short transformations you pass to another function. They are great when they are brief, and horrible when someone stretches them. If a lambda doesn't fit comfortably on one line, give it a name and make it a normal function.

sorted, min, max with the right key. The key argument is what gives all the power. You pass it a function that says "sort by this":

readings = [
    {"buoy": "A", "wave_height_m": 3.1},
    {"buoy": "B", "wave_height_m": 1.2},
    {"buoy": "C", "wave_height_m": 4.5},
]

# The buoy with the roughest sea
roughest = max(readings, key=lambda r: r["wave_height_m"])

# Sort from lowest to highest sea
by_wave = sorted(readings, key=lambda r: r["wave_height_m"])

Here the lambda is in its element: short, clear and at the service of max and sorted.

Closures, decorators and functools

Closures: functions that remember their context. A closure is a function defined inside another that "remembers" the outer function's variables even after it has finished:

def make_wave_limit_checker(limit_m: float):
    """Creates a function that checks whether a wave exceeds a given limit."""

    def is_within_limit(wave_height_m: float) -> bool:
        return wave_height_m < limit_m

    return is_within_limit


is_safe_for_diving = make_wave_limit_checker(1.5)
is_safe_for_crane = make_wave_limit_checker(2.5)

is_safe_for_diving(1.2)  # True
is_safe_for_crane(1.2)  # True

is_within_limit remembers the limit_m it was created with. We have manufactured two different checkers without repeating the logic or using global variables.

Decorators: adding behavior without touching the function. A decorator wraps a function to add something to it (measure times, cache, log) without modifying its code:

import functools
import time


def measure_time(function):
    """Measures and displays how long the decorated function takes."""

    @functools.wraps(function)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = function(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{function.__name__} took {elapsed:.4f} s")
        return result

    return wrapper


@measure_time
def process_large_dataset(readings: list[float]) -> float:
    return average_wave_height(readings)

The @functools.wraps(function) preserves the name and docstring of the original function, so the decorator doesn't disguise it. This is, by the way, one of the few places where *args, **kwargs is justified: a decorator has to accept any signature.

functools.partial: fixing arguments and reusing. It takes a function and "freezes" some of its arguments, returning a shorter version:

from functools import partial


def is_operational(wave_limit_m: float, wave_height_m: float) -> bool:
    return wave_height_m < wave_limit_m


is_operational_for_diving = partial(is_operational, 1.5)
is_operational_for_diving(1.2)  # True

It is the first cousin of the closure, and often shorter to write. We will come back to it later, when we talk about constants and configuration.

functools.lru_cache: memoization. If a function is expensive and you call it many times with the same arguments, cache the result and don't recompute it:

from functools import lru_cache


@lru_cache(maxsize=128)
def expensive_forecast(wind_speed_kn: float, hours_ahead: int) -> float:
    """Expensive forecast that depends only on its arguments."""
    ...

The second time you call it with the same arguments, the answer is instant. That said, lru_cache only works with pure functions: if it depended on the time or on a file, it would return you stale data. Another reason to write pure functions.

Iterators and generators for large data

This is where the functional style gets serious with real marine data, those sensor files that don't fit in memory.

Iterables and iterators. An iterable is something you can go through (a list, a file). An iterator is the object that delivers the elements one by one, on demand. The key: it doesn't need to have everything loaded at once.

Generators with yield. A generator is a function that, instead of returning everything at once with return, delivers values one by one with yield. Perfect for processing a huge file record by record without loading it whole:

def read_wave_heights(path: str):
    """Reads wave heights from the buoy log, line by line.

    It does not load the whole file into memory: it delivers each value on demand.

    Args:
        path: File path, with the wave height in the second column.

    Yields:
        The wave height of each reading, in meters.
    """
    with open(path) as file:
        next(file)  # Skip the header
        for line in file:
            fields = line.strip().split(",")
            yield float(fields[1])

Even if the file has millions of buoy readings, only one line is in memory at a time.

Lazy evaluation. Generators compute only what is used. If you chain transformations and at the end only look at the first ten results, the generator only processes those ten. It doesn't do extra work.

The itertools module. The toolbox for chaining and grouping sequences lazily. Some you will use:

import itertools

# The first 100 records of a long stream of readings
first_hundred = itertools.islice(read_wave_heights("buoy.csv"), 100)

# Chain several files as if they were one
all_readings = itertools.chain(
    read_wave_heights("day1.csv"),
    read_wave_heights("day2.csv"),
)

Guided example: processing a buoy log without loading it into memory

Let's bring together generators and chained transformations to process a file of millions of buoy readings, filtering and transforming without loading anything whole:

def clean_wave_heights(path: str):
    """Delivers valid wave heights, in feet.

    Reads the file record by record, discards the impossible readings
    (negative or absurdly high) and converts from meters to feet.
    """
    raw_heights = read_wave_heights(path)
    valid_heights = (height for height in raw_heights if 0.0 <= height < 20.0)
    in_feet = (height * 3.281 for height in valid_heights)
    return in_feet


def average_wave_height_feet(path: str) -> float:
    """Calculates the average wave height of the whole log, in feet."""
    total = 0.0
    count = 0
    for height in clean_wave_heights(path):
        total += height
        count += 1
    return total / count if count else 0.0

Each (... for ... in ...) is a generator expression: it defines a lazy transformation that doesn't run until someone requests a value. The read -> filter -> convert chain processes one record at a time, from start to finish, with constant memory consumption whether the file weighs a megabyte or a gigabyte. This is what a loop with intermediate lists doesn't give you: if you loaded everything into lists, you would run out of memory.

Constants and configuration without breaking purity

Here there is a real tension worth understanding well, because it affects how you structure all of your code.

Under the functional style, functions should be pure. But a pure function should not read a global constant, because then it depends on something external that doesn't appear in its signature. And constants, even if they don't change, complicate the tests: you have to import them or simulate them.

The problem, with an example:

WAVE_LIMIT_M = 2.5


def is_operational(wave_height_m: float) -> bool:
    return wave_height_m < WAVE_LIMIT_M

It looks harmless, but what happens if the operational limit changes by zone? And if where you call this function you already have your own limit? The function has stopped being portable, and to test it with a different limit you have to touch the global constant. It has become fragile.

Solution 1: inject the constant and fix it with partial. The base function receives everything as parameters, and then we create the concrete versions by fixing the constants:

from functools import partial


def is_operational(wave_limit_m: float, wave_height_m: float) -> bool:
    return wave_height_m < wave_limit_m


# Versions by zone, fixing the limit of each one
is_operational_north_sea = partial(is_operational, 2.0)
is_operational_mediterranean = partial(is_operational, 2.8)

is_operational_north_sea(1.8)  # True
is_operational_mediterranean(2.5)  # True

The base function is one hundred percent reusable and trivial to test: you pass it different configurations as arguments and that's it. We gain maximum purity. The cost is some visual complexity if your team isn't used to partial.

Solution 2: encapsulate the constants in a closure (factory pattern). You group the functions that share context inside a constructor function, without resorting to classes or mutable state:

def create_operational_rules(wave_limit_m: float, wind_limit_kn: float):
    """Creates the operational rules for a specific zone."""

    def is_operational(wave_height_m: float, wind_speed_kn: float) -> bool:
        return wave_height_m < wave_limit_m and wind_speed_kn < wind_limit_kn

    def margin_to_limit(wave_height_m: float) -> float:
        return wave_limit_m - wave_height_m

    return is_operational, margin_to_limit


north_sea_operational, north_sea_margin = create_operational_rules(2.0, 18.0)

The constants stay trapped in the closure and we logically group the functions that share context. The drawback is that it can be hard to read if the inner functions grow too much.

The trade-off. There is no single answer, and this is the important thing you take away from this section: the more a function looks outward, the easier it is to write, but the harder it is to test in isolation. Python is multi-paradigm, so you are not forced to marry one approach. What you must do is be aware of the trade-off and choose on purpose, not out of inertia.