7. Concurrency

A warning before we start: most scripts don't need concurrency, and adding it without thinking brings complexity and bugs that are hard to reproduce. But there comes a point (reading a hundred buoys over the network, processing a thousand files) where doing everything in a single line is too slow. This lesson gives you the criteria to know when to reach for it and which tool to use.

Concurrency, parallelism and the GIL

Two words that get confused. Concurrency is managing several tasks that advance overlapping, taking advantage of the moments when one is waiting. Parallelism is running them literally at the same time on several cores of the processor.

The distinction that really matters in practice is a different one: is your work about waiting or about computing?

• I/O-bound: the program spends its time waiting for something external, the network, the disk, a sensor. The processor is twiddling its thumbs.
• CPU-bound: the program burns processor doing calculations, a spectral analysis of the swell, for example.

And here the GIL (Global Interpreter Lock) comes in: in CPython, the standard interpreter, only one Python thread runs at a time. Translated: threads do not give you real computing parallelism, but they are perfect for I/O, because while one thread waits for the network, another advances. From that distinction comes the whole decision rule.

concurrent.futures: threads for the wait

For I/O-bound work, the gateway is ThreadPoolExecutor. It splits the work across several threads and, while some wait, others progress. The API is the same one you already know from map:

from concurrent.futures import ThreadPoolExecutor

buoy_ids = ["north-sea-1", "north-sea-2", "med-1"]

with ThreadPoolExecutor(max_workers=8) as executor:
    weathers = list(executor.map(fetch_weather, buoy_ids))

If fetch_weather takes a second per buoy waiting for the network, doing it in a single line is three seconds; with the pool, a little over one. Notice the with: the executor is a context manager, exactly what you just learned, and on exit it waits for all the tasks to finish and closes itself.

Processes for the computation

When the bottleneck is the processor, threads don't help, because of the GIL. There you need processes, each with its own interpreter and its own GIL. The change is a single word, ProcessPoolExecutor, because it shares the API:

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor() as executor:
    results = list(executor.map(spectral_analysis, big_files))

The price to pay: the data travels between processes serialized (with pickle), so it is best to pass simple things and functions defined at module level. Pure functions, with no shared state, are the ones that split up best.

asyncio: thousands of waits at once

When it is not ten waits but ten thousand (a service that queries many APIs), creating one thread per each doesn't scale. That is what asyncio is for: a single thread that juggles thousands of tasks through async and await.

import asyncio


async def fetch_weather(buoy_id: str) -> float:
    await asyncio.sleep(0.1)  # simulates the network wait
    return 2.3


async def fetch_all(buoy_ids: list[str]) -> list[float]:
    tasks = [fetch_weather(buoy_id) for buoy_id in buoy_ids]
    return await asyncio.gather(*tasks)


results = asyncio.run(fetch_all(["north-sea-1", "med-1"]))

An async function is a coroutine: await marks the points where it yields its turn while it waits, so another task can advance. asyncio.gather launches them all at once and collects the results. To talk to real APIs you would use an async library like httpx or aiohttp, not requests, which is synchronous and would block the thread.

Which to choose

You don't have to decide blindly, there is a clear rule:

And an idea that closes another circle of the course: functional style is your best ally in concurrency. Pure functions don't share mutable state, so they can be split across threads, processes or coroutines with no race conditions and no locks. Shared, mutable state, the one we have spent the whole course avoiding, is precisely what turns concurrency into a hell of bugs impossible to reproduce. You isolate the effects, keep the core pure, and parallelizing it becomes almost free.

For testing, the usual advice: separate the pure logic from the concurrent layer. Test the calculation with values, synchronous and with no surprises; the layer of threads or coroutines stays thin and is checked separately.

Guided example: averaging a folder of logs in parallel

We pick up the folder of CSVs from the previous lesson. Reading files is I/O work, so threads fit. We separate the pure calculation (one function per file) from the concurrent orchestration. In src/metocean_tools/batch.py:

from concurrent.futures import ThreadPoolExecutor
from pathlib import Path


def average_of_file(path: Path) -> float:
    """Average wave height of a single file, in meters."""
    with path.open(encoding="utf-8") as file:
        next(file)  # skip the header
        heights = [float(line.strip().split(",")[1]) for line in file]
    return sum(heights) / len(heights) if heights else 0.0


def averages_for_folder(folder: Path) -> dict[str, float]:
    """Average all the folder's CSVs at once and return name -> average."""
    files = sorted(folder.glob("*.csv"))
    with ThreadPoolExecutor() as executor:
        averages = executor.map(average_of_file, files)
    return {path.name: average for path, average in zip(files, averages)}

Each file is processed in a different thread and, while one waits for the disk, another computes. If the folder has a hundred logs, the difference with doing it in a single line is enormous. And since average_of_file shares no state with the other calls, we haven't needed a single lock.