Lesson 27 of 40 Async Advanced ⏱ 35 min

Threading, Multiprocessing & concurrent.futures

Use threads for I/O-bound tasks, processes for CPU-bound work, ThreadPoolExecutor, ProcessPoolExecutor, and the GIL explained.

Part 1: What You Will Learn

Distinguish I/O-bound from CPU-bound tasks.
Use ThreadPoolExecutor for concurrent waiting tasks.
Use ProcessPoolExecutor for CPU-heavy work.
Understand why the GIL affects CPU-bound Python threads.

Part 2: Key Concepts

Threads share one process and are lightweight, making them useful when tasks spend much of their time waiting for I/O. Processes have separate Python interpreters and can run CPU-heavy Python code in parallel across cores. concurrent.futures provides a common high-level API for both approaches.

Part 3: Topic-Specific Code Example

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import time


def simulated_download(file_id: int) -> str:
    time.sleep(1)  # Represents network or file I/O.
    return f"file-{file_id} downloaded"


def count_primes(limit: int) -> int:
    count = 0
    for number in range(2, limit + 1):
        is_prime = True
        divisor = 2
        while divisor * divisor <= number:
            if number % divisor == 0:
                is_prime = False
                break
            divisor += 1
        if is_prime:
            count += 1
    return count


def main() -> None:
    with ThreadPoolExecutor(max_workers=4) as executor:
        results = executor.map(simulated_download, range(1, 5))
        print(list(results))

    limits = [40_000, 42_000, 44_000, 46_000]
    with ProcessPoolExecutor() as executor:
        prime_counts = executor.map(count_primes, limits)
        print(list(zip(limits, prime_counts)))

if __name__ == "__main__":
    main()

Part 4: How the Example Works

The download simulation mostly waits, so threads can overlap that waiting time effectively. Prime counting spends its time executing Python calculations; processes can use multiple CPU cores without being constrained by one process’s Global Interpreter Lock (GIL). On Windows, the if __name__ == "__main__" guard is essential when starting worker processes.

Part 5: Hands-On Practice

Mini project — Concurrent File Analyzer. Use a thread pool to read several text files concurrently and count lines. Then create a CPU-heavy word-frequency calculation and compare thread and process pool execution times using time.perf_counter().

Part 6: Next Steps

Run and modify the examples in Visual Studio 2026, then continue to Lesson 28. Return to Python Tutorial Home to review the complete curriculum.

📘 Want the complete guide with projects? Get the book →