Traditional technology stacks like Java, JavaScript, TypeScript, and C# offer a lot of flexibility and scalability for large-scale applications, but their inherently nondeterministic nature makes it very difficult to get below a p50 latency of 500 microseconds. While there are ways to optimize garbage collection and other Java components for latency, most standard Java libraries do not focus on low-level concepts like aligning data to cache lines or reducing cache line misses. C and C++ applications optimize for a specific platform at the cost of portability, but that trade-off is exactly what gives you access to manual memory management and the ability to avoid multithreading pitfalls like false sharing. When the goal is to reduce latency, you need lower-level languages like C, C++, or Rust on critical paths. I will not cover pure hardware implementations like FPGAs or ASICs here since they are not required for our 50-microsecond latency target.

Important Definitions

Before diving into the architecture and hardware optimizations, it will be helpful to establish exactly how performance is measured in a low-latency environment.

Wire-to-wire latency

Wire-to-wire latency is measured from the moment a market data packet is received on the NIC from the exchange to the moment an order is sent back to the exchange on the same NIC. You compute it as order send timestamp minus market data receive timestamp. The most precise measurements require NIC cards with hardware timestamping enabled, so the NIC stamps every inbound and outbound packet with a hardware timestamp that you can later use to analyze the latency graph. Exanic cards provide APIs for retrieving these hardware timestamps.

Assumptions About the System

To hit the latency target of under 50 microseconds, you need to optimize both hardware and server setup alongside your software, and the following assumptions underpin everything in this article.

• Colocated host: Most stock exchanges in traditional finance provide colocation services where your servers sit physically adjacent to the exchange’s servers, removing extra network jitter from the equation.

• UDP market data access: Most stock exchanges distribute market data via UDP multicast to all subscribed parties, and this is the fastest way to consume and react to it.

• Kernel bypass NIC: The system runs on NIC cards enabled for kernel bypass and tuned for low-latency operation.

• Single venue: These systems are single-venue, meaning at any point they process market data to generate orders for one stock exchange only. For multi-venue setups, the assumption is one colocated host per venue with no shared critical-path processing.

Workload Model

Most of the trading systems follow this model of data processing in abstract terms:

Every strategy engine takes market data from the exchange as its primary input and produces orders sent to the exchange as its primary output, moving through multiple stages as described below. Exchange responses are a secondary input to the engine but are not on the hot path for latency calculations.

Market Data Ingestion

The first layer is responsible for ingesting the market data (raw or normalized) from the network card. This layer routinely processes terabytes of data per day and can exceed a petabyte on busy days at venues like CME or Eurex. On an average day at CME you can expect 20 to 30 million trades and 100 to 150 million open interest events in a single segment. Processing a full L1 feed for one segment means receiving traffic in the range of a few terabytes to hundreds of terabytes per day, so both software and hardware need to be extremely optimized.

Filtering and Checks

Filtering and checks come next, cleaning up incoming data and validating it before it reaches downstream stages. This layer handles packet drops, invalid market data, trading phase transitions like moving from an active session to a halted one, and anomalous price changes that might indicate a software bug. When something goes wrong at this stage, the right response is often to instruct strategies to stop rather than pass bad data forward.

Pricing Engine

The pricing engine receives valid and normalized market data and can range from a simple no-op or currency conversion all the way to a full ML-powered model. Lightweight pricing fits comfortably on the critical path. Complex ML models that require large amounts of memory and processing do not, because they would blow the latency budget. In practice, ML-powered engines in low-latency applications split into two processes: time-consuming training and inference model preparation happen off the critical path, and a lightweight inference step runs on the critical path using small linearized models fed configuration parameters from the heavier ML output. This keeps the heavy lifting off the hot path while still giving the strategy access to model-driven signals. Colocation also constrains available compute, so you want to reserve it for work that directly impacts critical path performance.

Strategy Evaluation

Strategy evaluation compares transformed or theoretical prices against real prices and fires an order when it identifies an opportunity. Strategies generally fall into three categories.

• Maker: Maker strategies keep quotes on both the buy and sell sides to provide liquidity and take advantage of the bid-ask spread. Exchanges often provide incentives to HFT firms that add liquidity this way.

• Taker: Taker strategies identify mispricing and fire IOC or FOK orders for immediate execution.

• Hedging: Hedging strategies reduce risk or book profit by executing the opposite side of an existing position.

Order Aggregation and Routing (OMS)

Order aggregation and routing (OMS) can be combined with strategy evaluation in simple setups, but should be separated as complexity grows. It serves three purposes: aggregating orders from multiple strategies to reduce the number of orders sent to the exchange, managing exchange responses and passing normalized results back to the strategy, and handling the routing logic that grows more complex as your strategy count increases.

Latency Map: Where the Latency Lies

Now that the high-level components are clear, it is worth mapping out where the latency actually sits so you can allocate your budget with precision. Beyond the business logic itself, your latency profile depends on CPU model and generation, NIC type and model, operating system, software architecture, and business logic complexity.

Beyond the business logic stages, several factors contribute latency that stays invisible if you only look at application code. These factors come from hardware, OS and kernel processing, network behavior, and I/O, and they play an outsized role in ultra-low-latency systems. Even if they occur infrequently, they introduce jitter and latency spikes that directly hurt your p99. Although many of these latency sources depend on a multitude of factors, you can work with approximate cost ranges that provide a useful benchmark, and the table below gives those ranges for the most common sources.

Engineering Pillars

Addressing these latency sources requires a set of engineering decisions baked into the application design itself, not patched in afterwards. The pillars below cover the most impactful areas.

Kernel Bypass

Traditionally, any application reading from the network goes through the kernel as an intermediary, which means a system call copies data from the NIC into kernel space before the application can access it by copying from kernel space into its own buffers. That double-copy plus the system call overhead adds up quickly because control needs to transfer to and from the kernel on every read, and the way to avoid it is to access the NIC directly through its driver. DPDK polling mode is one of the best approaches for this and can save roughly 20 to 45 microseconds when tuned correctly for market data processing.

Specialized hardware is also worth considering for additional gains on top of software-level kernel bypass. Exanic and Solarflare cards are widely used in the trading industry and provide zero-copy data access, hardware filtering, hardware timestamps, and flexible flow steering.

Memory Management

Most shared or larger resources get allocated on the heap through dynamic allocation using malloc or new, which requires a system call to expand the heap and introduces additional latency. Low-latency systems avoid this by pre-allocating memory segments at startup and building custom allocators on top of those segments to distribute memory during runtime, with the primary goal of eliminating all system calls from the hot path entirely. At startup you make one large allocation to cover the maximum memory any component might need, and during trading hours your custom allocator distributes from that pool without ever touching the OS. Most large-scale systems package this into a library of containers, allocators, and pools that lets you use vectors, maps, and arrays without the penalty of dynamic allocation by implementing custom versions of these types or custom allocators.

In practice it is always worth benchmarking different allocators to find the one that performs best for your specific workload, since allocators have different configuration parameters that trade off between latency, throughput, and fragmentation.

CPU and NUMA Affinity

Modern CPUs use NUMA architecture, dividing processors into nodes where each node has its own processors, caches, and memory, and communication between nodes happens over interconnects that add measurable latency when crossed on a hot path. For critical processes, you want to pin threads to cores within the same NUMA node and ensure that the data those threads access is also allocated on the same node. Non-critical processes like logging, compliance reporting, and observability can run on a separate NUMA node since they care more about throughput than latency.

An example core allocation for a machine with 8 cores per NUMA node and 2 NUMA nodes might look like this.

This layout lets you run two independent sets of strategies, one per NUMA node, while keeping logging and observability isolated from the critical path.

Time Sources and Clocks

Getting timestamps right matters a lot more in low-latency systems than in most applications because you are measuring at nanosecond granularity, and the table below covers the main options along with their trade-offs. For cross-host applications, you also need a way to synchronize time across multiple machines at nanosecond precision — PTP hardware clocks are the standard approach for this.

Network Tuning

Sockets and hardware both need tuning to get the best latency and throughput out of your setup. The key adjustments are disabling interrupts and relying on polling to avoid breaking application flow, increasing the RX buffer size to reduce packet loss, and increasing the TX buffer backlog queue size, with many other hardware-specific options available depending on your configuration.

Observability and Monitoring

Monitoring is not optional in low-latency systems because without good observability you cannot see where latency is being spent. The key metrics to track are first-order-out latency for both software and wire-to-wire paths, p50, p95, p99, and p99.9 latencies at both levels, syscall latencies using bcc and bpftrace, CPU utilization to find headroom and spot contention, and NIC-level metrics using tools appropriate to your hardware.

Failure Modes

Deployment Checklist

The hardware tuning techniques below are commonly used across trading organizations, but because you are tuning at the hardware level the right set of parameters depends on your specific hardware. These techniques also carry trade-offs like higher power consumption and increased CPU heat, so test carefully and find the optimal configuration for your application before applying them in production.

Hardware and BIOS

• Disable C-states and Turbo Boost

• Set CPU governor to performance mode

• Lock PCIe ASPM off

• Enable hugepages

OS Tuning

• Use isolcpus, nohz_full, and rcu_nocbs kernel parameters

• Apply sysctl buffer tuning

• Use mlockall to pin memory and prevent paging

Process and Code

• Use static preallocation and avoid dynamic containers on the hot path

• Prefetch and cache-align structs

• Use lock-free SPSC queues

• Use TSC for all timing on the critical path

Common Pitfalls

• False sharing between threads on the same cache line: This is one of the most common mistakes in multithreaded systems. When two threads write to variables that share a cache line, even if those variables are logically unrelated, the cache coherency protocol treats the entire line as contested and forces unnecessary cross-core coordination. Many cache miss problems in multithreaded environments trace back to this.

• Chatty IPC over sockets or message queues: Chatty IPC adds up quickly. Many distributed systems default to message queues, RPCs, or other message-passing mechanisms, but these carry much higher latency in practice even when they scale better horizontally. For low-latency IPC between processes on the same host, shared memory is the right approach.

• Premature SIMD optimization: Premature SIMD can actually increase latency when your workload is memory-bound rather than compute-bound. SIMD brings real benefits in the right situations but also adds complexity. Always benchmark before and after adding vectorization because the performance impact can go in either direction depending on your memory access pattern.

• TSC skew across cores: TSC skew is a problem that bites HFT systems specifically because they measure at nanosecond granularity. TSC is the standard clock for critical path timing because it does not require a system call, but TSC can drift across cores on the same CPU. Left uncorrected, that skew introduces measurement errors that look like real latency spikes and can obscure genuine performance bugs. Check your CPU documentation for the recommended synchronization approach and apply it regularly.

• Running without baseline histograms: Running without baseline histograms makes all of the above invisible. Every parameter including compiler settings, execution environment, hardware, and network can affect latency, and changing a single line of code can produce dramatically different results. Measuring p50 and p99 histograms at every stage of the pipeline is the only reliable way to know where your latency is coming from.

Further reading /Additional resources

1. Linux kernel vs DPDK

2. AF_XDP: Zero-Copy Packet Processing in Linux — Netdev 2024:

3. Benchmarking allocators

4. Time Synchronization in Distributed Trading Systems

5. BPF Performance Tools by Brendan Gregg

6. Intel BIOS optimizations

7. RHEL 7 low-latency tuning guide

8. IPC vs shared memory:

9. Incorrect use of SIMD:

Unlike traditional approaches to asynchronous programming, such as threads and callbacks, coroutines eliminate complexity by providing a way to write asynchronous code that looks and behaves like synchronous code. This chapter introduces you to the fundamentals of coroutines, teaching you how to avoid issues with callbacks and enabling you to manage concurrency gracefully and efficiently.

Kotlin’s coroutines framework is built on top of lightweight threads, providing developers with the tools to execute tasks in parallel, handle timeouts and cancellations, and switch between different execution contexts. Throughout this chapter, we will uncover how coroutines work, why they are an improvement over traditional concurrency models, and how to leverage them to build robust and responsive applications.

We will start by understanding what coroutines are and how they differ from threads and other concurrency mechanisms. Next, we will dive into the implementation of coroutines, learning about coroutine builders such as launch and async, as well as the importance of suspending functions in asynchronous workflows. We will then explore strategies for managing timeouts and cancellations, ensuring our applications remain efficient and responsive under various conditions. Finally, we will tackle the advanced concepts of context and scope, which are essential for organizing and managing coroutine life cycles effectively.

By the end of this chapter, you will have a comprehensive understanding of Kotlin coroutines, enabling you to write clean, maintainable, and high-performing asynchronous code. Whether you’re developing Android applications, server-side services, or any system requiring concurrent operations, mastering coroutines will empower you to tackle complex challenges with confidence and precision.

We’re going to cover the following topics:

• What is a coroutine?

• Coroutine implementation

• Timeouts and cancellations

• Contexts and scope

Technical requirements

You can use any text editor or Integrated Development Environment (IDE) of your choice with the Java language. We recommend IntelliJ IDEA Community Edition (CE). This tool already comes with everything you need to work with both Java and Kotlin. The software requirements for this chapter, with required versions, are as follows:

• Java JDK 17 or higher

• IntelliJ IDEA CE

• Kotlin 1.x or 2.x

The code used in the chapters can be found in the book’s accompanying GitHub repository: https://github.com/PacktPublishing/Kotlin-for-Java-Developers.

What is a coroutine?

To simplify what a coroutine is, let’s first look at the official definition: “A coroutine is an instance of a suspendable computation.” While technically accurate, it may seem too abstract. Instead, let’s break it down with a practical and relatable analogy.

Imagine you are dining at a restaurant. A waiter comes to your table, takes your food order, and heads to the kitchen to pass the order on to the chefs. Now, imagine the waiter stays in the kitchen, doing nothing but waiting for your food to be prepared. This would be inefficient, as the waiter could be serving drinks, attending to other tables, or bringing bread to your table while the food is being cooked.

A smarter approach is for the waiter to leave the instructions with the kitchen staff and return to attend to other customers or tasks. Once your food is ready, the kitchen would alert the waiter, who would promptly bring the dishes to your table.

This efficiency is exactly how coroutines operate. In programming, there’s a “main thread” (like the waiter), which handles primary tasks (like serving the diners in our analogy). However, certain tasks (such as cooking the food) can take an unknown amount of time. These tasks are better handled asynchronously, without blocking the main thread.

The restaurant analogy

To make the concept of coroutines easier to grasp, let’s stick with the restaurant analogy. This comparison helps illustrate how asynchronous tasks and concurrency work in Kotlin using coroutines:

Let’s try to understand this analogy:

1. Main thread as the waiter: The main thread is responsible for visible and interactive tasks (like serving diners). It shouldn’t be blocked (or stuck in the kitchen).

2. Asynchronous tasks as the kitchen: Some operations, like fetching data from the internet or reading from a database, are similar to cooking food in the kitchen. These tasks can take time and don’t need the waiter (the main thread) to stand idle while waiting for them to finish.

3. Coroutines as the waiter’s delegation: The waiter can delegate tasks such as food preparation to the kitchen while continuing to handle other responsibilities (serving drinks, taking new orders). Similarly, coroutines allow you to “pause” tasks and resume them later without stopping the main thread.

4. Concurrent coroutines for multiple tasks: Just as a waiter can manage multiple orders at once, you can create multiple coroutines to handle different tasks simultaneously. For instance, one chef can prepare appetizers, another can handle main courses, and yet another can prepare desserts – all while the waiter continues serving diners.

In programming terms

Kotlin simplifies asynchronous programming, but it doesn’t include heavy low-level APIs in its standard library. Instead, it provides just enough tools for libraries, such as kotlinx.coroutines, to build powerful coroutine-based utilities. Unlike some other languages, Kotlin doesn’t treat async and await as keywords or include them directly in the language. Instead, it uses suspending functions, which make asynchronous code safer and easier to use compared to traditional approaches such as futures or promises.

To better understand how coroutines work, let’s map their behavior to real-world and programming concepts. This helps clarify how Kotlin approaches asynchronous programming in a more intuitive way:

• Main thread: The core execution thread that interacts with users (like serving diners).

• Coroutine: A lightweight task that runs asynchronously and can be paused and resumed without blocking the main thread.

• Suspension: Like the waiter leaving instructions in the kitchen, a coroutine can “pause” its execution, allowing the main thread to continue working until the task is complete.

The kotlinx.coroutines library, developed by JetBrains, is a feature-rich library that provides high-level tools for working with coroutines. It includes useful functions such as launch and async, which simplify working with background tasks.

Explanation of coroutines for a Java programmer

It’s important to know that Java does not have coroutines. However, there are many ways we can achieve the results that coroutines achieve, and it is very likely that those of you who know Java might know some of these ways.

While there are projects such as Project Loom and Quasar that attempt to bring coroutine-like behavior to Java, they require third-party dependencies and are not yet widely adopted in the Java ecosystem.

Java thread handling was enhanced with virtual threads in version 21. They are a more lightweight threading model. Yet, virtual threads and coroutines serve different purposes. Virtual threads are primarily used for high-throughput, I/O-bound server applications based on a thread-per-request model. Coroutines, on the other hand, are typically used for various asynchronous tasks.

Note that it is perfectly possible for a coroutine to use a virtual thread to execute its tasks, especially if these tasks include blocking I/O operations.

In this section, we explored the fundamental concept of coroutines, learning how they provide a powerful and efficient alternative to traditional asynchronous programming models such as threads, futures, and promises. Using the analogy of a waiter in a restaurant, we visualized how coroutines delegate long-running tasks to the background while keeping the main thread free to handle interactive or essential tasks. Key characteristics of coroutines, such as their non-blocking nature, ease of readability, and lightweight execution, were highlighted as significant advantages over Java’s concurrency options. For Java developers, the comparison to traditional tools such as threads and futures underlined the simplicity and efficiency that coroutines bring to the table.

In the next section, we will dive deeper into the practical aspects of working with coroutines. You’ll learn how to create and manage coroutines using coroutine builders and suspending functions, setting the foundation for writing clean and effective asynchronous code in Kotlin. Let’s begin our hands-on journey into coroutine implementation!

Coroutine implementation

Now let’s see how to implement coroutines in a Kotlin project. For this, we’ll start from scratch. First, we’ll do something very simple; we can consider it the “Hello World“ of coroutines. Then we’ll increase the difficulty of the example and show other features.

We will start by adding the possibility of using coroutines in our project, and for that, we will add the dependency.

One thing to note is that using kotlinx.coroutines has become a convention in the Kotlin community, making it easier to collaborate and understand code.

We don’t recommend trying to memorize the versions, since over time, these change, and they change faster than you can imagine. It is more important to know what the official kotlinx.coroutines repository is. GitHub is a reliable source to obtain information about the latest versions and news: https://github.com/Kotlin/kotlinx.coroutines.

In general terms, we can say that we need to add the coroutines dependency. We can find it in our build.gradle.kts file, and the configuration would be something like this:

dependencies {
testImplementation(kotlin("test"))
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.0")
}

For coroutines, we add the one reference to kotlinx 1.8.0, which is enough for what we will do in this chapter. Now, in our first contact with coroutines, we are going to write Hello World. This will be our starting point for this entire chapter. This is the code:

import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
fun main() = runBlocking {
    launch {
        delay(1000L)
        println("Kotlin World! ")
    }
    println("Hello")
}

The imports come from the kotlinx.coroutines package, which contains the main tools for working with coroutines in Kotlin:

• delay: A function that suspends the execution of a coroutine for a specific time (in milliseconds) without blocking the thread.

• launch: A function that starts a new coroutine in parallel with the existing execution.

• runBlocking: A function that runs a blocking coroutine, useful for simple examples or tests where we want to block the main thread until the inner coroutines finish.

The imports should be familiar. The first part that is noticeably different from basic “Hello World“ code is the following:

fun main() = runBlocking {

The runBlocking instruction is used here to allow main to run as a coroutine. It blocks the main thread until all coroutines within its block finish executing.

This ensures that the program does not terminate before the coroutines finish.

Let’s look at the launch instruction:

launch {
    delay(1000L)
    println("Kotlin World!")
}

launch creates a new coroutine. This coroutine runs concurrently with the rest of the code inside runBlocking.

Inside this coroutine, we have delay(1000L), which suspends the coroutine for 1 second (1,000 milliseconds) without blocking the thread. This means that the main thread is free to perform other tasks in the meantime, and after one second, it prints "Kotlin World!".

Finally, we have the last instruction:

println("Hello")

Since the coroutine does not block the thread, this message is printed before the coroutine finishes its task.

Summarizing the flow, we can describe it as follows:

1. The main function is executed and enters the runBlocking block.

2. Inside runBlocking, the following occurs:

   1. A coroutine is launched with launch.

   2. Execution continues without waiting for the coroutine to finish, so println("Hello") is executed immediately.

3. In the meantime, the coroutine executes delay(1000L), which suspends the coroutine for one second.

4. After the delay, it prints "Kotlin World!".

5. The program waits inside runBlocking until all coroutines inside its block finish. Only then does runBlocking complete and the program terminates.

After executing the code, we see the following as a result:

Hello
Kotlin World!

The reason we started with Hello World is simple – we are studying the simplest possible case to show how coroutines allow concurrent tasks (such as waiting a second) to be expressed without blocking the entire scope of execution. In this example, runBlocking blocks the main thread until all coroutines inside complete, but the coroutine launched with launch runs concurrently within that scope.

Compared to a traditional threading approach, this is more efficient and easier to read.

We introduced the concepts of delay and launch in a simple context. Now we will look at another example. We will simulate a restaurant scenario to demonstrate how coroutines handle asynchronous tasks.

We want to demonstrate how to perform multiple tasks simultaneously

We are going to show how not to block the main thread while waiting for long-running tasks. We will illustrate coordination between different asynchronous operations, something similar to what we mentioned about the waiter and the restaurant. This is the new code:

import kotlinx.coroutines.*
fun main() = runBlocking {
    println("Taking order...")
    // Coroutine 1: Preparing food in the kitchen
    val food = async { prepareFood() }
    // Coroutine 2: Serving drinks
    serveDrinks()
    // Wait for food to be ready and serve it
    println("Food is ready: ${food.await()}")
}
suspend fun prepareFood(): String {
    delay(3000) // Simulate food preparation
    return "Delicious meal"
}
fun serveDrinks() {
    println("Serving drinks to the table...")
}

The runBlocking block represents the main thread where primary actions (such as interacting with diners) occur. As mentioned before, the runBlocking instruction is a function that runs a block of code as a blocking coroutine. This means that the program will not end until all the code inside this block finishes. In this example, it represents the main thread, which acts as the waiter handling multiple tasks.

println("Taking order...")

The preceding line of code is printed immediately on the main thread (representing the waiter).

val food = async { prepareFood() }

async launches a new coroutine to execute the prepareFood() function concurrently.

async is a function that returns a Deferred object, which is like a “future” in Java. This object can be awaited or revisited later to get the results of the operation.

In this case, it represents the cook who starts preparing the food in the background.

Let us look at the prepareFood function:

suspend fun prepareFood(): String {
    delay(3000) // Simulate food preparation
    return "Delicious meal"
}

The prepareFood function is a suspended function, meaning that it can pause its execution and later resume exactly where it left off, without blocking the underlying thread while it is waiting. However, a suspend function can only be called from another suspend function or within a coroutine. It is also important to note that while the thread is not blocked, the coroutine that invokes prepareFood will remain suspended until the function resumes.

delay(3000) simulates a meal preparation time of three seconds. During this time, the coroutine is suspended, allowing the main thread to do other tasks.

After the delay, it returns Delicious meal.

While the food is being prepared (the async coroutine is suspended), the main thread continues to execute this function:

serveDrinks()

It prints the message Serving drinks to the table....

Here, the waiter (main thread) serves drinks while waiting for the cook to finish the food:

println("Food is ready: ${food.await()}")

food.await() pauses execution at this point until the prepareFood() coroutine finishes and returns its result.

Once prepareFood() returns Delicious meal, the main thread prints the following:

Food is ready: Delicious meal

This is how the flow goes:

• runBlocking starts the main block.

• The waiter takes the order and starts serving the drinks.

• Meanwhile, the cook (coroutine) prepares the food in the background.

• After three seconds, the food is ready and the waiter serves it.

Returning to our analogy, we’ve already seen the following:

• Waiter: This is the main thread that organizes tasks. It can serve drinks and handle other orders without getting stuck waiting for the food to be ready.

• Cook: This is a coroutine that works independently in the background, preparing the food.

• async and await: async starts the cook’s work and await makes sure the waiter waits for the result before serving the food.

• delay: This simulates the time it takes the cook to prepare the food.

This is what we see after running the program:

Taking order...
Serving drinks to the table...
Food is ready: Delicious meal

The main goal of this exercise was to demonstrate how coroutines allow you to do several things “at the same time” efficiently, just like a good waiter in a restaurant.

We are going to show another example that simulates loading book data from an API using coroutines in Kotlin. The repository (BookRepository) has a suspended function that simulates a two-second delay before returning the book data. In the BookScreen class, withContext(Dispatchers.IO) is used to execute the task in a context suitable for I/O operations, displaying success or error messages depending on the result. This demonstrates how to handle asynchronous tasks in an efficient and readable way. But we will build it in parts.

import kotlinx.coroutines.*
// Simulate a repository that fetches book data
class BookRepository {
    // Simulate an API call to fetch a book
    suspend fun fetchBookFromApi(bookId: String): Book {
        delay(2000) // Simulate network response time
        return Book(bookId, "Clean Code", "Robert C. Martin")
    }
}

BookRepository has a suspend function, fetchBookFromApi, which simulates calling an API to get the data for a book.

It uses delay(2000) to represent a two-second network delay, and then returns a Book object as the result.

We add this code, which defines the class that the fetchBookFromApi function returns:

data class Book(
    val id: String,
    val title: String,
    val author: String
)

Book is a data class that represents information about a book, with properties such as id, title, and author.

class BookScreen {
    private val repository = BookRepository()
    fun loadBookData() = runBlocking {
    try {
        println(" Loading book information...")
        // Send the request in a coroutine
        val book = withContext(Dispatchers.IO) {
            repository.fetchBookFromApi("9780132350884")
}

BookScreen defines the loadBookData method, which uses runBlocking to handle loading book data within a coroutine.

The loadBookData method uses withContext(Dispatchers.IO) to perform the book-fetching task in a context optimized for I/O operations.

Finally, we define main, which will allow us to execute the code:

fun main() {
    val screen = BookScreen()
    screen.loadBookData()
}

In the main function, you simply create an instance of BookScreen and call the loadBookData method to start the process.

Summarizing the program flow, this would be as follows:

• It prints “Loading book information...“

• It waits two seconds, simulating the API response.

• If everything goes well, it prints "Book loaded: Clean Code by Robert C. Martin".

• If an error occurs, it prints "Error loading book information: ..."

Let’s take a closer look at what’s happening in this exercise. We’ve successfully used runBlocking and withContext to run the task asynchronously and on a separate thread (Dispatchers.IO), which is ideal for I/O tasks such as API calls.

This approach is ideal for I/O tasks such as API calls because it allows us to perform these operations without blocking the main thread.

By offloading the work to a different dispatcher, we ensure that time-consuming operations (such as reading from a database, writing to a file, or calling an external API) don’t freeze the UI or other critical processes. In Kotlin coroutines, Dispatchers.IO is specifically optimized for these types of tasks, giving us efficient and responsive code even when dealing with slow or unpredictable external systems.

We have included a try-catch block to catch possible exceptions that may occur during the API call, which is a good practice to make your code more robust.

Simulating API response time with a delay is useful to understand the behavior of coroutines in real-world scenarios.

In this section, we introduced the foundational ideas behind coroutines in Kotlin, focusing on how they enable writing asynchronous, non-blocking code that remains readable and efficient. We illustrated real-world analogies and scenarios to highlight how coroutines support structured concurrency and safe context switching, making them a powerful tool for managing concurrent tasks in modern applications.

It’s important to clarify when to use these different coroutine builders:

• runBlocking is typically used to bridge blocking and non-blocking code, for example, in tests or the main function of a small program. It blocks the current thread until its coroutine body finishes executing.

• In contrast, using coroutine builders such as launch or async within an existing coroutine scope, combined with context switching (withContext), lets you run asynchronous code without blocking any thread, ensuring better performance and responsiveness.

It is important to note, however, that coroutines are not magical. If you call blocking operations inside a suspend function – for example, using Thread.sleep() or synchronous I/O calls such as JDBC – you will still block the underlying thread. To fully benefit from coroutines, blocking calls should be replaced with non-blocking equivalents whenever possible.

So, while runBlocking is useful in specific entry points, using structured concurrency with launch, async, and withContext is preferred in production code to handle tasks concurrently and safely without halting your program’s execution.

Now that we have a solid understanding of how to implement and use coroutines effectively, the next section will introduce timeouts and cancellations. You’ll learn how to gracefully handle scenarios where tasks take too long or need to be interrupted, ensuring your applications remain responsive and efficient under various conditions. Let’s continue exploring how Kotlin coroutines can simplify even the most complex asynchronous workflows!

Timeouts and cancellations

Now we will see two important elements that are good to know about in coroutines: timeouts and cancellations. Timeouts are time limits that you set for an operation. If the operation takes longer than the specified time, it is automatically cancelled.

A simple way to describe them using examples of daily activities would be as follows:

• Ordering a pizza and getting the notice: “If it doesn’t arrive in 30 minutes, we cancel the order and refund you”.

• When an ATM cancels the operation if we don’t respond within a certain time.

Cancellations, as their name suggests, allow us to stop a coroutine in the middle of its execution. Let’s look at a couple of examples:

• Canceling a download in progress.

• Canceling a bank transfer before it is completed.

Thinking of timeouts and cancellations as elements that we can use, we could easily give some situations where we could use them:

• API calls that should not take too long.

• File downloads that can be cancelled.

• Database operations with a time limit.

• Background processes that can be cancelled by the user.

Taking our previous example of books as a basis, we are now going to modify it to have both cancellation and timeouts.

The definitions of BookRepository and Book remain the same; we just add more time to Delay. Before, we had 2000, but we will change the value to 3000. We increased the time period to later; we set a time limit of two seconds and allowed three seconds to execute it (the timeout):

class BookRepository {
    // Simulates an API call to get a book
    suspend fun fetchBookFromApi(bookId: String): Book {
        delay(3000) // Simulates network response time
        return Book(bookId, "Clean Code", "Robert C. Martin")
    }
}
data class Book(
    val id: String,
    val title: String,
    val author: String
)

Now we will change launch to make it look like this:

val job = launch {
    try {
        println(" Loading book information...")
        // Set a timeout for the operation
        val book = withTimeout(2000) { // 2 second timeout
            repository.fetchBookFromApi("9780132350884")
        }
        println(" Book loaded: ${book.title} by ${book.author}")
    } catch (e: TimeoutCancellationException) {
        println(" Timeout exceeded when loading book.")
    } catch (e: Exception) {
        println(" Error loading book information: ${e.message}")
    }
}

We set a time limit of 2,000 milliseconds to complete the fetchBookFromApi operation. If the task does not finish within this time, TimeoutCancellationException is thrown. This mechanism is essential in applications where it is critical to avoid long blockages and ensure fast responses.

In this statement, we can see that the approach uses job, as declared with launch, to handle the execution of the coroutine:

val job = launch {

This allows the job to be explicitly cancelled if necessary, providing flexibility to stop execution in specific situations. For example, the job could be cancelled with a call to job.cancelAndJoin() if the task is detected to be no longer relevant or if an external event occurs.

Exception handling is a key part of this approach:

} catch (e: TimeoutCancellationException) {
println(" Timeout exceeded when loading book.")

These last lines demonstrate how to handle scenarios where the time limit is exceeded, ensuring that the system can respond appropriately without crashing. Other exceptions are also handled to catch generic errors during execution.

While this approach is powerful, it adds complexity to the code. It requires explicitly handling both the job and exceptions, which may be unnecessary for simple tasks. However, in cases where strict time limits or precise cancellations are needed, this structure proves invaluable.

Outside of loadBookData, we will also modify the code in the following way:

// Change: Cancel the job after 4 seconds
delay(4000)
println(" Canceling the loading of the book...")
job.cancelAndJoin() // Cancel the job and wait for it to finish
println("Task cancelled.")

Notice that we have introduced a delay of 4,000 milliseconds and manual cancellation. Also notice that we use a job to cancel the task explicitly with job.cancelAndJoin().

The cancellation and timeout scenario introduces a more controlled approach to handling asynchronous tasks in Kotlin. Using launch and withTimeout, a time limit is set for the execution of an operation.

This ensures that the task does not exceed the time we decide, throwing an exception (TimeoutCancellationException) if the time runs out. Also, by encapsulating the logic in a job, the task can be cancelled explicitly when necessary, optimizing resources and allowing better management of multiple concurrent tasks.

This example we developed includes robust exception handling to differentiate between a timeout and other errors. In summary, this model is ideal for applications where time limits and cancellation of irrelevant or delayed tasks are required, although it adds a bit more complexity to the code.

This section emphasized the importance of timeouts and cancellations in effective coroutine management. By enforcing time limits and enabling controlled interruption of tasks, Kotlin coroutines provide mechanisms to keep applications responsive and resilient, especially when dealing with potentially long-running operations.

We also introduced the concept of cancellations, allowing tasks to be stopped explicitly when they are no longer needed. By leveraging the job object, we illustrated how to manage and cancel ongoing tasks using job.cancelAndJoin(), ensuring efficient resource management. Additionally, we incorporated robust exception handling to distinguish between timeout-specific issues (TimeoutCancellationException) and other errors, adding resilience to our code.

By combining these techniques, we showed how to handle asynchronous tasks in a controlled and efficient manner, enabling developers to create responsive applications that gracefully handle delays and interruptions. This approach is particularly useful in scenarios involving API calls, file downloads, or user-cancelled operations, providing flexibility and reliability.

Next, we’ll explore contexts and scopes, diving deeper into how coroutines manage their execution environments and life cycle. You’ll learn about coroutine dispatchers, structured concurrency, and how to properly manage coroutine life cycles to maintain clean, predictable, and efficient code. Let’s continue building our understanding of Kotlin coroutines with these advanced concepts!

Contexts and scope

Context and scope are fundamental concepts in the world of Kotlin coroutines and play a crucial role in managing their life cycle and behavior.

A context in Kotlin coroutines is like a container that groups key information about how and where a coroutine will run. This context includes the following:

• Dispatcher: Defines which thread or thread group the coroutine will run on, as follows:

  • Dispatchers.IO for I/O operations (such as working with files or network calls).

  • Dispatchers.Main for updating the UI.

• Job: Represents the specific task the coroutine performs. It also allows you to cancel it or monitor its progress.

• Other attributes: Can include things such as the following:

  • CoroutineName: A useful identifier to distinguish coroutines.

  • NonCancellable: Indicates that the coroutine should not be cancelled, even if its parent or associated scope is.

In addition, you can put custom elements into the coroutine context, such as an Mapped Diagnostic Context (MDC) for logging. It is also important to remember that thread-local variables are not effective with coroutines, since a coroutine may resume on a different thread. If you need thread-local-like behavior, you should use ThreadContextElement to propagate context across suspensions.

In short, a coroutine’s context sets up its execution environment: it defines where it runs, how it interacts with other processes, and what rules it should follow while working. It’s like giving the coroutine a roadmap and tools to get it to perform its task correctly.

The code for BookRepository and Book remains the same; we don’t have to make any changes.

The changes start with the BookScreen code. We add the scope as follows:

class BookScreen {
    private val repository = BookRepository()
    private val scope = CoroutineScope(Dispatchers.Main + Job())

CoroutineScope is used, which automatically encapsulates the context and the associated job. With this change, instead of the previously temporary scope used with runBlocking, now there is a permanent scope with CoroutineScope(Dispatchers.Main + Job()).

This makes it easier to manage related tasks, allowing all coroutines associated with the scope to be cancelled with a single call (scope.cancel()).

We need better organization for larger applications or those with multiple coroutines.

Let’s make some changes in the structure:

fun loadBookDetails(bookId: String) {
    scope.launch {
    withContext(Dispatchers.IO) {
        // code here
        }
    }
}

We notice with these changes that a specific scope is defined at the class level and dispatchers are explicitly specified. In addition, we are now handling specific execution contexts, and we are going to allow global cancellation of all coroutines.

Regarding the execution context, we can also say that with the previous code, dispatchers were not specified, and now we are indicating one, dispatchers.IO, for input and output operations.

We also need to change val book. It should look like this:

val book = withTimeout(3000) {
    // Set a timeout of 3 seconds
    withContext(Dispatchers.IO) {
        // Run the task in the I/O Dispatcher
        repository.fetchBookFromApi(bookId)
    }
}

We add this function(cancelAllTasks) after loadBookDetails:

fun cancelAllTasks() {
    scope.cancel() // Cancels all coroutines in the Scope
    println("All tasks were cancelled.")
}

We centralize canceling all active coroutines of the scope with scope.cancel() in the cancelAllTasks() method.

The main code part now looks like this:

fun main() {
    val screen = BookScreen()
    // Start loading book details
    screen.loadBookDetails("1")
    // Simulate task cancellation after 5 seconds
    runBlocking {
        delay(5000)
        screen.cancelAllTasks()
    }
}

With the loadBookDetails instruction, we will start loading the book details to display them. With the second part, runBlocking, we would be simulating the cancellation of tasks after five seconds.

So, let’s get to the differences:

• Life cycle management:

  • runBlocking blocks the main thread until all the internal coroutines complete. It’s useful for testing or small examples, but it’s not ideal for real applications because it freezes the main thread while it waits. The problem is that the thread is blocked, which affects performance if there are more concurrent tasks.

  • Using CoroutineScope acts as a container that allows you to manage all the coroutines created within it. This is ideal for real-world applications because it allows you to manage the life cycle of multiple coroutines in a centralized manner.

  • We can see that in the code, we can now cancel all related coroutines at once using scope.cancel(). This is useful in situations where, for example, a user leaves a screen in a mobile app.

• Dispatchers:

  • Before the changes, no dispatcher was specified. All code runs in the default context of runBlocking. This meant that tasks such as fetchBookFromApi run on the same thread as runBlocking. This can be problematic if you are doing intensive tasks, such as API calls or database access, because they might block the main thread.

  • By implementing Dispatchers.IO, we take advantage of the fact that it is designed for I/O tasks, such as network calls. This ensures that these tasks are executed in a thread optimized for operations of this type, without blocking the main thread.

  • So, we clearly separate the network tasks (in Dispatchers.IO) from the UI-related tasks (in Dispatchers.Main).

• Cancellation of coroutines:

  • Previously, we cancelled manually with job.cancelAndJoin(). This stops the specific coroutine and waits for all resources to be released. This is effective but requires you to control each job individually. Switching to scope.cancel() stops all coroutines associated with CoroutineScope. This is cleaner and more scalable, especially if you have multiple related tasks.

• Separation of contexts:

  • Initially, there was no explicit separation of contexts. All operations, such as API calls and result handling, are executed in the same thread. With the changes, the contexts were separated clearly, as we can see:

    withContext(Dispatchers.IO) { repository.fetchBookFromApi(bookId)
    }

Networking code (API call) runs in Dispatchers.IO, while the rest of the code (e.g., printing to the console) can run in Dispatchers.Main. This improves performance and keeps the UI smooth.

In summary, the final code represents a more mature and professional implementation of coroutine handling in Kotlin, where each operation is executed in the most appropriate context: the UI in the main dispatcher and network operations in the I/O dispatcher. This approach not only optimizes the use of system resources but also provides a clearer and more maintainable structure, where responsibilities are well defined and separated. Operation cancellation is handled more robustly through a shared scope, allowing for efficient coroutine life cycle management.

Furthermore, this implementation pattern better reflects real-world practices in app development, especially on Android, where it is crucial to properly handle execution contexts and component life cycles. By using CoroutineScope defined at the class level along with specific dispatchers, you achieve more predictable and maintainable code, which can scale better as your app grows and becomes more complex.

Summary

In this chapter, we delved into Kotlin coroutines, one of the most transformative features of the language for handling asynchronous programming. Here is a summary of what we’ve learned.

We started by demystifying coroutines through relatable analogies, such as a waiter in a restaurant. This helped visualize how coroutines enable concurrent, non-blocking tasks while keeping the main thread free for interactive operations.

We introduced coroutine builders such as launch and runBlocking, along with suspending functions such as delay. Through practical examples such as “Hello World“ and the restaurant scenario, we showcased how to create and manage coroutines effectively.

We explored the critical concepts of setting time limits (withTimeout) and manually canceling tasks (job.cancelAndJoin). These mechanisms ensure efficient resource usage and help handle long-running tasks gracefully, as seen in the book-fetching example.

Finally, we covered how CoroutineScope and dispatchers are used to manage execution contexts and life cycles. We demonstrated how separating contexts (e.g., Dispatchers.IO for network calls and Dispatchers.Main for UI updates) improves application performance and maintainability. We also highlighted the importance of structured concurrency, enabling efficient cancellation and life cycle management for related tasks.

This chapter provided a robust foundation for understanding and applying Kotlin coroutines, a game-changing feature that simplifies asynchronous programming. By transitioning from basic examples to real-world scenarios, you now understand how coroutines enhance concurrency, improve code readability, and make resource management more efficient. These concepts are critical for developing scalable and responsive applications, whether for Android, server-side, or other Kotlin-based projects.

If you want to dig deeper into the mechanics of moving from Java to Kotlin—writing idiomatic Kotlin, handling null safety, using coroutines for concurrency, and taking advantage of features like extension functions and DSLs—check out Kotlin for Java Developers by José Dimas Luján Castillo and Ron Veen (Packt, Oct 2025). Written for experienced Java developers, it teaches Kotlin by mapping concepts directly to familiar Java constructs and then goes further into interoperability, generics, data and sealed classes, coroutines and flows, and DSL design—across backend, Android, and cross-platform development.

With thanks to AI Engineer Gerald Parker for his technical review.

Eino ADK, pronounced “I know”, is a multi-agent development framework designed for Go, developed and hardened in real-world use at ByteDance. Its design philosophy is “keep simple things simple, and make complex things possible”. Open-sourced at the start of 2025, Eino’s promise to Go developers is they can focus on implementing business logic without worrying about underlying technical complexity.

In this article co-written with the team behind Eino, we will discuss:

• What Eino ADK is

• Core agent patterns in Eino along with real use cases

• Example code for building a simple project manager agent

Introduction to Eino ADK

Agents are quickly becoming the mainstream way to deploy LLMs, from intelligent customer service to automated office work. With them, the following pain points are emerging:

• LLMs are not bridged well with business systems, resulting in agents that can only engage in “empty talk.”

• Lack of state management causes agents to frequently “forget” when performing tasks.

• Complex interactive processes increase development difficulty even further.

Eino ADK was created to provide Go developers with a complete, flexible, and powerful agent development framework that addresses these core challenges head-on.

Recap: What is an Agent?

You can think of an agent as an independent, intelligent entity that can understand instructions, perform tasks, and provide responses – capable of autonomous learning, adaptation, and decision-making. Its main functions include:

• Reasoning: An agent can analyze data, identify patterns, and use logic and available information to draw conclusions, make inferences, and solve problems.

• Action: An agent takes actions or executes tasks to achieve goals based on decisions, plans, or external inputs.

• Observation: An agent autonomously collects relevant information (for example, through computer vision, natural language processing, or sensor data analysis) to understand the context and lay the foundation for informed decision-making.

• Planning: An agent can determine necessary steps, evaluate potential actions, and select the best course of action based on available information and expected outcomes.

• Collaboration: An agent can effectively work together with others (human or other agents) in complex and dynamic environments.

Any scenario that requires interaction with a LLM can be abstracted as an agent. For example:

• An agent for querying weather information

• An agent for booking meetings

• An agent capable of answering questions in a specific domain

What is Eino ADK?

Eino ADK is a complete system for developing intelligent agents. It breaks down complex AI applications into independent, composable intelligent agent units, allowing developers to build agentic systems the way they would assemble Lego blocks.

Its features include:

• Less glue code: Unified interfaces and event streams make complex task decomposition more natural.

• Rapid orchestration: Pre-built paradigms + workflows allow you to build pipelines in minutes.

• More controllable: Interruptible, resumable, and auditable, making the agent collaboration process “visible.”

Eino achieves this through unified abstract interfaces, flexible composition patterns, and powerful collaboration mechanisms.

The Core: Unified Agent Abstraction

The core of ADK is a concise and powerful agent interface:

type Agent interface {
    Name(ctx context.Context) string
    Description(ctx context.Context) string
    Run(ctx context.Context, input *AgentInput, options ...AgentRunOption) *AsyncIterator[*AgentEvent]
}

Each agent has:

• A clear identity (Name)

• A clear responsibility (Description)

• A standardized execution method (Run)

This provides a basis for discovery and invocation between agents. Simple Q&A robots or complex multi-step task processing systems can be implemented through this unified interface.

Core Agent Patterns

The following sections lay out the core patterns that make up Eino ADK. Each will be introduced, explained, and then brief skeleton code demonstrated.

ChatModelAgent: The Brain

ChatModelAgent is the most important pre-built component in Eino ADK. It encapsulates the interaction logic with LLMs and implements the classic ReAct (Reason-Act-Observe) pattern. The process is as follows:

1. Reason: Call the LLM.

2. Act: The LLM returns a tool call request.

3. Act: ChatModelAgent executes the tool.

4. Observe: The tool’s result is returned to the LLM, which continues to generate based on the previous context until the model determines that no more tool calls are needed.

The core of the ReAct pattern is a closed loop of Reason → Act → Observe → Reason Again, which addresses the pain points of traditional agents that either act blindly or have a disconnect between reasoning and action.

Here are some possible practical scenarios:

• Generating a market analysis report:

  • Reason-1: To assess the potential of a sector, information on policy support, industry growth rate, profitability of leading companies, and supply chain bottlenecks is needed.

  • Action-1: Call an API to get overall financial data for the industry.

  • Observe-1: Analyze the data.

  • Reason-2: Determine that the industry has high growth and policy endorsement, but rising upstream prices may squeeze downstream profits. Decide that further verification is needed.

  • Act-2: Call an API to get detailed data on supply and demand, industry research reports, and so on.

  • Observe-2: Analyze the data.

  • Reason-3: Integrate the conclusions to generate an analysis report with key data sources.

Using the ReAct pattern avoids information overload caused by collecting all the information at once. It focuses on core issues through gradual reasoning and uses data to verify thoughts rather than relying on intuition. The process is also explainable, improving the accuracy of the generated report.

• IT fault operations and maintenance:

  • Reason-1: Identify common causes of failures, such as “CPU overload, insufficient memory, full disk, or service crash.” Basic monitoring data needs to be checked first.

  • Act-1: Call the “Monitoring System API” to query server metrics.

  • Observe-1: Analyze the server metrics.

  • Reason-2: Determine the main cause. For example, if CPU utilization is abnormal, investigate which processes have high CPU usage.

  • Act-2: Use a “Process Management Tool” to check the top processes for any abnormal services.

  • Observe-2: Discover that the logging service is abnormal, possibly due to “log file too large” or “configuration error.” Verify these conditions before planning the next step.

  • Reason-3: Check the logging service’s configuration and log file size.

  • Act-3: Execute a bash command.

  • Observe-4: Find that the log file is too large, and the configuration does not enable rotation or set a maximum log size.

  • Reason-4: Provide a feasible solution to the O&M engineer: clean up the logs, modify the configuration to enable rotation, and restart the logging service and application.

The ReAct pattern gradually narrows down the problem scope, avoiding blind operations. Each step is well-founded, making it easy for O&M engineers to perform secondary verification before implementing solutions, and providing a basis for subsequent reviews and preventative measures.

ChatModelAgent leverages the capabilities of LLMs for reasoning, understanding natural language, making decisions, generating responses, and interacting with tools, acting as the “thinking” part of an intelligent agent application.

Here is skeleton code for a ChatModelAgent with ReAct capabilities:

import "github.com/cloudwego/eino/adk"

// Create a ReAct ChatModelAgent with multiple tools
chatAgent := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Name: "intelligent_assistant",
    Description: "An intelligent assistant capable of using multiple tools to solve complex problems",
    Instruction: "You are a professional assistant who can use the tools provided to help users solve problems",
    Model: openaiModel,
    ToolsConfig: adk.ToolsConfig{
        Tools: []tool.BaseTool{
            searchTool,
            calculatorTool,
            weatherTool,
        },
    },
})

WorkflowAgents: The Sophisticated Pipeline

Eino ADK provides the WorkflowAgents pattern, designed to coordinate the execution flow of sub-agents. It manages the running of agents through predefined logic to produce a deterministic execution process, helping to achieve a predictable and controllable output.

You can arrange and combine the following patterns as needed:

• SequentialAgent

• ParallelAgent

• LoopAgent

And combine them with ChatModelAgent to construct a complete workflow pipeline. Here are descriptions of each agent in detail:

• SequentialAgent: Executes the registered agents in the configuration in sequence once, and then ends. The operation follows these principles:

  • Linear execution: Strictly executes in the order of the SubAgents array.

  • Passing of run results: Each agent in the configuration can get the complete input of the SequentialAgent and the output of the preceding agent.

  • Support for early exit: If any sub-agent produces an exit/interrupt action, the entire Sequential process will be terminated immediately.

• Possible practical scenarios include:

  • Data ETL: ExtractAgent (extracts order data from MySQL) → TransformAgent (cleans null values, formats dates) → LoadAgent (loads into a data warehouse)

  • CI/CD pipeline: CodeCloneAgent (pulls code from a code repository) → UnitTestAgent (runs unit tests, returns an error and analysis report when a test case fails) → CompileAgent (compiles the code) → DeployAgent (deploys to the target environment)

Skeleton code for a SequentialAgent:

import "github.com/cloudwego/eino/adk"

// Execute in sequence: create a research plan -> search for materials -> write a report
sequential := adk.NewSequentialAgent(ctx, &adk.SequentialAgentConfig{
    Name: "research_pipeline",
    SubAgents: []adk.Agent{
        planAgent,   // Create a research plan
        searchAgent, // Search for materials
        writeAgent,  // Write a report
    },
})

• ParallelAgent: Executes the registered agents in the configuration concurrently, and ends after all agents have finished executing. The operation follows these principles:

  • Concurrent execution: All sub-agents start at the same time and execute in parallel in separate goroutines.

  • Shared input: All sub-agents receive the same initial input as when the ParallelAgent was called.

  • Waiting and result aggregation: Internally uses sync.WaitGroup to wait for all sub-agents to complete, collects the execution results of all sub-agents, and outputs them to the AsyncIterator in the order they are received.

• Possible practical scenarios include:

  • Multi-source data collection: MySQLCollector (collects user table) + PostgreSQLCollector (collects order table) + MongoDBCollector (collects product reviews)

  • Multi-channel push: SocialPushAgent (pushes to social media account) + SMSPushAgent (sends SMS) + AppPushAgent (pushes to app)

Skeleton code for a ParallelAgent:

import "github.com/cloudwego/eino/adk"

// Concurrently execute sentiment analysis + keyword extraction + content summarization
parallel := adk.NewParallelAgent(ctx, &adk.ParallelAgentConfig{
    Name: "multi_analysis",
    SubAgents: []adk.Agent{
        sentimentAgent, // Sentiment analysis
        keywordAgent,   // Keyword extraction
        summaryAgent,   // Content summarization
    },
})

• LoopAgent: Executes the registered agents in the configuration in sequence and loops multiple times. The operation follows these principles:

  • Loop execution: Repeatedly executes the SubAgents sequence, with each loop being a complete Sequential execution process.

  • Accumulation of run results: The results of each iteration are accumulated, and the input of subsequent iterations can access all historical information.

  • Conditional exit: Supports terminating the loop by outputting an event containing an ExitAction or reaching the maximum number of iterations. Configuring MaxIterations=0 means an infinite loop.

• Possible practical scenarios include:

  • Data Synchronization: CheckUpdateAgent (checks for incremental changes in the source database) → IncrementalSyncAgent (synchronizes incremental data) → VerifySyncAgent (verifies consistency)

  • Pressure Testing: StartClientAgent (starts a test client) → SendRequestsAgent (sends requests) → CollectMetricsAgent (collects performance metrics)

Skeleton code for a LoopAgent:

import "github.com/cloudwego/eino/adk"

// Loop 5 times, with the sequence each time being: analyze the current state -> propose an improvement plan -> verify the improvement effect
loop := adk.NewLoopAgent(ctx, &adk.LoopAgentConfig{
    Name: "iterative_optimization",
    SubAgents: []adk.Agent{
        analyzeAgent,   // Analyze the current state
        improveAgent,   // Propose an improvement plan
        validateAgent,  // Verify the improvement effect
    },
    MaxIterations: 5,
})

Pre-built Multi-Agent Paradigms

Eino ADK provides three pre-built multi-agent paradigms:

• The Supervisor pattern for centralized coordination

• The Plan-Execute pattern for structured problem solving

• The DeepAgents pattern for complex workflows that require multi-step, multi-role collaboration

Developers can use them out of the box without having to design collaboration logic from scratch. The following sections lay out the patterns along with skeleton code.

Supervisor Pattern

The Supervisor agent is a centralized multi-agent collaboration pattern, designed to provide a solution for the general scenario of centralized decision-making and distributed execution.

It consists of a Supervisor agent and multiple sub-agents, where:

• The Supervisor agent is responsible for task allocation, summarizing the results after the sub-agents are completed, and making the next decision.

• The sub-agents focus on executing specific tasks and automatically return control of the task to the Supervisor after completion.

The Supervisor pattern has the following characteristics:

• Centralized control: The Supervisor uniformly manages the sub-agents and can dynamically adjust task allocation based on the input and the execution results of the sub-agents.

• Deterministic callback: After a sub-agent has finished executing, it will return the running result to the Supervisor agent, avoiding interruption of the collaboration process.

• Loosely coupled extension: Sub-agents can be developed, tested, and replaced independently, which is useful for expansion and maintenance.

This hierarchical structure of the Supervisor pattern is suitable for scenarios that dynamically coordinate multiple specialized agents to complete complex tasks, such as:

• Research project management: The Supervisor assigns research, experimentation, and report writing tasks to different sub-agents.

• Customer service processes: The Supervisor assigns tasks to technical support, after-sales, and sales sub-agents, based on the type of user problem.

Skeleton code:

import "github.com/cloudwego/eino/adk/prebuilt/supervisor"

// Research project management: create a multi-agent in supervisor mode
// Contains three sub-agents: research, experimentation, and report
supervisor, err := supervisor.New(ctx, &supervisor.Config{
    SupervisorAgent: supervisorAgent,
    SubAgents: []adk.Agent{
        researchAgent,
        experimentationAgent,
        reportAgent,
    },
})

Plan-Execute-Replan Pattern

The Plan-Execute-Replan agent is a multi-agent collaboration pattern based on the “plan-execute-reflect” paradigm (refer to the 2023 paper Plan-and-Solve Prompting by Wang L. et al). Through the collaborative work of three core intelligent agents, it achieves structured planning of tasks, tool call execution, progress evaluation, and dynamic replanning:

• Planner: Generates a structured initial task plan with detailed steps based on the user’s goals.

• Executor: Executes the first step in the current plan.

• Replanner: Evaluates the execution progress and decides whether to revise the plan and continue to have the Executor run it, or to end the task.

The Plan-Execute-Replan pattern has the following characteristics:

• Clear hierarchical architecture: By breaking down the task into three stages –planning, execution, and reflection/replanning – a hierarchical cognitive process is formed, which embodies the closed-loop cognitive strategy of “think before you act, then adjust based on feedback.”

• Dynamic iterative optimization: The Replanner determines in real time whether the task is complete or needs to be adjusted based on the execution results and current progress, and supports dynamic replanning. This mechanism effectively solves the bottleneck of traditional single-planning – where there is difficulty coping with environmental changes and task uncertainty – and improves the robustness and flexibility of the system.

• Clear responsibilities and loose coupling: The Plan-Execute-Replan pattern consists of multiple intelligent agents working together, and supports independent development, testing, and replacement. The modular design is useful for expansion and maintenance and conforms to engineering best practices.

• Good extensibility: It does not rely on a specific language model, tool, or agent, which is convenient for integrating diverse external resources to meet the needs of different application scenarios.

The “plan → execute → replan” closed-loop structure of the pattern is suitable for complex task scenarios that require multi-step reasoning, dynamic adjustment, and tool integration, such as:

• Complex research analysis: Decompose research questions through planning, perform multiple rounds of data retrieval and calculation, and dynamically adjust research directions and hypotheses to improve the depth and accuracy of the analysis.

• Automated workflow management: Decompose complex business processes into structured steps, combine them with a variety of tools (such as database queries, API calls, and compute services) to execute them step-by-step, and dynamically optimize the process based on execution results.

• Multi-step problem-solving: Scenarios that require step-by-step reasoning and multi-tool collaboration, such as legal consultation, technical diagnosis, and strategy formulation, to ensure that each step of execution has feedback and adjustment.

• Intelligent assistant task execution: Supports intelligent assistants to plan task steps based on user goals, call external tools to complete specific operations, and adjust subsequent plans based on replanning and thinking combined with user feedback, improving the completeness and accuracy of task completion.

Skeleton code:

import "github.com/cloudwego/eino/adk/prebuilt/planexecute"

// A research assistant in Plan-Execute mode
researchAssistant := planexecute.New(ctx, &planexecute.Config{
    Planner: adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
        Name:        "research_planner",
        Instruction: "Create a detailed research plan, including literature research, data collection, analysis methods, etc.",
        Model:       gpt4Model,
    }),
    Executor: adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
        Name: "research_executor",
        ToolsConfig: adk.ToolsConfig{
            Tools: []tool.BaseTool{
                scholarSearchTool,
                dataAnalysisTool,
                citationTool,
            },
        },
    }),
    Replanner: replannerAgent,
})

DeepAgents Pattern

DeepAgents is a multi-agent pattern unified under the coordination of a MainAgent. The MainAgent leverages a ChatModel with tool-calling capabilities to operate through a ReAct workflow:

• It decomposes user goals into structured to-do items and records progress through WriteTodos.

• It selects and invokes corresponding sub-agents to execute subtasks through the unified TaskTool interface; main/sub-agent contexts are isolated to prevent intermediate steps from contaminating the main workflow.

• It aggregates results returned by various sub-agents; when necessary, it calls WriteTodos again to update progress or perform replanning until completion.

The characteristics of the DeepAgents pattern include:

• Enhanced task decomposition and progress management: WriteTodos creates clear subtasks and milestones, making complex goals decomposable and trackable.

• More robust context isolation: Sub-agents execute in “clean” contexts while the main agent only aggregates results, reducing interference from redundant reasoning chains and tool call traces on the main workflow.

• Unified delegation interface with easy extensibility: TaskTool abstracts all sub-agents and tool capabilities into a unified calling surface, facilitating the addition or replacement of specialized sub-agents.

• Flexible closed-loop of planning and execution: Planning functions as a tool that can be called on demand; for simple tasks, unnecessary planning can be skipped, reducing LLM call costs and time consumption.

• Boundaries and trade-offs: Over-decomposition increases call frequency and costs; it places higher demands on subtask division and prompt optimization, requiring models to possess stable tool-calling and planning capabilities.

The core value of DeepAgent lies in automatically handling complex workflows that require multi-step, multi-role collaboration. It’s not just an executor of single functions but a “project manager” with deep thinking, planning, and dynamic adjustment capabilities.

Suitable application scenarios include:

• Complex business processes with multi-role collaboration: Centered around R&D, testing, release, legal, and operations roles, with centralized delegation of subtasks and unified aggregation; each stage sets gateways and rollback strategies, with visible progress and retry capabilities.

• Phased management of long workflows: Planning decomposes steps such as data cleaning, validation, lineage analysis, and quality inspection; sub-agents run in isolated contexts; when exceptions occur, only relevant stages are re-run, with unified reconciliation and aggregation of outputs.

• Execution environments requiring strict context isolation: A unified interface collects materials and requests, with TaskTool routing subtasks such as legal, risk control, and finance separately; boundaries between subtasks are clearly defined and mutually invisible, with auditable progress and traceability, and failures can be retried without affecting other stages.

Skeleton code:

import "github.com/cloudwego/eino/adk/prebuilt/deep"

agent, err := deep.New(ctx, &deep.Config{
    Name:       "deep-agent",
    ChatModel:  gpt4Model,
    SubAgents: []adk.Agent{
        LegalAgent,
        RiskControlAgent,
        FinanceAgent,
    },
    MaxIteration: 100,
})

Example: Project Manager Intelligent Agent

Now you have covered the basics, it’s time to create an agent. Install the latest version of Eino using the following command:

go get github.com/cloudwego/eino@latest

In this example, you’ll see how to build a Project Manager intelligent agent for a variety of scenarios. It uses the Supervisor pattern, and the functions of each agent are as follows:

• ResearchAgent: Responsible for researching and generating feasible solutions, supporting interruption and receiving additional context information from users to improve the accuracy of research plan generation.

• CodeAgent: Uses knowledge base tools to recall relevant knowledge as a reference to generate high-quality code.

• ReviewAgent: Uses a sequential workflow to orchestrate three steps: problem analysis, evaluation generation, and evaluation verification to review research results/coding results and provide reasonable evaluations so project managers can make decisions.

• ProjectManagerAgent: Routes and coordinates multiple sub-intelligent agents responsible for different dimensions of work based on dynamic user input.

The design of this example covers most of the concepts introduced in the article.

The core code is as follows. The complete code can be found in the source code provided in the eino-examples project:

func main() {
    ctx := context.Background()

    // Init chat model for agents
    tcm, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
        APIKey:  os.Getenv("OPENAI_API_KEY"),
        Model:   os.Getenv("OPENAI_MODEL"),
        BaseURL: os.Getenv("OPENAI_BASE_URL"),
        ByAzure: func() bool {
            return os.Getenv("OPENAI_BY_AZURE") == "true"
        }(),
    })
    if err != nil {
        log.Fatal(err)
    }

    // Init research agent
    researchAgent, err := agents.NewResearchAgent(ctx, tcm)
    if err != nil {
        log.Fatal(err)
    }

    // Init code agent
    codeAgent, err := agents.NewCodeAgent(ctx, tcm)
    if err != nil {
        log.Fatal(err)
    }

    // Init technical agent
    reviewAgent, err := agents.NewReviewAgent(ctx, tcm)
    if err != nil {
        log.Fatal(err)
    }

    // Init project manager agent
    s, err := agents.NewProjectManagerAgent(ctx, tcm)
    if err != nil {
        log.Fatal(err)
    }

    // Combine agents into ADK supervisor pattern
    // Supervisor: project manager
    // Sub-agents: researcher / coder / reviewer
    supervisorAgent, err := supervisor.New(ctx, &supervisor.Config{
        Supervisor: s,
        SubAgents:  []adk.Agent{researchAgent, codeAgent, reviewAgent},
    })
    if err != nil {
        log.Fatal(err)
    }

    // Init Agent runner
    runner := adk.NewRunner(ctx, adk.RunnerConfig{
        Agent:           supervisorAgent,
        EnableStreaming: true,               // enable stream output
        CheckPointStore: newInMemoryStore(),  // enable checkpoint for interrupt & resume
    })

    // Replace it with your own query
    query := "please generate a simple ai chat project with python."
    checkpointID := "1"

    // Start runner with a new checkpoint id
    iter := runner.Query(ctx, query, adk.WithCheckPointID(checkpointID))

    interrupted := false
    for {
        event, ok := iter.Next()
        if !ok {
            break
        }

        if event.Err != nil {
            log.Fatal(event.Err)
        }

        if event.Action != nil && event.Action.Interrupted != nil {
            interrupted = true
        }

        prints.Event(event)
    }

    if !interrupted {
        return
    }

    // interrupt and ask for additional user context
    scanner := bufio.NewScanner(os.Stdin)
    fmt.Print("\ninput additional context for web search: ")
    scanner.Scan()
    fmt.Println()
    nInput := scanner.Text()

    // Resume by checkpoint id, with additional user context injection
    iter, err = runner.Resume(
        ctx,
        checkpointID,
        adk.WithToolOptions([]tool.Option{agents.WithNewInput(nInput)}),
    )
    if err != nil {
        log.Fatal(err)
    }

    for {
        event, ok := iter.Next()
        if !ok {
            break
        }

        if event.Err != nil {
            log.Fatal(event.Err)
        }

        prints.Event(event)
    }
}

If you stop and think about how you would build this example with regular tools, the advantages of ADK will become apparent:

Useful Resources

Find out more about Eino using the following links:

• Read the docs

• Browse source code on GitHub

• Explore examples on GitHub

If you have questions about Eino, head to GitHub to fill out an issue.

If you want to get in touch with the Eino Team Lead, Li Pandeng, reach out to lipandeng [at] bytedance [dot] com.

Architecting fundamentals can help navigate these complexities and the successful building and operation of these sorts of systems. Historically, an architect would aid in the building of a system and then would be able to sort of “walk away” or execute a new project. In this age of AI, this is simply not the case. The challenges of observability, integrated development, testing, the voice of the user, and their centrality to business outcomes place new demands on an architect.

Challenges that impact AI enabled systems

• Complex integrations: For example, is the system going to be a hybrid cloud or on premise deployment? What dependencies exist on external software?

• High performance bar: the system is expected to be deployed to impact key business functions or activities that the organization shall depend upon.

• Depending on the technology, using the AI/ML can incur significant costs.

• If not well architected or instrumented, the system can be a challenge to troubleshoot.

What can architecture do to help?

• By modeling and conducting robust conceptual designs, the unique perspective and aspects of the AI technology are identified.

• The use of tactics and patterns partitions the system for better performance and meeting non-functional requirements.

• Robust architecting helps keep a holistic view of the end to end system.

The rise of AI architects

History has conclusively shown that as modern technology progresses, the practice of architecting also evolves. We are now witnessing the rise of the AI architect. The modern AI architect will be closer to operations and must help with managing the complexity of systems. These include:

• The architect is the principal approver for the completion of acceptance gates for going from an AI prototype to a production deployment. He ensures the concept has merit, evaluates the results of the prototype, guides design of the production model, evaluates the performance of prototypes, and finally accepts the production deployment.

• They integrate and synthesize solutions to balance the data science, data engineering, software development, operations, and business teams.

• The architect is the principal owner of the non-functional requirements and interfaces across the AI system.

• They must be involved in the observability, where they need to understand what decisions the software is making.

From Idea to Production Life cycle

As AI engineering is maturing, a notional process is emerging that takes an AI idea to production. The initial stage is discovery and evaluation. It looks to determine if the data, algorithms, and compute resources exist to make a system that will have impact. The next stage is prototyping; here one now adds complexity to the software, both by adding more production domain elements, to test how well the new system would work. It is at the end of the prototype phase where a key decision is made to determine if it is warranted to take the prototype to a production environment.

With the passage out of the prototyping phase, a controlled execution of the model is done, where it is observed and monitored to ensure it is working as expected and delivering value. With the monitoring phase complete, a controlled rollout of the system is done to production, with another layer of monitoring to ensure the systems it is now impacting still also function as expected. Finally, with these gates completed, a full deployment can be done and now the system goes into a monitor and evaluation phase. The evaluation is to see if the model needs to be adapted or retrained.

One key item is as the system is now in operation – a new set of metrics for system observability need to be captured. Below are some basic ones; this should not be considered exhaustive:

1. Accuracy

2. Data throughput

3. Area under the Curve

4. Time for processing

5. Cost per transaction

How often these metrics should be reviewed really depends on how time dependent AI performance is to the enterprise. That said, these observability metrics should be reviewed frequently by the operations teams, and on a weekly basis or sooner by the other stakeholders, to see how AI model performance is impacting their areas of responsibilities.

Specific areas an AI architect can influence

As has been mentioned, the AI architect needs to have a solid understanding of, and be able to communicate in the language of, AI/ML and data science topics. This includes the perspective in understanding the role of non-functional requirements as applied to AI/ML systems. How to extend their knowledge for architecture modeling should incorporate systems where there is to be a sort of non-human decision making. They will also need to fortify and understand how users and stakeholders of the system are to be impacted, aided, and potential friction points. They will need to be close to guiding and evaluating the rapid prototyping in support of software development. The architect needs to consider interfaces and how their management will impact the decision making of the system. The use of AI/ML technologies inherently add a layer of complexity. The architect acts at his own peril with the notion that a simple AI application can readily be made in a less rigorous manner. Adding a layer of AI into almost any application raises the complexity significantly.

The AI architect must by necessity straddle AI/ML and software engineering. This is to ensure the architect can navigate the alphabet soup of techniques, algorithmic aspects, and technologies that exist. It is very easy for an AI architect to be overcome by the various other roles needed to deploy modern AI systems. In many enterprises, there are typically several functional teams: Data Team, Development Team, ML Ops, Legal/Compliance. Below are some of the major facets that the architect influences and needs to be involved with:

Data Team

The architect works with the data team to ensure that the product question can indeed be answered by the data that exists or is to be used. He is pivotal in levying requirements for the speed, format, and consistency checks that must be met.

Development Team

The architect works to ensure top-level designs, interfaces, and non-functional requirements are being met by the designs being created. He is more in a reviewer mode. He also aids in clarifying engineering requirements and providing guidance to address design questions and challenges.

MLOps Team

The architect, in this role, defines how well the production system is required to perform, and how the system is to be instrumented and monitored for correct execution. He will also define the canaries and fail-safe mechanisms to ensure issues with the AI components do not compromise the full production environment. The mechanisms for model and system updating are also laid out by the architect.

Legal and Compliance Team

In this capacity, the architect provides the technical insight, design guidance, and evidence that the AI system is meeting the legal and compliance needs of the enterprise. The architect also oversees re-design activities and troubleshooting to ensure legal and compliance aspects are indeed met.

Incident Run Book

There needs to exist a run book that the architect is a principal in its development. A run book should look to use a layered manner to address an error in the AI model or its interfaces. Clear mechanisms and architecture need to exist to be able to disengage the model from a production system, and put the production system into a nominal configuration. The run book should also provide for clear expected outputs of the different stages of the system – that is, it should be documented what “correct” looks like and why at each stage to enable quicker troubleshooting. The error handling within the system should also be readily indexed in the run book so that system, data, and model tracing can occur. Finally, the run book should also include clear traceability to software repositories, configuration information, and points of contact.

Conclusion

These are exciting times for being an AI architect. The field is in its infancy and there are many excellent and challenging domains where one can make a significant impact. In this new era one needs to accept that there will be constant learning and adaptation. As the techniques and concepts of AI/ML continue to increase, the combinations and applications simply keep growing. That said, having a solid foundational knowledge of AI/ML concepts shall mitigate short-term knowledge gaps. Are you ready to become an AI architect?

Chapter 1: Thinking Computationally (from The C++ Programmer's Mindset By Sam Morley) © Packt (All Rights Reserved)

1.08MB ∙ PDF file

Download

Solving problems is a central part of being a programmer, as well as a useful skill for everyday life. The methodology is broadly the same wherever you look: identify smaller, more tractable challenges; realize these as instances of a general class of problem; solve the intermediate challenges; and put everything together as a sequence of simple steps to solve the larger problem. In computer science, we call this computational thinking. This chapter serves as an introduction to the basic components of computational thinking at a high level. The objectives are to lay the foundation for more detailed analysis and in-depth examples later in the book. The first part of the chapter introduces the four components of computational thinking (decomposition, abstraction, pattern recognition, and algorithm design). The second half of the chapter deals with C++ specifically and identifies some aspects of the C++ language and standard library that can not only help implement efficient solutions, but also help you think about the problems themselves. It is important to remember that the four components of computational thinking are not a step-by-step guide to solving problems. Learning how and when these different components come together to deliver a solution relies on a good knowledge of the tools and methodologies available to you as the solver, and on your past experience. This chapter will help you get started with building the necessary foundations of the theory and set the stage for building out some basic examples to get you started with tackling larger and more complex problems later. Solving problems is an iterative process. There will be many failed attempts and false starts. This is a necessary part of the process. The last part of the chapter deals with good software practices that will enable you to iterate quickly and easily on your designs and arrive at a correct and usable solution more quickly. In this chapter, we’re going to cover the following main topics: • The components of computational thinking • Decomposing problems • Building abstractions and recognizing common patterns • Understanding algorithms • Using modern C++ and good practice Download to continue reading...

Download

To go deeper check out The C++ Programmer’s Mindset (Sam Morley, Packt, 1st ed., Nov 2025). The book introduces computational thinking as a practical framework—decomposition, abstraction, and pattern recognition—and shows how to apply it using modern C++ features to build solutions that are maintainable, efficient, and reusable. Across small examples and a larger case study, Morley covers using algorithms and data structures effectively, designing modular code, analyzing performance, and scaling work with concurrency, GPUs, and profiling tools—aimed at intermediate C++ developers who want to strengthen both their technical toolkit and the way they approach complex software challenges.

Here’s what some readers have said:

AI has emerged as a transformative force, revolutionizing industries and reshaping the way we interact with technology and the world around us. At its core, AI refers to computational models that mimic human cognitive functions, including learning from data, recognizing patterns, making decisions, and even interacting with their environment. This revolutionary technology spans a wide spectrum, from simple rule-based systems to sophisticated deep learning models, each with unique applications and capabilities.

A major aspect of any AI system is that the results of the inference being done need to be relevant and trusted. To ensure trust is gained and maintained, the use of strong architecture is paramount. One not only architects the technology but also how the technology is going to be used, managed, and evaluated by the span of stakeholders. The stakeholders need to be able to pinpoint issues, rapidly correct model parameters, and deploy changes in a deliberate and rapid manner. In more common parlance, the architecting and supporting processes can be described as “guard rails.” How one employs guard rails is very specific to the domain and use case that is to use the AI technology. There are classes of guard rails that can be discussed – for example, the use of canaries to judge model correctness from a known gold standard, time and data flow metrics to judge model performance, and the use of filters and robust data quality checks so that only consistent and correct data enters the system. Another class of guardrails is human system interfaces, such as alerting frameworks to classify errors and monitors, the use of troubleshooting tools, and preset protocols for handling unexpected errors. Written procedures or guidance from modeling allow for the maintenance of a system without the need to call upon the model developer to do troubleshooting.

Trust is a paramount consideration for system success, so one needs to architect a system with that in mind. In many ways, the presentation and lessons learned described in this book look to ensure trust in an AI system.

This chapter highlights, in a broad sense, the key aspects of AI architecture considerations that drive a successful AI implementation. The topics are as follows:

• Introduction and key AI concepts

• Components of an AI system

• AI technologies and microservices

• AI systems and technical considerations

• Deployment considerations

Introduction to AI systems: architecting the future of intelligence

AI systems are the embodiment of AI, acting as the engines that power intelligent applications and services. These systems are intricate constructs, meticulously designed to perform a diverse range of tasks, from image recognition and natural language processing to autonomous decision-making and complex problem-solving.

The architecture of an AI system functions as a detailed technical blueprint, specifying its structural organization and the precise interactions between its various components. These components include the following:

• Hardware infrastructure: CPUs for general processing, GPUs for parallel computation, TPUs for tensor operations, and specialized AI accelerators

• Software frameworks: TensorFlow, PyTorch, JAX, and other libraries that enable model development

• Algorithmic implementations: Machine learning algorithms, neural network architectures, and inference engines

• Data pipelines: ETL processes, feature stores, and data management systems

All these elements work in a coordinated operation to enable the system to fulfill its designed objectives efficiently and reliably.

A well-architected AI system achieves several critical technical requirements:

• Optimal performance: Maximizes computational efficiency to deliver responsive and accurate results with minimal latency. This involves an optimized model design, efficient resource allocation, and hardware-aware implementations that fully utilize available computing capabilities.

• Scalability: Handles growing workloads and expanding datasets through both horizontal scaling (adding more machines) and vertical scaling (adding more powerful machines) without performance degradation. Modern AI architectures must accommodate increasing data volumes, user bases, and computational demands.

• Efficiency: Reduces computational resource consumption, energy usage, and operational costs through techniques such as model quantization, knowledge distillation, and optimized inference paths. Efficient AI systems minimize their resource footprint while maintaining functional effectiveness.

• Reliability: Ensures consistent operation with high-availability metrics, even when facing unexpected data patterns, input variations, or system failures. This requires robust error handling, graceful degradation capabilities, and comprehensive monitoring systems. Given that AI technologies can be both deterministic and non-deterministic, consideration must be given to allow for human intervention. This intervention needs to span the gamut from simple monitoring to a full suite of testing infrastructure.

• Security: Implements comprehensive data protection measures and defends against adversarial attacks, data poisoning, and model vulnerabilities. AI systems must maintain data confidentiality and integrity, and be resilient against both traditional cybersecurity threats and AI-specific attacks.

• Explainability: Provides transparent visibility into algorithmic decision processes, supporting regulatory compliance, user trust, and system debugging. Modern AI architectures must balance performance with interpretability to meet growing demands for AI transparency.

The field of AI is constantly evolving, with new architectures and technologies emerging at a rapid pace. As we delve deeper into this fascinating domain, we will explore the various types of AI systems, their underlying principles, and the diverse applications that are shaping the future of technology and society.

What is an AI system?

An AI system is a computational model or a collection of models designed to perform tasks that typically require human intelligence. These systems are powered by algorithms and data, enabling them to learn from experience, adapt to new information, and make decisions or predictions.

From an implementation perspective, AI systems typically consist of several key layers:

1. Hardware: Encompasses compute resources such as CPU, GPU, TPUs, full-spectrum storage, and networking

2. Data layer: Handles data ingestion, storage, preprocessing, and feature engineering

3. Model layer: Contains the trained machine learning or deep learning models

4. Inference layer: Manages the execution of models against new data inputs

5. Application layer: Integrates AI capabilities into user-facing applications

6. Monitoring layer: Tracks system performance, data drift, and model health

AI systems can be classified into two broad categories:

• Narrow AI (weak AI): These systems are designed to excel at specific tasks within a limited domain. Examples include image recognition software, spam filters, and recommendation engines. While they may be highly proficient at their designated tasks, they lack the ability to generalize their knowledge in other areas.

• General AI (strong AI): This is a theoretical concept of an AI system that possesses human-level intelligence and can perform any intellectual task that a human can. It would have the ability to reason, plan, solve problems, learn from experience, and understand complex ideas across diverse domains. While general AI remains a distant goal, significant progress has been made in developing systems with increasingly sophisticated capabilities.

The pervasive impact of AI infrastructure: powering intelligent solutions across industries

Well-architected AI infrastructure, encompassing the hardware, software, and networks that support AI applications, is the driving force behind the transformative impact of AI across industries. This infrastructure enables the deployment and scaling of AI models, algorithms, and frameworks, unlocking their full potential to address complex challenges and deliver innovative solutions.

• Healthcare:

  • Accelerated medical image analysis: High-performance computing clusters and specialized hardware accelerators enable rapid processing of medical images, facilitating faster and more accurate diagnosis.

  • Data-driven insights: Scalable storage and processing infrastructure empowers AI-driven analytics on vast patient datasets, leading to personalized treatment plans and improved patient outcomes.

  • Real-time monitoring: Cloud-based AI infrastructure enables continuous monitoring of patient vitals and other health data, facilitating timely interventions and proactive care.

• Finance:

  • Robust fraud detection: Distributed computing and real-time analytics platforms empower AI models to detect fraudulent transactions with greater accuracy and speed, protecting financial institutions and consumers.

  • Optimized trading strategies: High-frequency trading algorithms leverage low-latency networks and powerful computational resources to execute trades with precision and efficiency, maximizing returns.

  • Personalized financial services: Cloud-based AI infrastructure enables the deployment of robo-advisors and other AI-powered tools that provide tailored financial advice and services to individuals.

• Autonomous vehicles:

  • Real-time sensor fusion: High-throughput data pipelines and edge computing infrastructure enable the rapid processing of sensor data from cameras, lidar, radar, and other sources, ensuring timely decision-making for autonomous vehicles.

  • Enhanced object recognition: Deep learning models trained on massive datasets and deployed on specialized hardware accelerators enable accurate and reliable identification of objects in the environment.

  • Optimized navigation: Cloud-based mapping and navigation services, combined with onboard AI processing, provide autonomous vehicles with real-time information and guidance for safe and efficient navigation.

The continued development and optimization of AI infrastructure will play a crucial role in realizing the full potential of AI across industries. By providing the foundation for performant and scalable AI solutions, this infrastructure is poised to transform the way we live and work.

Key components of AI system architectures

AI systems, in their essence, are complex structures designed to emulate human cognitive abilities such as learning, reasoning, and problem-solving. To achieve these capabilities, AI systems rely on a well-defined architecture comprising several interconnected components, each playing a crucial role in the overall functioning of the system. Understanding these key components is fundamental to comprehending the inner workings and potential of AI.

• Data components: Data serves as the lifeblood of any AI system, acting as the raw material upon which the system learns and improves. Data can exist in multiple forms:

  • Structured data: Organized in predefined formats such as databases and spreadsheets

  • Semi-structured data: Partially organized information such as JSON or XML files

  • Unstructured data: Raw information, including text documents, images, audio recordings, and video files

The quality, quantity, and relevance of the data significantly impact the AI system’s performance and ability to generalize to new situations.

• Algorithmic frameworks: Algorithms are engines driving AI systems, providing instructions and logic for processing data and generating intelligent outputs. Machine learning algorithms, a subset of AI algorithms, empower systems to learn patterns and relationships from data, enabling them to make predictions, classifications, or decisions. Common algorithmic approaches in production AI systems include the following:

  • Traditional machine learning: Linear regression, random forests, gradient boosting, and support vector machines

  • Deep learning: Convolutional neural networks (CNNs), Recurrent Neural Networks (RNNs), transformers, and graph neural networks

  • Reinforcement learning: Q-learning, policy gradient methods, and actor-critic architectures

The selection of appropriate algorithms depends on the specific problem domain, available data characteristics, and performance requirements.

• Model architectures: Models represent the culmination of the learning process in AI systems. They are mathematical representations of the knowledge extracted from data, encapsulating the patterns, relationships, and insights discovered by the algorithms. These models can be simple or complex, depending on the nature of the task and the algorithm used. Model architectures range between the following:

  • Simple linear models: Easily interpretable but limited in capability

  • Ensemble models: Combining multiple simpler models for improved performance

  • Deep neural networks: Complex architectures with millions or billions of parameters

Once trained, models are used to make predictions or decisions on new, unseen data.

• Infrastructure: The infrastructure component encompasses the hardware and software resources that provide the computational power and environment necessary for AI systems to operate. Key infrastructure elements include the following:

  • Computational resources: High-performance servers, specialized AI accelerators (GPUs, TPUs, FPGAs), and distributed computing clusters

  • Storage systems: High-throughput, scalable storage for training data and model artifacts

  • Networking components: Low-latency interconnects for distributed training and inference

• Development frameworks: Software libraries such as TensorFlow, PyTorch, and Hugging Face that streamline AI development and deployment.

Understanding these key components and their interactions provides a solid foundation for comprehending the complex landscape of AI system architectures. By carefully designing and optimizing each component, researchers and engineers can build AI systems that are capable of tackling a wide range of tasks and applications, from image recognition and natural language processing to autonomous driving and drug discovery. The integration of AI capabilities into existing software stacks requires thoughtful architectural considerations to successfully incorporate intelligence while addressing the unique requirements that AI components introduce. These specific requirements and architectural approaches form the central focus of this book. Due to the complexity of AI systems, the nature of the deployment approach is paramount. The next section will discuss the use of microservice architectures that provide a balance between performance and modularity.

Microservice architectures: a modular approach to building complex AI systems

As AI systems grow in complexity, traditional monolithic architectures can become unwieldy, hindering development speed and flexibility. Microservice architectures offer a compelling alternative by breaking down these complex systems into smaller, independent services. Each microservice focuses on a specific function and communicates with others through well-defined APIs.

Advantages of microservices for AI

• Enhanced agility and flexibility: Teams can independently develop, deploy, and update each microservice, using the most suitable technologies and programming languages for each task. This accelerates development cycles and allows for easier experimentation and innovation.

• Improved scalability: Microservices can be scaled horizontally to meet specific demand, ensuring optimal resource utilization. For example, a service handling image processing can be scaled independently of a service responsible for natural language understanding.

• Increased resilience and fault isolation: If a microservice fails, the impact is localized, minimizing disruption to the entire system. This enhances overall reliability and simplifies troubleshooting.

• Technological diversity: Microservice architectures empower teams to leverage the best tools for each task, promoting innovation and allowing for gradual technology upgrades.

Challenges of microservice architectures

• Increased complexity: Managing a multitude of services and their interactions requires robust orchestration and monitoring tools. Service discovery, load balancing, and failure handling become critical considerations.

• Communication overhead: Excessive inter-service communication can introduce latency and impact overall performance. The careful design of APIs and communication patterns is essential to mitigate this issue.

• Data consistency: Maintaining data consistency across distributed services can be challenging. Strategies such as eventual consistency or distributed transactions may be required to ensure data integrity.

Real-world example: conversational AI microservices implementation

To illustrate how a microservices approach can streamline a conversational AI solution, let us examine a practical example that demonstrates how these principles come to life. This section explores a conversational AI system – such as a chatbot or virtual assistant – built using a four-service microservices architecture with an API gateway.

The four core microservices

Figure 1.6 illustrates the high-level design of our conversational AI system:

The architecture consists of four core specialized services plus an API gateway:

1. Language understanding service:

   • Primary functions: Intent classification, entity identification/extraction, and hosting of NLP models.

   • Data and models: References one or more NLP model databases (for example, transformer-based classifiers).

   • Key interactions: Receives the user’s text (through the API gateway), determines the user’s intent (e.g., “Check account balance”), and extracts relevant entities (e.g., “date,” “location,” “product name”).

2. Dialog management service:

   • Primary functions: Oversees conversation flow, handles session state, and orchestrates the next step in the dialog.

   • Data and state: Maintains conversation context in a dedicated state database.

   • Key interactions: Logs conversation events (asynchronously) and updates or retrieves session details to guide the flow (e.g., “Greeting,” “Confirmation,” “Next step”).

3. Knowledge response service

   • Primary functions: Retrieves relevant information and formulates responses. This might involve querying a knowledge base (e.g., FAQs, product info) or assembling template-based replies.

   • Data and templates: Stores domain-specific data in a knowledge DB and uses templates or generative mechanisms for response creation.

   • Key interactions: Receives queries from the dialog management service, finds or composes the best response, and returns it for final delivery to the user.

4. Conversation analytics service:

   • Primary functions: Processes logs and usage metrics for reporting, visualization, and deeper analytics (e.g., intent distribution, user satisfaction trends).

   • Data and reporting: Maintains analytics data in a separate database for dashboards or offline processing.

   • Key interactions: Collects asynchronous event logs from the dialog management service and other components to measure performance, track user behavior, and provide insights that could improve the system over time.

Role of the API gateway

Although not counted as one of the four microservices, the API gateway is a vital component at the front of the architecture. It does the following:

• Receives requests from the user (via text or other channels)

• Initializes the session and routes incoming data to the language understanding service

• Forwards recognized intents and updates to the dialog management service

• Passes replies from downstream services back to the user

By centralizing traffic management, the API gateway enforces consistent security, throttling, and monitoring policies while keeping each microservice isolated and independently scalable.

Conversation flow sequence

To illustrate how these microservices interact during a typical user journey, Figure 1.7 shows the sequence of calls between them in a single conversation cycle:

The sequence progresses as follows:

1. User → API gateway: The user sends a request (e.g., a chat message). The API gateway initializes the session (if needed) and forwards the message to the language understanding service.

2. Language understanding service:

   • Performs intent classification and entity identification.

   • Returns a recognized intent (e.g., “CheckWeather”) and any extracted entities (e.g., date, location) to the API gateway.

3. Dialog management service:

   • Receives recognized intent from the API gateway.

   • Logs conversation events (asynchronously) into the conversation analytics service.

   • Updates or retrieves the session state (e.g., user’s location or recent conversation context).

4. Knowledge response service:

   • Once the dialog management service determines additional data is needed (e.g., weather info, product detail), it sends a query to the knowledge response service.

   • This service fetches the necessary information or constructs a response template (e.g., “The weather for your location is sunny with 75°F...”).

5. Conversation analytics service (asynchronous logging):

   • Continuously receives usage data and conversation logs from the dialog management service (and possibly from the knowledge response service).

   • Processes and stores these logs for future reporting (e.g., monthly usage dashboards, model performance metrics).

6. Reply to the user:

   • The knowledge response service’s formulated answer is routed back through the dialog management service (if necessary, for final session updates) and then returned via the API gateway.

   • The user receives the reply and the interaction concludes.

Key aspects of microservice communication

• Synchronous versus asynchronous calls:

  • Requests that must return immediately (e.g., generating a response for the user) use synchronous calls.

  • Logging or analytics operations are typically performed asynchronously to avoid slowing down the core conversation loop.

• Stateful versus stateless components:

  • Dialog management requires tracking session state, while other services (e.g., language understanding) often benefit from stateless designs for simpler scaling.

  • The dialog management service may require robust state management solutions, such as distributed caches or databases.

• Service autonomy:

  • Each microservice can be updated or replaced independently without affecting the rest of the system.

  • The language understanding service’s NLP models may need frequent retraining. Because it is a separate service, such updates can be deployed without disrupting the other services.

• Data isolation:

  • Services manage their own domain data. Dialog management stores conversation state, knowledge response holds domain facts, and analytics maintains interaction logs.

  • Sensitive user data should be restricted to the dialog management service’s state store when necessary, minimizing the exposure across the entire system.

Implementation considerations for conversational AI microservices

1. Scaling independently:

   • The language understanding service can be scaled up or down based on incoming message load (e.g., horizontal autoscaling for peak chat traffic).

   • The dialog management service maintains conversation state and may require different scaling strategies.

   • The knowledge response service often scales according to the complexity of information retrieval.

   • The conversation analytics service can be scaled separately, especially if analytics workloads (such as report generation) spike at different times than user requests.

2. Latency management:

   • Conversational AI systems aim for near real-time interactions. Minimizing network hops and communication overhead between services is crucial. Using lightweight communication protocols helps ensure the system performs well at scale.

3. Fault isolation:

   • If one service fails (for instance, the knowledge response service goes offline), the rest of the system can still handle other tasks or offer fallback behaviors (e.g., an apology response or a redirect to a human agent).

4. Monitoring and observability:

   • Robust logging and observability practices are crucial to ensure the system remains resilient to service failures or slowdowns. The conversation analytics service plays a key role in tracking system health and performance.

Why microservices for conversational AI?

Breaking down a conversational AI system into these four specialized services confers significant benefits in maintainability, scalability, and team agility. Each service can evolve independently, allowing rapid iteration on NLP models, conversation flows, and knowledge retrieval strategies without risking a “big bang” failure across the entire application.

At the same time, careful attention to inter-service communication is crucial. As the sequence diagram shows, multiple hops occur for every user request. Using lightweight communication protocols and distinguishing between synchronous and asynchronous operations helps maintain system responsiveness.

The example of conversational AI powerfully illustrates how the microservices approach enables balancing flexibility, fault tolerance, and iterative innovation. The lessons learned here – such as independently scaling critical services, isolating data for security, and ensuring graceful failure modes – apply broadly to a wide array of AI-driven solutions.

This real-world implementation pattern demonstrates that while microservices add complexity, the benefits they bring to AI systems – particularly those requiring frequent updates, variable scaling, and component-level innovation – often outweigh the challenges when properly architected and implemented.

Considerations for an AI system

Creating a well-designed AI system architecture necessitates careful consideration of several key factors. These factors ensure that the system not only functions effectively but also adapts to future demands and challenges.

Scalability: handling growing data and model complexity

AI systems often encounter growing volumes of data and increasingly complex models. Scalability is the ability of a system to handle this growth without compromising performance. Effective strategies include the following:

• Horizontal scaling: This involves adding more compute resources to distribute the workload. For instance, in a cloud environment, you might deploy additional virtual machines or containers to handle increased traffic. Kubernetes can orchestrate these containers, ensuring that the workload is evenly distributed.

• Vertical scaling: Enhancing existing resources with more powerful hardware. For example, upgrading a server’s CPU or GPUs, adding more RAM, or using SSDs instead of HDDs to improve I/O performance.

• Distributed computing: Utilizing frameworks such as Apache Spark or Hadoop to process data across multiple nodes. This approach breaks down large datasets into smaller chunks that can be processed in parallel, significantly reducing processing time. For instance, Spark’s Resilient Distributed Datasets (RDDs) allow for in-memory processing, which is much faster than traditional disk-based processing.

Performance: optimization techniques

In many AI applications, real-time or near-real-time processing is crucial. Techniques to optimize performance include the following:

• Hardware acceleration: Leveraging GPUs or TPUs for computationally intensive tasks – for example, TensorFlow and PyTorch can utilize CUDA cores in NVIDIA GPUs to accelerate deep learning model training.

• Parallel processing: Dividing tasks into smaller sub-tasks that can be executed concurrently. In Python, libraries such as multiprocessing or concurrent.futures can be used to parallelize tasks – for instance, training multiple models simultaneously or processing different data batches in parallel.

• Algorithm optimization: Choosing or designing algorithms with lower computational complexity. For example, using approximate nearest neighbor algorithms for large-scale similarity search instead of exact methods, which are computationally expensive.

Reliability: fault tolerance, error handling, and redundancy

Reliability is paramount, especially in critical applications. To ensure system uptime and data integrity, strategies such as fault tolerance, error handling, and redundancy are employed:

• Fault tolerance: The system can continue operating even if some components fail. For example, in a microservices architecture, if one service fails, others can continue to function. Tools such as Netflix’s Hystrix can be used to implement circuit breakers to manage failures.

• Error handling: Mechanisms are in place to detect and correct errors gracefully – for instance, using try-catch blocks in code to handle exceptions and logging errors for further analysis.

• Redundancy: Critical components are duplicated to prevent single points of failure – for example, using RAID configurations for disk storage or deploying services in multiple availability zones in cloud environments to ensure high availability.

Security: data privacy and model robustness

AI systems often handle sensitive data, making security a top priority. Key considerations include the following:

• Data encryption: Protecting data at rest and in transit – for instance, using AES encryption for data stored in databases and TLS for data transmitted over networks. The use of encryption approaches needs to be considered and tested thoroughly to scope the impact on model and system performance.

• Access control: Implementing strict authorization and authentication mechanisms – for example, using OAuth 2.0 for secure API access and role-based access control (RBAC) to manage permissions.

• Model robustness: Guarding against adversarial attacks that could manipulate the system. Techniques such as adversarial training, where the model is trained on both normal and adversarial examples, can help improve robustness. Additionally, you can deploy anomaly detection systems to monitor for unusual patterns in data input.

Data modeling: catalogs and ontologies

In the realm of AI, data is not just a valuable asset but the very foundation upon which intelligent systems are built. As AI models rely heavily on vast amounts of data to learn and make informed decisions, effective management and organization of this data becomes paramount. This is where data catalogs and ontologies step in as indispensable tools for navigating the complexities of data landscapes within AI architectures.

Catalogs serve as centralized repositories of metadata, providing comprehensive information about the data assets within an AI system. They act as a comprehensive index, offering insights into the data’s location, schema, lineage, quality, and other relevant attributes. By consolidating this information in a structured and accessible manner, data catalogs empower data scientists, engineers, and analysts to gain a deeper understanding of their data resources, streamline their workflows, and ensure data governance.

Ontologies give a semantic representation of the data elements within the domain. They can aid the data engineer in understanding how and why data elements are associated and improve processing pipelines. Ontologies also give data scientists context for model development and updating.

The technical and functional attributes of AI systems have been discussed. The next section discusses the different ways to implement systems in a modern cloud context. The use of cloud technology ensures that one can readily scale an AI system based on actual demand and provides for flexibility in resource allocations.

Modern AI deployment paradigms

As AI systems continue to evolve, new deployment paradigms have emerged to address specific requirements and use cases. This section explores two significant approaches: cloud-native AI architectures and edge AI deployments.

Cloud-native AI architectures

The increasing complexity and scale of AI applications have led to the adoption of cloud-native architectures. These architectures leverage the scalability, flexibility, and cost-efficiency of cloud computing platforms to enable efficient development, deployment, and management of AI systems. In a cloud-native architecture, AI components are designed to run seamlessly in cloud environments, taking advantage of specialized services for storage, compute, and networking.

Key characteristics of cloud-native AI architectures include the following:

• Containerization: AI applications are packaged into lightweight, portable containers using technologies such as Docker, ensuring consistency across development, testing, and production environments.

• Orchestration: Container orchestration platforms such as Kubernetes manage the deployment, scaling, and operation of application containers across clusters of hosts.

• Microservices: As discussed earlier, breaking down AI systems into smaller, independent services enables more efficient resource utilization and easier scaling.

• Serverless computing: Platforms such as AWS Lambda, Azure Functions, and Google Cloud Functions allow developers to focus on writing code without worrying about the underlying infrastructure, particularly useful for event-driven AI workloads.

• Managed services: Cloud providers offer specialized AI services such as fully managed machine learning platforms (e.g., Amazon SageMaker, Microsoft Azure ML, Google Vertex AI) that streamline the development and deployment process.

• Cloud-native versus lift-and-shift: Cloud-native AI components are specifically designed to leverage the benefits of cloud environments, such as auto-scaling, serverless computing, and managed services. This approach offers greater flexibility, scalability, and cost-efficiency compared to simply “lifting and shifting” existing on-premises AI systems to the cloud without architectural modifications.

Data lakes and data warehouses in AI architectures: foundations for data-driven intelligence

In the realm of AI, data is the cornerstone of innovation and progress. AI models thrive on massive volumes of data, leveraging it to learn patterns, make predictions, and generate valuable insights. However, effectively managing and harnessing the vast amounts of data involved in AI projects necessitates specialized storage and management solutions. Two prominent concepts that have emerged in this context are data lakes and data warehouses.

Data lakes: a vast reservoir of raw data

Data lakes serve as expansive repositories where raw data is stored in its native format. They are designed to accommodate structured, semi-structured, and unstructured data from diverse sources. The flexibility of data lakes makes them ideal for storing large volumes of data that may not have a predefined purpose or structure.

• Key characteristics:

  • Schema-on-read: Data lakes do not enforce a strict schema during ingestion, allowing for flexibility in data types and structures. The schema is defined during analysis or processing, empowering users to adapt to evolving data requirements.

  • Cost-effective scalability: Data lakes can easily scale to accommodate growing data volumes, making them a cost-effective solution for storing massive datasets.

  • Support for diverse data: Data lakes can handle a wide range of data, including sensor readings, social media feeds, log files, and more.

  • Ideal for exploratory analysis: Data lakes provide a fertile ground for data scientists and analysts to explore data, identify patterns, and generate hypotheses.

• Example use cases:

  • An e-commerce company might store clickstream data, customer reviews, and social media interactions in a data lake for subsequent analysis and personalization efforts.

  • A healthcare organization could use a data lake to store medical images, electronic health records, and genomic data for research and development of AI-driven diagnostic tools.

Data warehouses: structured repositories for analytics

Data warehouses are structured repositories that house processed and curated data, transformed into a consistent format for analysis and reporting purposes. One can build and develop ontologies to organize and provide semantic structure to the data that comes into the system. Ontologies also provide a mechanism to better manage and control model performance by making relationships between data elements explicit.

They excel at facilitating efficient querying and analysis, making them indispensable for business intelligence and decision support applications.

• Key characteristics:

  • Schema-on-write: Data warehouses enforce a predefined schema during data ingestion, ensuring data consistency and integrity.

  • Optimized for querying: Data warehouses employ optimized data structures and indexing techniques to accelerate data retrieval and analysis, enabling faster insights.

  • Support for structured data: Data warehouses are primarily designed for structured data, such as transactional data, customer information, and financial records.

  • Ideal for business intelligence: Data warehouses empower organizations to generate reports, dashboards, and visualizations for informed decision-making.

• Example use cases:

  • A financial institution might use a data warehouse to store transaction data, customer information, and market trends for risk analysis and fraud detection.

  • A manufacturing company could leverage a data warehouse to analyze production data, supply chain metrics, and customer feedback to optimize operations and improve product quality.

The synergy of data lakes and data warehouses

In many AI architectures, data lakes and data warehouses complement each other. Raw data is first ingested into a data lake, where it undergoes cleansing, transformation, and enrichment. The refined data is then transferred to a data warehouse for further analysis and reporting. This synergistic approach enables organizations to leverage the flexibility of data lakes for data exploration and the structure of data warehouses for decision support, creating a robust foundation for data-driven AI applications.

AI on cloud computing: a game-changer for AI

The convergence of AI and cloud computing has opened up a new frontier of possibilities for organizations seeking to leverage the power of AI. Cloud computing provides a scalable, flexible, and cost-effective infrastructure for developing, deploying, and scaling AI applications. By harnessing the capabilities of the cloud, businesses can overcome the limitations of traditional on-premises AI solutions and accelerate innovation.

Benefits of cloud-based AI

Cloud-based AI offers several key advantages that make it an attractive option for organizations of all sizes:

• Scalability: Cloud resources can be easily scaled up or down to meet the fluctuating demands of AI workloads. This elasticity allows organizations to handle large datasets, train complex models, and process vast amounts of data without having to invest in and maintain expensive hardware infrastructure.

• Flexibility: Cloud platforms provide a wide range of AI services and tools, giving organizations the flexibility to choose the best options for their specific needs. This allows businesses to experiment with different AI approaches, quickly iterate on models, and adapt to changing requirements.

• Cost-efficiency: Cloud-based AI can be more cost-effective than on-premises solutions. Organizations only pay for the resources they consume, eliminating the need for upfront capital investments in hardware and software. Additionally, cloud providers often offer pay-as-you-go pricing models, which can further reduce costs.

By leveraging the power of cloud-based AI, organizations can unlock new levels of innovation, efficiency, and competitiveness.

Major cloud AI platforms: accelerating innovation with comprehensive toolsets

Major cloud providers have emerged as key players in the AI landscape, offering comprehensive suites of AI services and tools that cater to a wide range of needs. These platforms provide a one-stop shop for businesses and developers looking to leverage the power of AI in their applications and workflows.

Key cloud AI platforms

• Google Cloud AI platform (Vertex AI): This unified platform streamlines the entire Machine Learning (ML) lifecycle, from building and training models to deploying and managing them in production. Vertex AI’s AutoML feature simplifies model development for users with limited ML expertise, while the model garden offers a collection of pre-trained models ready for deployment. Vertex AI Pipelines orchestrates complex ML workflows, enabling efficient experimentation and automation.

• Amazon SageMaker: A fully managed service, SageMaker empowers users to build, train, and deploy ML models at scale. It boasts a wide array of built-in algorithms and frameworks, making it accessible to both beginners and experienced practitioners. SageMaker’s scalability and integration with other AWS services make it a popular choice for enterprise-grade AI solutions.

• Amazon Bedrock: This cutting-edge service democratizes access to Foundation Models (FMs) from leading AI start-ups and Amazon itself through a simple API. Bedrock enables developers to harness the power of state-of-the-art generative AI capabilities without having to build and train complex models from scratch.

• Microsoft Azure AI: This platform offers a diverse range of AI services, including pre-built AI models for computer vision, speech recognition, natural language processing, and decision-making. Azure Machine Learning allows users to create and deploy custom AI models, while the platform’s extensive integration with other Azure services makes it a versatile choice for a variety of AI applications.

These cloud AI platforms provide a powerful and accessible way for organizations to incorporate AI into their operations, accelerating innovation and driving business value.

Summary

In this chapter, we have explored the fundamental principles of AI system architecture, establishing a comprehensive framework for understanding the building blocks that power intelligent systems. We examined the core components – data as the lifeblood, algorithmic frameworks that enable learning, model architectures that encapsulate intelligence, and infrastructure that provides computational resources – along with architectural patterns such as microservices that offer modularity and flexibility. Critical design considerations of scalability, performance, reliability, and security were discussed as essential elements for robust AI systems that can grow with increasing demands while remaining resilient and protected.

The landscape of AI deployment continues to evolve rapidly, with cloud-native architectures leveraging containerization, orchestration, and serverless computing to achieve unprecedented efficiency. The synergy between data lakes, data warehouses, and data catalogs creates a solid foundation for data-driven intelligence, while major cloud platforms democratize access to sophisticated AI capabilities. As we move forward, these foundational principles will guide the development of AI systems that are not only powerful but also scalable, reliable, and secure – enabling the next generation of innovations across industries.

Relevant reading

• Baheti, Radhakisan, and Helen Gill. “Cyber-Physical Systems.” The Impact of Control Technology, edited by Tariq Samad and Anuradha M. Annaswamy, IEEE Control Systems Society, 2011, pp. 161–66.

• Bass, Len, Paul Clements, and Rick Kazman. Software Architecture in Practice. 3rd ed., Addison-Wesley, 2012.

• Hazelwood, Kim, et al. “Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective.” 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA), IEEE, 2018, pp. 620–629.

• LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep Learning.” Nature, vol. 521, no. 7553, 2015, pp. 436–44, doi.org/10.1038/nature14539.

• Mao, Hongzi, et al. “Resource Management with Deep Reinforcement Learning.” Proceedings of the 15th ACM Workshop on Hot Topics in Networks (HotNets-XV), ACM, 2016, pp. 50–56, people.csail.mit.edu/alizadeh/papers/deeprm-hotnets16.pdf.

• National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST, 2023.

• Patterson, David, et al. “Carbon Emissions and Large Neural Network Training.” arXiv, 2021, arxiv.org/abs/2104.10350.

• Sculley, D., et al. “Hidden Technical Debt in Machine Learning Systems.” Advances in Neural Information Processing Systems, vol. 28, 2015, pp. 2503–2511.

To go deeper on designing robust, scalable AI-enabled systems—from integrating machine learning into existing architectures to managing risks like underperformance, cost overruns, and operational complexity—check out Architecting AI Software Systems by Richard D Avila and Imran Ahmad (Packt, 2025). Through a structured progression of architectural concepts, real-world case studies, and hands-on exercises (including a fictional AI-enabled system you can dissect end to end), it shows software and systems architects, CTOs, VPs of Engineering, AI/ML engineers, and developers how to select the right models and data pipelines, use architectural models to ensure cohesion, simulate and optimize AI performance through iteration, and apply patterns and heuristics to integrate AI into large-scale systems with strong user experience and performance—so you can confidently architect AI-driven products across a range of domains.

Here’s what some readers have said:

In my previous post, The Memory Illusion, I demonstrated that LLM memory doesn’t require vector databases or sophisticated architectures. It’s fundamentally just text management. We built a proof-of-concept in ~150 lines of Python that stored memories in a simple markdown file. It worked. But it had an amusing limitation: The user had to remember to tell the AI to remember.

The system required explicit commands. Want the LLM to store your name? Type !remember “My name is Alex”. Want it to know your project preferences? Another !remember command. The irony was sharp: we’d outsourced memory to technology, only to burden ourselves with managing that memory manually.

This wasn’t an oversight. It was a conscious design decision. Our application code controlled every memory operation through explicit if/else logic. The host app was the memory manager, and the LLM was simply our text processor.

But what if we handed that authority to the LLM itself?

This isn’t an incremental improvement or “v2” of the same approach. It’s a fundamentally different philosophy: trusting the LLM to autonomously manage its own memory. The technical implications are profound. We move from programming specific behaviors to setting high-level intentions. We shift from writing parsing logic to defining trust boundaries.

This approach enables the system to handle its own errors, organize information without explicit rules, and maintain its own memory hygiene. All without writing a single if/else statement.

The Explicit Approach: When Code Defines Every Decision

In my original POC, every memory operation required explicit user commands. Here’s what a typical session looked like:

[You]: Hello, I’m working on a React app
[Claude]: Hi! What kind of React app are you building?
[You]: !remember I am building a React e-commerce application
[Claude]: [Memory saved]

Here we see that required use of !remember. Adding to the user’s cognitive load alongside their actual work.

Behind the scenes, our code intercepted every message, parsed for commands, managed file operations, and reconstructed the prompt with memories for each interaction. We were the brain; the LLM was just processing text within our constraints.

This gave us complete control. We defined in code the exact format of the memory file. Want memories timestamped? We coded it. Want them categorized? More code. Every behavior was explicit, predictable, testable.

This is how we’ve built software applications since the inception of the craft. We write the logic, we define the control flow, we handle the edge cases. It’s comfortable, familiar territory. But when working with LLMs, this traditional approach means we’re not fully leveraging what makes them truly powerful: their ability to understand context and make intelligent decisions autonomously.

The LLM’s contextual intelligence sits idle while our code makes every decision. This intelligence was trained on billions of examples of how humans organize and retrieve information.

Most importantly, users had to remember the commands, creating friction in your app’s usability. Edge cases multiplied. The code grew ever larger as we handled more scenarios, more commands, more special cases. We were swimming upstream against the fundamental capabilities of modern LLMs.

The Implicit Approach: LLM as Autonomous Manager

The paradigm shift is what matters. We’re implementing a harness that grants the LLM autonomous authority. While this example uses the Claude Agent SDK, the pattern can be implemented with other SDKs or custom code. The key is delegation of decision-making, not specific tooling.

Let’s look at the skeleton of our memory tool implementation:
Full code found in the companion app

# memory_tool.py 
class LocalFilesystemMemoryTool(BetaAbstractMemoryTool):
    “”“
    The LLM calls these methods autonomously based on context.
    We provide the infrastructure; Claude makes the decisions.
    “”“

    # The hook methods (memory tools) Claude is made aware of:

    @override
    def view(self, command):
        “”“Claude calls this to read memories or list files”“”
        # Validate path, read file/directory, return contents

    @override
    def create(self, command):
        “”“Claude calls this to create new memory files”“”
        # Validate path, write file, log operation

    @override
    def str_replace(self, command):
        “”“Claude calls this to update existing memories”“”
        # Find text, replace it, handle errors

    @override
    def insert(self, command):
        “”“Claude calls this to add lines to memories”“”
        # Insert at specific line number

    @override
    def delete(self, command):
        “”“Claude calls this to remove memories”“”
        # Delete files or directories

    @override
    def rename(self, command):
        “”“Claude calls this to reorganize memories”“”
        # Move or rename files

The SDK provides these hook methods as the interface contract. We implement the file operations; Claude decides when to invoke them. No command parsing required from us.

The system prompt grants authority. Instead of telling Claude what to remember, we grant it authority:

This shift is profound. We’re not programming behaviors anymore. We’re setting intentions and trusting Claude to execute them. The orchestration that once consumed lines of code now happens autonomously, guided by our system prompt.

This hands-off approach of providing just the infrastructure hooks while delegating all decision-making to the LLM may seem minimal. That’s precisely the point.

If you’ve spent years writing deterministic code, this delegation feels uncomfortable. You’re trusting the LLM to make architectural decisions you once controlled. That discomfort is valid. Let’s add some clarity to the real world implications of this approach.

Under the Hood: Autonomous Decisions in Action

Let’s trace what actually happens during a conversation. These examples come from real sessions the companion app’s trace and render diagram features.

Creating New Memory

[You]: “I’m starting a new project. I’d like to build a web app for solo entrepreneurs
       to track their time and projects”

~Claude internally~: Creates new memory at path `project_solo_entrepreneur_app.txt`

[Claude]: “That’s an exciting project! A time and project tracker specifically for solo entrepreneurs ...”

You didn’t ask Claude to remember anything. Claude autonomously decided this information was worth preserving. It even made an architectural decision. It created a new project file rather than appending to a general profile. These are judgment calls the LLM now makes without any explicit rules from us beyond guidance in the system prompt.

Self-Correction

What happens when the LLM makes a mistake? In traditional systems, errors bubble up to users. Here’s how autonomous authority changes that dynamic:

[You]: “Let’s add these MVP features: client profiles, task tracking, time logging”

~Claude internally~: Tries to update `solo_entrepreneur.txt`,
gets FileNotFoundError, checks memory folder, finds correct
filename `project_solo_entrepreneur_app.txt`, successfully updates

[Claude]: “Got it! I’ve noted those MVP features for your project.”

Claude made a mistake with the filename, detected the error, diagnosed the issue, and self-corrected. All autonomously. No error messages to the user, no manual intervention required.

Our code deliberately re-raises exceptions to Claude rather than hiding them. This design turns errors into recovery opportunities. Claude receives the FileNotFoundError, understands what went wrong, and adapts its strategy. This resilience emerges naturally from giving the LLM authority rather than rigid rules.

Context-Aware Retrieval

The LLM decides when to check memories. No rules tell it to look for context on greetings, it just understands that’s useful:

[You]: “hello”

~Claude internally~: Checks memory folder, finds and reads
`user_preferences.txt` containing “Vacationing at the beach
as of 2025-11-03”

[Claude]: “Hi! How are things going at the beach today?”

Claude found relevant information and used it naturally in conversation.

These autonomous behaviors all emerge without us programming them explicitly. They include deciding on storage structure, recovering from errors, and retrieving contextual information.

Code vs. Prompts: Where Control Lives

The shift to implicit memory doesn’t mean abandoning all control. It means being strategic about where that control resides. By examining what stays in code versus what moves to prompts, we can understand the architecture of trust in AI systems.

Hard Boundaries in Code

The memory_tool.py file shows what explicit choices we’re still making in code:

Security Boundaries: The _validate_path() method explicitly ensures all operations stay within the /memories directory. This is hard-coded protection against path traversal attacks. It’s a security boundary we enforce in code, not through prompts.

Logging & Tracing: Every operation is explicitly logged for debugging and audit trails. We know every tool call Claude made to the memory system and what was read or written.

These coded constraints are examples of the guardrails we build to create a safe sandbox within which Claude operates autonomously. They’re guarantees enforced by our infrastructure, not suggestions in a prompt. In a production system you will expand on these guardrails until you have an acceptable risk level for your line of business.

The Power of Hooks: Beyond File Persistence

The SDK’s hook-based architecture (BetaAbstractMemoryTool) enables remarkable flexibility. While our implementation uses a filesystem backend, you could implement these same hooks to:

• Store memories in a PostgreSQL database for multi-user applications

• Use Redis for high-performance, distributed memory systems

• Implement vector embeddings for semantic memory retrieval

• Create hybrid systems that combine multiple storage backends

The LLM doesn’t care about the backend. It just calls the hooks and trusts the implementation to handle the details. This separation of concerns enables powerful architectural flexibility while maintaining the same autonomous decision-making paradigm.

Guidance Through Prompts

Between the hard constraints of code and the full autonomy of the LLM lies prompt-based guidance. These are strong suggestions that shape behavior without guarantees:

Keep your memory folder organized. Update existing files rather
than creating duplicates. Include metadata like dates when relevant.

When the memory folder exceeds 20 files, consolidate related
memories into broader topic files. Archive outdated information
with clear labels.

This achieves implicit memory maintenance without explicit commands such as !compact. The LLM monitors its own memory health and performs housekeeping autonomously.

For more structure in the memory files, we could specify a frontmatter format for all new memories. This creates a sort of pseudo-schema to be leveraged by the LLM and our instrumentation code:

All memory files should start with:
---
created: YYYY-MM-DD
relevance: current|archive
---

This gives Claude a framework without removing its autonomy. It can still make contextual decisions within these guidelines.

Making the Implicit Explicit: Explanatory Mode

For demonstration purposes in the companion chat app, I added explanatory versions of the prompts. These make the implicit decision-making visible for learning:

# Directive in the system prompt

IMPORTANT: You are a teaching tool to help the user understand how you
process and utilize memories. After each natural conversational response,
add a concise summary of the memory actions you took and why.

This produces illuminating results:

[You]: “ok, I think I’ll set a goal to finish the outline today, by noon”

[Claude]: “Excellent goal! Finishing the outline by noon today (2025-11-10) gives ...”

---
Memory Actions Taken:
- Updated article status to reflect current goal: completing the outline by noon today (2025-11-10)
- This helps track progress and provides context if we discuss the article later today or in future sessions

This explanatory mode becomes a powerful tool for understanding the LLM’s approach to memory and tuning your system prompts. It transforms the black box into a glass box, revealing the autonomous reasoning. Use it to validate that your prompt guidance is achieving the intended behaviors, then switch back to normal operation once satisfied.

The beauty of this division is that higher risk critical decisions stay in code where they’re guaranteed, while contextual decisions that benefit from intelligence and flexibility live in prompts. As models improve, the prompt-based behaviors get smarter automatically, while our security boundaries remain firm.

Conclusion: Learning from Scale

There’s a principle in AI research that keeps proving itself true. Systems built on general methods and scaled computation consistently outperform those with hand-crafted rules. Rich Sutton calls this “The Bitter Lesson”. Bitter because it means our clever, specialized solutions inevitably lose to simpler approaches that leverage raw intelligence.

The shift from explicit to implicit memory perfectly illustrates this principle. And it reveals what becomes possible when we stop fighting it.

The Power of Delegated Intelligence

We’ve seen the behaviors that emerge: autonomous memory creation, contextual reorganization, self-healing from errors. These weren’t programmed. They emerged from granting the LLM authority within safe boundaries.

This pattern extends beyond memory. It extends to complex decision-making in your application. Routing requests, organizing data, managing workflows. All can potentially be delegated to intelligence rather than encoded in logic. The infrastructure you build becomes a framework for capabilities you haven’t even imagined yet.

Hybrid model: Codify workflows that need precise control as tools, then let the LLM autonomously decide when and how to use them. This gives you explicit control over critical operations while still leveraging the LLM’s decision-making for orchestration.

As models improve, your system automatically gets better. When the next generation releases, you update one parameter. Your existing infrastructure suddenly makes smarter decisions. No refactoring. No new edge cases. The same hooks you implement today become more capable tomorrow.

Trust but Verify: Your Implementation Philosophy

This shift changes how we architect AI systems. Instead of writing decision trees, we adopt a “trust but verify” philosophy:

• Trust: Grant the AI authority through system prompts

• Verify: Monitor the behaviors that emerge

• Guide: Adjust prompts based on observed patterns

• Iterate: Refine boundaries as models improve

We’re reallocating the effort once spent on explicit control logic to validation and evaluation. Same total engineering effort, fundamentally better product. The interesting work moves from implementing specific behaviors to designing systems that exhibit emergent intelligence while maintaining appropriate guardrails.

The Path Forward

But here’s my challenge to you: run an experiment. The gap between what models can do and what we think they can do is often surprising. Many teams discover their explicit controls were solving problems the LLM could handle autonomously (and often better).

You can test this today. The companion app provides a complete learning laboratory:

• Multiple prompting strategies for testing delegation

• Full session tracing of every memory decision

• Visual sequence diagrams exported from conversations

• Real-time observation of memory folder organization

Clone it. Run it. Watch what emerges. Add your own prompts. Test your assumptions. Even if the model isn’t ready for your use case today, you’ll have the framework when the next version drops.

The original memory post showed that LLM memory is just text management. This exploration reveals a deeper pattern: we’re moving from programming behaviors to orchestrating capabilities. The question isn’t whether to trust AI with decisions. It’s understanding which decisions, with what boundaries, and how to monitor the results.

The tools are ready. The models are capable. The only thing standing between you and implicit memory is running that first experiment.

Want to explore further?

• Original approach: simple_llm_memory_poc

• Implicit approach: implicit-memory-system-poc

• Claude Agent SDK: Documentation

• Memory Tool: Official docs

• Anthropic on Agents: Building Effective Agents

About the Author: Sam Keen publishes at at AlteredCraft. Subscribe for content that dives into the paradigm shifts AI brings to software development in addition to a free weekly roundup of the latest AI Tutorials, Tool, and News relevant to software developers.

Now, scale up this problem. Imagine a massive distributed system where 10GB of data lives on each node, and every bit is replicated across five different nodes for reliability. When a network partition occurs and a node comes back online, is it really practical to sync all 10GB of data across every node, every time? Of course not.

This is the very challenge that engineers at Cassandra, DynamoDB, and even in blockchain technology have faced. How do you quickly identify and fix just the pieces of data that have changed, without wasting massive amounts of time and bandwidth checking everything, everywhere?

The secret lies in two powerful concepts: Merkle trees and anti-entropy protocols.

In this article, we’ll step into an engineer’s shoes — exploring how these data structures and algorithms allow distributed databases like DynamoDB and Cassandra to efficiently detect, compare, and synchronize changes. We’ll break down what Merkle trees and anti-entropy actually are, why they matter, and how you can implement them yourself (with hands-on examples in Golang).

Ready to see how modern distributed systems stay fast and consistent, even at a massive scale? Let’s dive in!

Understanding Merkle Trees

Imagine you’re the head of accounting at a global company with offices in 100 cities worldwide. Your responsibility: keep track of all expenses across every location. Reconciling records for every single office, every time, would be a logistical nightmare — not to mention a massive waste of time and resources.

To solve this, you create a smarter system:

• Instead of checking every office one by one, you appoint an assistant who keeps a running total.

• Now, you only compare your assistant’s master total with your own. If the totals match, you’re in sync. If not, you zero in on where the change happened.

• Taking it further, you group offices by regions (say, Asia-Pacific and USA). Each region reports upward. When there’s a mismatch, you drill down region by region, then country, then city.

This hierarchical, divide-and-narrow approach is precisely how Merkle trees work in distributed systems.

What is a Merkle Tree?

A Merkle tree is a data structure designed for efficient and secure verification of large sets of data. Here’s the basic anatomy:

• Leaf Nodes: Each one holds a hash of actual data — like an office’s transactions.

• Internal (Branch) Nodes: Each combines and hashes its children’s hashes, summarizing everything below.

• Root Node: The master hash — changing if anything underneath changes.

Key insight: Instead of comparing all data, you compare hashes at each level, honing in only where there’s a mismatch. This scales logarithmically, not linearly.

Historical Context:
Devised by Ralph Merkle in 1979, Merkle trees are now central to blockchains, distributed file systems, and database consistency.

How Merkle Trees Work

a. The Hashing Process

• Hashing the Leaves: Pass each data item (file, transaction, etc.) through a hash function (e.g., SHA-256). These are the leaf nodes.

• Building Up the Tree: Pair up the leaf hashes, concatenate and hash again to form parents, repeating until you reach the root.

• Checking for Changes: Any single data change changes its hash and all parents up to the root. Comparing roots lets you instantly check if two datasets match.

b. Example: Tiny Merkle Tree

Suppose you have four data blocks: A, B, C, D.
Step 1: Hash each block.
Step 2: Hash pairs: (A+B), (C+D).
Step 3: Hash those two: (AB + CD) = Root.
If just B changes, only three hashes need recalculating, and you can precisely spot the change.

Real-World Applications of Merkle Trees

• Blockchains: Store all transactions for a block as a Merkle tree. The root in the block header allows anyone to verify a transaction’s inclusion.

• Versioned File Systems (e.g., Git): Every commit is represented as a Merkle root; differences between commits highlight only what’s changed.

• Distributed Databases (Cassandra, DynamoDB): Use Merkle trees for anti-entropy. Only out-of-sync segments are reconciled, not the full dataset.

Merkle trees guarantee tamper-evidence and make large-scale, bandwidth-efficient consistency possible.

What is Anti-Entropy?

Back to our global accounts team: over time, small differences creep into each office’s ledger — network delays, miscommunications, independent corrections. These inconsistencies, or entropy, must be regularly tracked down and fixed.
Anti-entropy is your systematic, efficient approach to reconciling just the differences, not everything.

Anti-Entropy Mechanisms

• Gossip Protocols: Random offices synchronize with each other, gradually spreading updates network-wide.

• Vector Clocks: Track who changed what, and when, to resolve conflicts precisely.

• Merkle Trees: Group and summarize records as hashes; compare just summaries first to find divergences efficiently.

How Merkle Trees Empower Anti-Entropy in Real Systems

This is exactly the strategy used by distributed databases like Cassandra and DynamoDB. Each node in the system summarizes its stored data using a Merkle tree. When it’s time to reconcile — maybe after a network glitch or data loss — nodes quickly compare root hashes, then only dive deeper where discrepancies are found. This process scales effortlessly, allowing massive companies (and databases) to keep their records in harmony with minimal effort.

So, instead of drowning in a sea of spreadsheets, your accounts team uses anti-entropy protocols — and especially Merkle trees — to work smarter, not harder, keeping the entire global ledger consistent, up-to-date, and audit-ready.

In essence:
Anti-entropy, led by the power of Merkle trees, turns the messy challenge of financial reconciliation across a giant organization into a fast, targeted process — ensuring global consistency with a fraction of the work. This is the same magic powering reliable, scale-proof distributed databases today.

Implementing a Merkle Tree in Code

To truly appreciate the power of Merkle trees in distributed systems, let’s walk through a practical implementation in Go. This example captures the fundamental operations: building the tree, generating cryptographic hashes, and efficiently identifying differences between trees.

Building the Merkle Tree

At the core, a Merkle tree organizes data blocks into a hierarchical hash structure. Our implementation breaks down like this:

• We start by dividing data into buckets (or partitions), each containing a fixed number of keys.

• Each bucket’s combined data is hashed, creating a leaf node.

• We then recursively pair leaf nodes, concatenate their hashes, and hash again to form parent nodes.

• This process repeats until a single root node remains, representing a compact summary of the entire dataset.

Here’s how these pieces fit together in code:

// MerkleNode represents a leaf or internal node with its hash and children.
type MerkleNode struct {
    Hash     string
    Left     *MerkleNode
    Right    *MerkleNode
    KeyRange []string  // Only non-empty for leaves, marks keys covered
}

// Tree manages the Merkle tree for a dataset or node partition.
type Tree struct {
    BucketSize int       // Number of keys per leaf bucket
    Root       *MerkleNode
    mu         sync.RWMutex  // Protects concurrent access
}

When you call the Build method with sorted keys and their corresponding data, it hashes data bucket by bucket to create leaves:

func (t *Tree) buildLeaves(keys []string, kvs map[string][]byte) []*MerkleNode {
    var leaves []*MerkleNode
    for i := 0; i < len(keys); i += t.BucketSize {
        end := min(i + t.BucketSize, len(keys))
        bucket := keys[i:end]
        
        // Concatenate keys and their values (hex encoded)
        data := “”
        for _, k := range bucket {
            data += k + “:” + hex.EncodeToString(kvs[k])
        }
        h := sha256.Sum256([]byte(data))
        
        leaves = append(leaves, &MerkleNode{
            Hash: hex.EncodeToString(h[:]),
            KeyRange: bucket,
        })
    }
    return leaves
}

The buildMerkle function then recursively builds parents by hashing pairs of child hashes, gracefully handling odd numbers of nodes:

func buildMerkle(nodes []*MerkleNode) *MerkleNode {
    if len(nodes) == 0 {
        return nil
    }
    if len(nodes) == 1 {
        return nodes[0]
    }
    var parents []*MerkleNode
    for i := 0; i < len(nodes); i += 2 {
        if i+1 < len(nodes) {
            data := nodes[i].Hash + nodes[i+1].Hash
            h := sha256.Sum256([]byte(data))
            parents = append(parents, &MerkleNode{
                Hash: hex.EncodeToString(h[:]),
                Left: nodes[i],
                Right: nodes[i+1],
            })
        } else {
            parents = append(parents, nodes[i])
        }
    }
    return buildMerkle(parents)
}

After building, the single root hash summarizes the whole dataset:

func (t *Tree) RootHash() string {
    t.mu.RLock()
    defer t.mu.RUnlock()
    if t.Root == nil {
        return “”
    }
    return t.Root.Hash
}

Comparing Trees to Identify Differences

One of the biggest benefits of Merkle trees is their ability to efficiently detect exactly where two datasets differ — without scanning everything. This is done by recursively comparing node hashes from the roots downward:

// Diff returns key ranges that differ between two Merkle trees.
func (t *Tree) Diff(other *Tree) ([][]string, error) {
    t.mu.RLock()
    defer t.mu.RUnlock()
    if t.Root == nil || other.Root == nil {
        return nil, errors.New(”cannot diff: tree(s) not built”)
    }
    diffs := make([][]string, 0)
    diffHelper(t.Root, other.Root, &diffs)
    return diffs, nil
}

// Helper to recursively collect differing leaf key ranges.
func diffHelper(a, b *MerkleNode, diffs *[][]string) {
    if a == nil || b == nil {
        return
    }
    if a.Hash == b.Hash {
        return // subtree matches; no difference
    }
    if a.Left == nil && a.Right == nil && b.Left == nil && b.Right == nil {
        // Both leaves differ; record key range
        *diffs = append(*diffs, a.KeyRange)
        return
    }
    if a.Left != nil && b.Left != nil {
        diffHelper(a.Left, b.Left, diffs)
    }
    if a.Right != nil && b.Right != nil {
        diffHelper(a.Right, b.Right, diffs)
    }
}

This targeted approach means two large datasets can reconcile efficiently by syncing only the data within the mismatched key ranges.

Summary

This Go implementation demonstrates how Merkle trees:

• Build a cryptographic hash tree from raw data in buckets to leaves to root.

• Provide a single summary root hash for efficient data verification.

• Enable fast difference detection by recursive hash comparison, isolating only exact data shards that need syncing.

Understanding and implementing these core operations is the first step toward applying Merkle trees for secure, scalable data consistency in real distributed systems.

Using Merkle Trees for Anti-Entropy: Step-by-Step

1. Each node builds its Merkle tree over its dataset.

2. Nodes exchange root hashes. If equal, they’re in sync; if not, mismatches exist.

3. Compare children hashes recursively until you reach mismatched leaves.

4. Sync only the data for mismatched leaves, not the entire dataset.

5. Rebuild trees and repeat as needed. This gives you fast, granular repair.

Performance Considerations

Efficiency:

• Merkle trees transform O(n)O(n) comparisons into O(log⁡n)O(logn), minimizing bandwidth and compute, critical at scale.

Trade-offs:

• Depth & Bucket Size: Small buckets yield deep trees and fine granularity for repairs; large buckets are faster to build but coarser.

• Branching Factor: Binary is simple, but higher factors further reduce tree height.

• Hash Function: Use cryptographic hashes for security, but know they’re slower than simple checksums.

Best Practices:

• Keep keys sorted and buckets consistent.

• Regularly audit cryptographic choice and correctness.

• Avoid over-deep trees or inconsistent bucketing, as these kill performance.

Case Study: Cassandra’s Anti-Entropy Repair

Scenario: A network partition causes one node to lag. When it rejoins, Cassandra must reconcile data — efficiently.

Process:

• Each node builds a Merkle tree for a partition.

• Nodes exchange Merkle roots; if mismatched, they compare subtrees.

• Drill down recursively until mismatching leaves found; sync only those ranges.

Visual Flow:

1. Build trees

2. Exchange and compare roots

3. Drill to mismatched children

4. Sync only affected ranges

Conclusion

Merkle trees are the backbone of fast, scalable, and resilient data synchronization in distributed systems. They enable efficient, targeted anti-entropy repairs, keeping massive databases like Cassandra and DynamoDB consistent with minimal overhead — even in the face of network partitions or data loss.

As data volumes grow and systems span continents, intelligent anti-entropy protocols will only become more crucial. The future? Expect even tighter cryptographic guarantees, hybrid data structures, and smarter reconciliation engines — keeping the world’s distributed data trustworthy and lightning-fast.

Stay Connected!

• 💡 Follow me on LinkedIn: Archit Agarwal

• 🎥 Subscribe to my YouTube: The Exception Handler

• 📬 Sign up for my newsletter: The Weekly Golang Journal

• ✍️ Follow me on Medium: @architagr

• 👨‍💻 Join my subreddit: r/GolangJournal

• 💡 Follow me on Twitter: @architagr

We will cover the following main topics:

• GitHub security features

• Managing access and permissions

• Best practices for repository security

GitHub security features

In this section, we’ll explore the various security features GitHub offers to help you protect your repositories and manage user access effectively.

Two-Factor Authentication (2FA)

Two-factor authentication adds an extra layer of security to your GitHub account beyond just your password. By requiring a second form of verification, it ensures that even if someone gets hold of your password, they won’t be able to access your account without the second factor.

2FA is crucial for securing your account for enhanced security, mitigating against credential theft, protecting against phishing, and complying with security standards in many organizations and projects. Which 2FA methods are configurable on GitHub?

Available 2FA methods

GitHub offers several methods for enabling Two-Factor Authentication (2FA) to enhance the security of your account. Here are the available 2FA methods:

• Time-Based One-Time Password (TOTP) authenticator apps:

  • Use apps such as Google Authenticator, Authy, or Microsoft Authenticator to generate a time-based code

  • Recommended for its reliability and security

• Short Message Service (SMS):

  • Receive a verification code via text message

  • Less secure compared to TOTP apps, but still an option

• Physical security keys:

  • Use hardware devices such as YubiKeys that support FIDO U2F or WebAuthn standards

  • Provide a high level of security by requiring physical possession of the key

• Virtual security keys:

  • Utilize built-in security features of personal devices, such as Windows Hello, Face ID, or Touch ID

  • Convenient and secure, leveraging device-specific authentication

• GitHub Mobile:

  • Use the GitHub Mobile app to authenticate using public-key cryptography

  • Does not rely on TOTP and provides a seamless experience

These methods offer flexibility and varying levels of security, allowing you to choose the one that best fits your needs. For the highest security, it’s recommended to use TOTP apps or physical security keys. Let’s take a look at how to set this up.

Certification tip

The GitHub Foundations exam often includes questions on 2FA setup and best practices. Make sure you understand both how to enable 2FA and how to use recovery methods.

Setting up 2FA on GitHub

To add an extra layer of security to your GitHub account, enable two-factor authentication (2FA) by following these steps:

1. Go to your GitHub individual account settings by clicking on your avatar in the top-right-hand corner and clicking on Settings.

2. In the left sidebar, under Access, click on Password and authentication.

3. If not enabled yet, select Enable two-factor authentication.

4. Under Two-factor authentication, click Add next to the 2FA method of choice.

5. Follow the prompts to set up 2FA using the method selected.

Optionally, you could set your preferred 2FA method if you enrolled in more than one.

That’s it! You’re all set up.

You will also notice Recovery codes under the Recovery options section (this section will appear only if the 2FA method is set).

Recovery codes are essential for regaining access to your GitHub account if you lose access to your 2FA credentials. These codes act as a backup method, allowing you to log in even if you can’t use your primary 2FA method, such as an authentication app or SMS. When you enable 2FA, GitHub provides a set of recovery codes that you should store securely, such as in a password manager or a safe place. If you ever lose access to your 2FA device, you can use one of these recovery codes to regain entry to your account, ensuring you are not permanently locked out.

GitHub gives you 8 recovery codes. Store them securely. You can regenerate these if needed, but old ones will be invalidated.

Branch protection rules

We discussed branch protection rules extensively in Chapter 5, Branching and Merging Strategies. Be sure to read this in preparation for your exam.

Branch protection rules help you enforce certain workflows and requirements before changes can be merged into your protected branches. This ensures that your codebase remains stable and secure. You can configure branch protection rules and, among many other measures, enforce code reviews, ensuring that all changes are reviewed and approved before they are merged.

Certification tip

You’ll need to know how to configure branch protection rules and enable Dependabot alerts for the exam.

Security configurations

GitHub provides various security settings as a collection that you can configure to enhance the security of the repositories in your organization. You can create a customized security configuration from scratch or choose the GitHub-recommended configuration that already comes preset with its settings.

GitHub-recommended security configurations are predefined settings that follow best practices to enhance security, such as enabling Dependabot alerts and secret scanning by default. Custom configurations, on the other hand, allow you to tailor security settings to meet specific needs or requirements of your project or organization, providing flexibility to adjust features such as branch protection rules and access controls.

To manage security settings at the organization level:

1. Go to your organization’s main page on GitHub (remember that this is an organization, not a repo). For more information on how to create an organization, review Lab 2.1 in Chapter 2, Navigating the GitHub Interface.

2. Click on Settings.

3. In the left sidebar, under Security, click Advanced Security to expand.

4. Then click on Configurations.

5. Choose to edit the GitHub-recommended security configuration by clicking on the edit (

   

   ) icon. Alternatively, you can click on New configuration to customize a new one.

6. Configure the security settings as needed.

To manage repository-specific security settings:

1. Navigate to the repository’s main page on GitHub.

2. Click on Settings.

3. In the left sidebar, under Security, click Advanced Security.

4. Enable or configure security features such as Dependabot alerts, secret scanning, and code scanning. The latter two may be missing from your view if the repository is private or internal.

   Secret scanning and code scanning are GitHub Advanced Security features and are only available as a paid subscription for private or internal repositories, or free if your repository is open source, that is, visibility is Public.

5. Optionally, if you wish to configure secret scanning or code scanning, click on General from the left sidebar and scroll down to Danger Zone. Next to Change repository visibility, click on Change visibility and select Change to public. Be sure to follow the instructions.

Certification tip

Questions may come up regarding changing the visibility of a repository from public to private, or vice versa. Be sure to understand the implications of making a repo public. Read more about public repositories in Chapter 2, Navigating the GitHub Interface and Chapter 11, Contributing to Open Source Projects.

Dependabot, secret scanning, and code scanning are all examples of security features that GitHub offers. Some of these features require a paid subscription, some are free only for public repositories, while others are completely free out of the box.

What is Dependabot?

Dependabot is a feature on GitHub that helps keep your project’s dependencies up to date automatically. It works by regularly checking your project’s dependency files (such as package.json, requirements.txt, etc.) for outdated packages and then creating pull requests to update them to the latest versions.

Key features of Dependabot include the following:

• Automated dependency updates: It scans your project and creates pull requests to update dependencies

• Security alerts: It integrates with GitHub’s security features to alert you about vulnerabilities in your dependencies and can automatically fix them

• Customizable configuration: You can configure how often it checks for updates, which dependencies to ignore, and more, using a dependabot.yml file

• Supports multiple languages: Works with JavaScript, Python, Ruby, Java, PHP, and more

Here is an example flow:

1. You enable Dependabot in your GitHub repository.

2. It checks for outdated or vulnerable dependencies.

3. It creates a pull request with the updated version.

4. You review and merge the pull request.

Now let’s talk about managing alerts and vulnerabilities.

Security alerts and vulnerability management

GitHub helps you stay on top of potential security issues with automated alerts and tools to manage vulnerabilities. Let’s examine some of these:

• Dependabot alerts and security updates:

  • Dependabot alerts: Automatically scans your dependencies for known vulnerabilities and notifies you if any are found

  • Dependabot security updates: Automatically generates pull requests to update vulnerable dependencies to secure versions

How can you enable and manage Dependabot alerts?

1. Go to the repository’s main page on GitHub.

2. Click on Settings.

3. In the left sidebar, under Security, click Advanced Security.

4. Under Dependabot alerts, click Enable if not already enabled.

5. Afterward, you can review and manage alerts from the Security tab of the repository.

• Code scanning:

  • CodeQL: A powerful code analysis engine that scans your code for security vulnerabilities and coding errors. It integrates with GitHub Actions to run scans on every push or pull request.

  • Autofix: Uses AI to suggest fixes for detected vulnerabilities, streamlining the remediation process.

• Secret scanning: Detects and alerts you if sensitive information, such as API keys or passwords, is accidentally committed to your repository. This helps prevent unauthorized access and potential security breaches.

• Security overview dashboard: Provides a centralized view of your security alerts and vulnerabilities across all repositories. This dashboard helps you prioritize and manage security issues more effectively.

• Vulnerability management integrations: Integrates with third-party vulnerability management tools to consolidate and prioritize vulnerabilities, automate risk mitigation, and visualize alerts within your existing security posture.

Talking about third-party integrations, GitHub supports receiving Static Analysis Results Interchange Format (SARIF) reports from various third-party security tools. Some of the commonly used tools include the following:

• ESLint: A popular tool for identifying and reporting on patterns found in ECMAScript/JavaScript code

• Bandit: A tool designed to find common security issues in Python code

• Brakeman: A static analysis tool that checks Ruby on Rails applications for security vulnerabilities

• Checkmarx: A comprehensive Static Application Security Testing (SAST) tool

• Fortify: A suite of tools for static and dynamic application security testing

• SonarQube: An open source platform for continuous inspection of code quality

• Veracode: A cloud-based service for static and dynamic application security testing

These tools generate SARIF files that can be uploaded to GitHub, allowing you to view and manage security alerts directly within your repository.

To handle security advisories and alerts, navigate to the Security tab of the repository (you will find this tab on both the organization and the repository levels). Examine the difference between the Security tabs of both levels. You will notice a stark difference in what you see. This is because the security overview at the organization level rolls up all the security advisories across all its repos, whereas the scope of the repo level is limited to only vulnerability findings of that repo.

These tools and features help you maintain a robust security posture by automating the detection and management of vulnerabilities, ensuring your codebase remains secure.

In the next section, we will talk about how permissions and authorization can help in securing your code.

Managing access and permissions

Effective management of access and permissions is crucial for maintaining the security and integrity of your GitHub repositories. This section delves into the various methods GitHub provides to control who has access to your repositories and what they can do.

User roles and permissions

GitHub offers a range of user roles to help you manage access and permissions effectively. Understanding these roles is key to maintaining a secure and organized workflow.

Overview of different user roles

User roles on GitHub come in three tiers: roles at the Enterprise level, Organization level, and Repository level. Here are the default roles at the Enterprise and Organization levels:

• Owner: The owner has full administrative access to the organization and its repositories. This role can manage settings, users, and billing.

• Member: Members have basic access to repositories, typically for contributing code. They can create issues, submit pull requests, and review code.

• Billing manager: Billing managers can manage billing settings such as changing billing plans, managing payment methods, downloading and receiving receipts, or managing sponsorships.

When inviting new collaborators to your organization for the first time, you choose one of these three.

Certification tip

Be prepared to identify role-based use cases. The exam may ask you to match specific user scenarios with appropriate GitHub roles.

Assigning roles to users

To manage access within your organization, you can assign specific roles to members by following these steps:

1. Navigate to your organization’s main page on GitHub.

2. Click on the People tab in the organization’s navigation bar.

3. If the user doesn’t already exist in the organization, you can invite them by clicking on Invite member.

4. Supply the user’s GitHub handle and click on Invite.

5. For an existing member, locate the user you want to assign a role to and click on the ellipsis dropdown next to their name and select Change role….

6. Select the appropriate role and click on Send invitation (for new invitations) or Change role (existing members).

1. Click on Change role…

2. Select the appropriate role (Owner or Member) from the list displayed.

3. Click on Change role.

4. Alternatively, if the user only needs to be a billing manager, click on the Invite a billing manager link at the bottom of the user invitation screen (Step 3), or go to the organization’s settings and select Billing and licensing from the left navigation bar and invite them.

In addition to these three, GitHub provides some more granular roles that help you define granular permissions to what a member can or cannot do at the different levels.

Here is a table of additional pre-defined roles that can be used:

In addition to these predefined roles, you can create custom roles with a select combination of permissions if one of these predefined roles doesn’t exactly fit your needs. This can be done at the Enterprise, Organization, and Repository levels.

The custom role feature is only available on GitHub Enterprise. In addition, the custom role at the Enterprise level is only available for preview as of early 2025. Refer to GitHub Docs for the latest availability.

Now let’s move on to using teams for access control.

Team management

Teams allow you to group users and manage their access to repositories more efficiently. This is particularly useful for larger organizations with multiple projects. A team can either be visible or secret.

Visible teams can be seen and @mentioned by members of the organization, while secret teams can only be seen by their members. This will be specified during team creation. Teams can also be nested, with one team being the parent of another team.

Let’s see how we create teams.

Creating and managing teams

Here’s how to create a team:

1. Go to your organization’s main page on GitHub.

2. Click on Teams in the organization’s navigation bar.

3. Click New team to create a new team.

4. Enter the team name and description, then click Create team.

5. Add members to the team by clicking Add a member and selecting users from the list.

Here are some important use cases for team visibility and notifications:

Assigning repository access to teams

To assign access to repositories to teams, do the following:

1. Navigate to the team’s page on GitHub.

2. Click on the Repositories tab in the team’s navigation bar.

3. Click Add a repository and select the repository you want to grant access to.

4. Set the desired permission level (Read, Triage, Write, or others) for the team. More roles and the ability to create custom roles are available if you have a paid GitHub Enterprise subscription.

Collaborator access control

Collaborators are individuals who are granted access to specific repositories. This is useful for managing external contributors or contractors.

Adding collaborators to repositories:

To give others access to a specific repository without adding them to the entire organization, you can add them as collaborators:

1. Navigate to the repository’s main page on GitHub.

2. Click on Settings in the repository’s navigation bar.

3. In the left sidebar, click Collaborators and teams (org-owned repos) or Collaborators (user-owned repos).

4. Click Add people and start typing the username of the person you want to add.

5. Select their name and set the appropriate permission level (Read, Triage, Write, Maintain, Admin) for the collaborator.

Setting permissions for collaborators:

Once collaborators are added, you can adjust their access levels using the following steps:

1. In the Collaborators and teams section, find the collaborator you want to manage.

2. Click on their permission dropdown and select the desired permission level.

One common pitfall is misconfigured roles or permissions, which can inadvertently expose sensitive code. For example, imagine a scenario where a developer creates a private repository for an internal tool but mistakenly assigns a Read role to an external contractor at the organization level. Because the repository inherits permissions from the organization, the contractor now has unintended access to the private repository. This kind of oversight can lead to accidental data leaks or unauthorized code access.

To prevent such incidents, always follow the principle of least privilege, regularly audit repository access, and use fine-grained personal access tokens for automation and integrations.

Another great feature to manage access and permissions is tokens. Two examples of tokens on GitHub are OAuth and Personal Access Tokens (PAT).

OAuth and personal access tokens

OAuth and personal access tokens provide secure ways to authenticate and authorize access to your GitHub account and repositories. These methods are essential for integrating third-party applications and services, that is, when you are not using an interactive login of a person. Here’s where you’ll find each of these on GitHub:

Managing OAuth applications

To review and manage third-party applications connected to your GitHub account, follow these steps:

1. Go to your GitHub account settings.

2. In the left sidebar, click Developer settings.

3. Click OAuth Apps to view and manage your OAuth applications.

4. Review the list of authorized applications and revoke access if necessary.

Creating and using personal access tokens

To authenticate non-interactive scripts or services, you can generate a personal access token as shown below:

1. Go to your GitHub account settings.

2. In the left sidebar, click Developer settings.

3. Click Personal access tokens.

4. Select either Fine-grained tokens or Tokens (classic) in the submenu.

5. Click Generate new token (for fine-grained tokens) or select Token (classic) again for the classic token option.

6. Select the scopes or permissions you want to grant this token, such as repo, admin:org, or user.

7. Click Generate token and copy the token for use in your applications. Store it securely, as it will not be displayed again.

You would notice by now that there are two types of personal access tokens on GitHub: classic and fine-grained.

Both types co-exist, with classic being the older. It is expected that GitHub will deprecate classic PAT in favour of fine-grained PAT in the future, but both of them can be used interchangeably today. Let’s quickly enumerate the differences between the two.

In summary, GitHub offers a range of measures to manage access and grant permissions to users and third-party applications and services. It is important to know which ones to combine to ensure the security of your code. Up next, let’s consider some good security practices for repo security.

Best practices for repository security

Ensuring the security of your repositories is paramount to protecting your code and maintaining the integrity of your projects. In this section, we’ll cover best practices for securing your repositories on GitHub.

Code scanning with Static Application Security Testing (SAST) tools

Static Application Security Testing (SAST) tools help identify security vulnerabilities in your codebase before they become issues in production. GitHub has a code scanning product. It is sold separately but can be integrated natively into your repos and organizations. It’s labelled GitHub Advanced Security (GHAS). GHAS is a paid subscription product, an add-on that you can purchase in addition to your GitHub subscription. If your codebase is open source, most of GHAS’s security features are free for use.

Here’s how to integrate and use these tools. You have to do this on a public repo if you haven’t purchased GHAS:

• Integrating SAST tools in your workflow: Here’s how to set up code scanning with SAST tools:

  1. Navigate to the repository’s main page on GitHub.

  2. Click on Security in the repository’s navigation bar.

  3. Click Set up code scanning.

  4. Choose a code scanning tool, such as CodeQL, and follow the prompts to configure it.

Examples of SAST tools include the following:

• CodeQL: A powerful code analysis engine that scans your code for security vulnerabilities and coding errors

• Dependabot Alerts: Automatically scans your dependencies for known vulnerabilities and notifies you if any are found

• Interpreting scan results: To review and act on scan results, take the following steps:

  1. Go to the Security tab of the repository.

  2. Click on Code scanning alerts to view the results.

  3. Review the alerts and take appropriate action to fix the identified issues.

CI/CD pipeline security measures

Securing your Continuous Integration/Continuous Deployment (CI/CD) pipelines is crucial to ensure that your code remains secure throughout the development lifecycle. Follow these steps to secure your pipelines:

• Securing CI/CD Pipelines Using GitHub Actions: To secure your CI/CD pipelines, take these steps:

  1. Navigate to the repository’s main page on GitHub.

  2. Click on Actions in the repository’s navigation bar.

  3. Set up workflows to include security checks, such as running SAST tools or dependency checks.

Mini-case example

A development team noticed that their CI/CD pipeline was deploying code with outdated dependencies that had known vulnerabilities. The issue stemmed from a missing dependency scanning step in their GitHub Actions workflow. To mitigate this, they integrated Dependabot and added a step in their workflow to run npm audit during each build. This change helped catch vulnerable packages early and prevented insecure code from reaching production.

• Implementing secrets management in workflows: To manage secrets securely in your workflows, take these steps:

  1. Go to the repository’s main page on GitHub.

  2. Click on Settings.

  3. In the left sidebar, click Secrets and variables.

  4. Then, select Actions from the submenu.

  5. Click New repository secret to add secrets, such as API keys or tokens, securely.

  6. Reference these secrets in your GitHub Actions workflows to avoid exposing sensitive information.

Monitoring and auditing activities

Regular monitoring and auditing of repository activities help you detect and respond to suspicious actions promptly. Here’s how to monitor and audit activities:

• Using audit logs to monitor repository activities: To monitor repository activities, take these steps:

  1. Go to your organization’s main page on GitHub.

  2. Click on Settings.

  3. In the left sidebar, under the Archive section, click Logs,

  4. Then, select Audit log from the submenu.

  5. Review the audit log entries to monitor activities such as user logins, repository changes, and permission updates.

• Setting up alerts for suspicious activities: To set up alerts for suspicious activities, take these steps:

  1. Use GitHub’s built-in security alerts to notify you of potential security issues.

  2. Integrate with third-party monitoring tools to receive real-time alerts for suspicious activities.

Incident response and recovery

Being prepared for security incidents and having a plan for recovery is essential for minimizing the impact of security breaches. Follow these steps for incident response and recovery:

• Preparing for security incidents: To prepare for security incidents, take these steps:

  1. Develop an incident response plan that outlines the steps to take in case of a security breach.

  2. Ensure that all team members are aware of the plan and their roles in the response process.

• Steps for incident response and recovery: To respond to and recover from security incidents, take these steps:

  1. Identify and contain the breach to prevent further damage.

  2. Investigate the cause of the breach and assess the impact.

  3. Remediate the vulnerabilities that led to the breach.

  4. Communicate with stakeholders and provide updates on the incident and recovery efforts.

  5. Review and update security policies and practices to prevent future incidents.

Et voila! This concludes the basics when it comes to security on GitHub. Let’s summarize what we learned.

Summary

In this chapter, we delved into the intricacies of security practices and user management on GitHub. We had already mastered the essentials of effective collaboration, but we know that with great code comes great responsibility. We explored the robust security features GitHub offers, such as two-factor authentication, which added an extra layer of security to our accounts. We learned about the various methods available for 2FA, including authenticator apps, SMS, physical security keys, and GitHub Mobile.

We also discussed branch protection rules, which ensured our codebase remained stable and secure by enforcing workflows and requirements before changes could be merged. Additionally, we examined security configurations, both GitHub-recommended and custom, to enhance the security of our repositories. We looked at the management of security alerts and vulnerabilities through tools such as Dependabot, CodeQL, and secret scanning, which helped us stay on top of potential security issues.

Managing access and permissions was another crucial aspect we covered. We understood the importance of user roles and permissions at different levels, from enterprise to repository, and how to assign these roles effectively. We also explored the use of teams for access control, creating and managing teams to streamline our workflow. Finally, we looked at OAuth and personal access tokens, which provided secure ways to authenticate and authorize access to our GitHub account and repositories.

Overall, this chapter equipped us with the knowledge and tools to maintain a robust security posture and manage user access effectively on GitHub.

Let’s do a short quiz.

Test your knowledge

Review all GitHub security and user management features, especially permission models, 2FA, and CI/CD hardening techniques – they appear frequently on the certification.

1. Which of the following methods is considered the most secure for enabling Two-Factor Authentication (2FA) on GitHub?

   1. Short Message Service (SMS)

   2. Time-Based One-Time Password (TOTP) authenticator apps

   3. Virtual security keys

   4. Physical security keys

2. What is the primary purpose of Dependabot Security Updates in GitHub?

   1. To scan your code for security vulnerabilities and coding errors

   2. To automatically generate pull requests to update vulnerable dependencies to secure versions

   3. To detect and alert you if sensitive information is accidentally committed to your repository

   4. To provide a centralized view of your security alerts and vulnerabilities across all repositories

3. Which role in GitHub is best suited to managing security policies, security alerts, and security configurations for an organization and all its repositories?

   1. Owner

   2. Security manager

   3. Admin

   4. CI/CD admin

Useful links

• Authentication documentation: https://docs.github.com/en/enterprise-cloud@latest/authentication

• About GitHub security features: https://docs.github.com/en/enterprise-cloud@latest/code-security/getting-started/github-security-features#about-githubs-security-features

• About GitHub Advanced Security: https://docs.github.com/en/enterprise-cloud@latest/get-started/learning-about-github/about-github-advanced-security

To build practical mastery of Git and GitHub— from version control basics to collaborative workflows, secure automation, and AI-assisted productivity—check out GitHub Foundations Certification Guide by Ayodeji Ayodele (Packt, 2025). Through step-by-step labs, real-world projects, and exam strategies, it helps you prepare for the GitHub Foundations certification while adopting best practices for issues and pull requests, GitHub Projects, privacy and security controls, and GitHub Copilot—so you can level up your skills and ship better software, faster.

Here’s what some readers have said:

The concept of Application Development Framework (ADF) has been well-known for a long time, but we need to set up a crystal-clear context for further reading. This is important because it helps us deal with this complex topic by setting common ground for definitions and classifications that will be used throughout the book. First, we explore the evolution of the idea of ADF. After that, we discover the differences and connections between other SDLC-focused technologies, such as Platforms, Libraries, SDKs, and APIs, to craft a brief but concise definition that helps us keep a big picture while diving deep into implementation topics. Then, we review the place of ADF in the Software Development Lifecycle to identify and prove the advantages of adopting an ADF.

In this chapter we’re going to cover the following main topics:

• Introduction and historical references

• Breaking down Application Development Framework

• Exploring ADF and Platforms, Libraries, SDKs, APIs

• Integrating into Software Development Lifecycle (SDLC) and Flow

• Differentiating ADF and other types of Frameworks

Introduction and historical references

Engineers have a long and productive history of creating building blocks for their own convenience. If we do not ignore this historical experience, we can learn many useful lessons for creating our own frameworks.

From the very beginning of the software industry, engineers and scientists have had a tendency to reuse their most successful and efficient ideas. There are quite a few historical practices that share the core objectives of a framework:

• Architectural Blueprints: Since ancient times, complex structures like buildings or ships were built based on detailed plans. These plans defined the overall structure, components, and relationships - similar to how frameworks provide a blueprint for software architecture.

• Modular Design in Engineering: Even before the computer age, engineers approached complex machines with a modular mindset. Think of early steam engines with interchangeable parts - a principle that carries over to software components within a framework.

• Mathematical Frameworks: For centuries, mathematicians have relied on established frameworks like algebra or calculus to solve problems. These frameworks provide a set of rules and structures that guide the approach to solving a specific type of problem.

While these aren’t direct equivalents to software frameworks, they all represent historical approaches to structuring complex systems in a way that aligns with the core function of a software development framework—to simplify the lives of its users when dealing with complex problems.

With the advent and adoption of computers, the concept of a framework has gone beyond the art of the elite and has become part of the daily work of many programmers. The idea evolved as computers themselves developed. Here are some contenders for the title of earliest software framework:

• Early Subroutine Libraries (1940s – 1950s): In the early days of computing, programmers might develop reusable code blocks for common tasks like mathematical functions or input/output routines. These weren’t full-fledged frameworks, but they offered a basic level of reusability and structure.

• FORTRAN Compilers (1950s): FORTRAN introduced the concept of high-level languages, allowing programmers to write code that is more human-readable than machine code. While not exactly a framework, it provided a foundational structure for building software.

• Operating Systems (1960s onwards): Operating systems like IBM’s OS/360 offered a platform for running applications. They provided core functionalities like memory management and device drivers, which later frameworks were built upon.

It’s important to remember that the concept of a software development framework as we know it today – offering a comprehensive set of tools, libraries, and design patterns – is a more recent development. However, these earlier practices laid the groundwork for the frameworks we use today. In the modern world, we can only imagine the practical purpose software created on top of one or multiple frameworks.

Note

This book uses both terms “software development framework (SDF)” and “application development framework (ADF)” interchangeably. Usually, “application” is not exactly the same as “software”: we have platforms, libraries, SDKs, engineering tools and frameworks as alternative kinds of software. But in the context of the topic (“building frameworks”) we can always safely assume that any “software” we are going to develop with our frameworks will serve the same purpose as “application” with a minor exception of “infrastructure management frameworks” which are mentioned explicitly.

While most of the information on the internet about Application Development Frameworks is focused on web and mobile development, we cannot ignore trending frameworks from a non-application software, such as

• Artificial Intelligence and Machine Learning (like PyTorch, TensorFlow, and Apache MXNet),

• Scheduled task management (like Celery, Temporal, and Apache Airflow),

• Infrastructure management (like Terraform, Pulumi, and Crossplane),

• Testing automation (like Selenium, Robot, and webdriverIO),

and many others, including “hybrid” frameworks that provide multiple capabilities at once.

Fortunately, foundational principles of building software frameworks are common between classic ADF and these emerging types of SDF.

There are also vertical ADFs, aiming to cover corresponding business domains. Examples of such frameworks include Gamedev (Flame, Monogame), Data Visualization (Shiny, Seaborn, TensorBoard), Hardware Instrumentation (LabVIEW), etc.

A StackOverflow research in 2023 that involved approximately 90,000 software engineers provided us with data about frameworks they use daily (See Figure 1.1 that summarizes one of the framework-related topics from this survey). In addition to those numbers, we know that many frameworks have their own communities outside of StackOverflow, which means that more than 100k engineers work with software frameworks on a daily basis.

Figure 1.1: StackOverflow research summary chart

With all the wide adoption of ADFs, it is confusing to see how many different inconsistent definitions and classifications we have all around the internet. In the following section, we will craft a brief and concise definition based on ADFs unique differentiators in a world of software engineering.

Breaking down Application Development Framework

I often see engineers mixing up libraries, frameworks, and Software Development Kits. They use these words like they mean the same thing. Even more confusion can come if we add APIs, platforms, and DSLs to the conversation. But to make our own framework, we need to understand all these things.

According to Dictionary.com, the formal definition of the term “framework” is “a basic structure, plan, or system, as of concepts, values, customs, or rules”.

The collective unconscious of humanity, also known as LLM, suggests the following definition for ADF: An application development framework is a software library offering a fundamental structure for building applications within a specific environment. It acts as a reusable foundation, supplying pre-defined functionalities and promoting code organization through established conventions. This approach streamlines development by reducing repetitive coding efforts.

Both definitions are formally correct (except for “library” part of the second one, which I will explain later in this section). But they focus on how the framework is designed but have a lack of explanation about how it works. I am going to fill this gap by adding my own:

Definition

A collection of pre-written code and tools that provide a structured approach to building applications. It simplifies development by enforcing architectural patterns: frameworks always dictate an execution flow, and stipulate specific way to structure your code, promoting maintainability, testability, low coupling, and reusability.

It is a common misunderstanding to confuse frameworks with other engineering concepts aiming towards reusability, like API, software library, SDK, and platform. And there is always a special “tooling” category, which covers a wide range of software from smaller console scripts to powerful configurable logs processing pipelines – they all live their own life as they only used by developers to support their routine tasks, so we keep them out of conversation. Let’s set clear boundaries to understand their differences to focus on the most important aspects of our topic.

Application Programming Interface

Starting from API (Application Programming Interface) as the lowest-level implementation of the development tooling. The traditional understanding of API included any exposed interface available to software developers to perform manipulation with an external subsystem. This external subsystem was treated as a “black box,” which means that the developer should not worry about its internal implementation, tech stack, and logic. Thus, API provides a complete set of methods to deal with it. Modern understanding of the API concept drifted towards over-the-network API, like HTTP/gRPC/websocket APIs. Events and message-based communication interfaces are also subsets of APIs – like webhooks.

The best practice of API definition is to use open standards like OpenAPI and AsyncAPI schema languages, or other less popular languages like RAML.org or APIBlueprint.org. However, it is acceptable to use proprietary or vendor-specific tools. API concept can also be visualized with a simple diagram notation (see Figure 1.2 below). Typical representatives of the API are as follows:

• SaaS products’ interfaces, e.g. PandaDoc API, or OpenAI LLM API

• Cloud management APIs; e.g. AWS Cloud Control API , or Azure REST API, or Google’s Service Management API

• Webhooks, like Zapier

• Internal/proprietary messaging-based events and commands schema registries that can be based on Confluent or Redpanda

Figure 1.2: Concept-level diagram of API

The diagram helps to see that the API purpose is to provide access to exposed “black box” functions.

Library

The next one to review is a software library as a collection of pre-written code or routines that developers can use to perform specific tasks or functions within their software applications.

Sometimes developers see any software library as a framework, but the purpose of the library is completely different – it focuses on runtime/operation concerns by implementing a common part of the system, like hardware I/O operations, network protocol, authorization sequence, ranking algorithm, IoT standard, etc. It is also common to have a library to transform a low-level API into a more developer-friendly form by adding enumerables, constants, and conditional logic over a binary code and method signatures of plain API. A Library usually operates as a gray box, which means that software developers can see its internal implementation, but it is rarely necessary. In some cases, libraries can come in binary format, which makes them black boxes.

Terminology across the industry is not always consistent, we can find other synonyms for the term “software library”:

• Package; usually means one or multiple software libraries that share the same license and they can be distributed as a single unit.

• Module; usually means a built-in software library, distributed with the program.

• Extension (aka add-on or plug-in); usually means a software library that follows specific program interface allowing external developers to modify original program behavior without changing any code in the original system.

Another important consideration is control flow. For a software library, it is common for developers to have full control over the library functions – so developers are responsible for invoking the library.

Writing software libraries is one of the most common tasks in the industry; many senior developers have experience of creating libs for internal company purposes, or contributing to open-source libs, or at least have them as a part of their pet projects.

Often, a software library evolves into an SDK or a framework after multiple iterations of improvements. And we definitely need to build libraries as part of the ADF development.

To understand the idea of software library better, we can use real examples:

• Algorithm libraries like math, or 3D, or ML

• Hardware abstraction libraries like HAL or device drivers

• Standard-compliant implementations like OpenAuth

• Programming helpers like Boost or Requests.py that provide developers with a pre-written code for HTTP requests lifecycle syntaxis helper.

• Any proprietary pluggable reusable code

Figure 1.3: Concept-level diagram of API and Library

The diagram suggests that Library can serve as a pre-implemented tool to integrate an API to the application; but basically, it can provide any pre-implemented code for reuse.

Software Development Kit

Similar functions can be also performed by Software Development Kits (SDKs) but they usually include much more than just a software Libraries; there is a list of possible SDK internals:

• Libraries.

• Tracing and Debugging tools.

• Documentation.

• Integrated development environments (IDEs)

• Tests.

• Plug-ins.

• Application programming interfaces (APIs)

• Sample code.

SDKs span both design-time (organizational) and runtime/operations (product) concerns but still with focus on a runtime. SDKs are also platform– or vendor-specific, they are developed by API or Platform vendors to improve their products adoption – see Android SDK created by Google (Alphabet) and Windows ASDK developed by Microsoft.

Sometimes bigger SDKs can be designed to include frameworks (like Apple SDK), but we can also see the opposite case, where an SDK is designed as an element of the framework. The following are examples of cases where SDKs are subsystems of ADFs in the list below:

• Operator SDK is part of the Operator Framework

• SDKs as developer-friendly lib wrappers for a particular framework, (for example) Treblle provides multiple SDKs including one for the Django framework

• SDKs have the same control flow as libraries have: the developer is responsible for invoking an SDK, while in the case of frameworks, we usually have the opposite control flow: the framework is responsible for invoking developer’s code. There is no exception if the framework is part of an SDK: the framework takes ownership over the control flow.

Figure 1.4: Concept diagram of API, Library, and SDK

The diagram above depicts the SDK as a super-entity for a library.

Framework

Let’s elaborate on our definition here:

A framework is a collection of pre-written code and tools that provides a structured approach to building applications. It simplifies development by enforcing architectural patterns: frameworks always dictate an execution flow, and stipulate specific way to structure your code, promoting maintainability, testability, low coupling, and reusability.

Of course, frameworks provide more than that, there are some examples below:

• Hide low-level complexity behind a higher-level abstraction;

• Promote faster development by providing pre-built binary/packaged components and functionalities;

But those additional benefits cannot be attributed exclusively to frameworks – libraries or SDKs both have the same value propositions.

The following are the Key aspects of this definition based on usage scenarios and key attributes:

• “Framework as abstraction” conceals repetitive code and low-level details by applying software libraries, acting as a higher-level interface. This allows developers to work with core functionalities without getting bogged down in implementation specifics. However, it usually gives an option of direct communication with levels under even if it is unnecessarily for overwhelming majority of scenarios; in brief, any framework has one or multiple libraries coming as a built-in option or pluggable 3rd-parties.

• “Frameworks as tooling” prioritize simplifying the development process by providing pre-built components, streamlined workflows, and reduced boilerplate code. Their primary focus is on accelerating development, while runtime considerations (like operations, maintenance, portability, performance) are a secondary benefit.

• “Framework as architectural constraint” establishes a blueprint for system architecture. It dictates core components, their interactions, and overall structure, influencing key design decisions for developers working within the framework’s constraints.

And finally, the relations between a frameworks and APIs, libraries and SDKs are usually follow the common pattern: ADF streamlines the software development flow for the organization, governs the control flow by invoking a custom code made by software developer, having libraries as a proxy to access external subsystem APIs, and allowing to plug in a third-party libraries or SDKs to handle specific integrations.

Figure 1.5: Concept diagram of ADF, SDK, Library, and API

There is an extended classification of the Application Development Frameworks:

• Web frameworks like Django, Node.js, Java Play, Ruby-on-Rails, etc

• Enterprise frameworks like Java Spring, Oracle ADF

• Low-code frameworks like Flutter, OpenBlocks, Appsmith

• AI/ML frameworks like TensorFlow, Keras, Apache MXNet

• Gamedev frameworks like Unity, UnrealEngine

• Mobile frameworks like React Native, Xamarin, Apache Cordova

• Microservice frameworks like GoMicro, Spring Boot, Molecular

• Test automation frameworks like Selenium, Appium, WebdriverIO

• Desktop OS frameworks like MFC, OS X framework, KDE framework

• Utility frameworks like Python Celery, ActiveTask,

• Custom frameworks – proprietary ones build for internal use, usually applying ADF format for a domain objects and rules

• As Frameworks are our focus area, we add more detailed specification for three ADFs to better highlight their commonalities.

The first one to analyze is Django:

• It employs numerous libraries in pluggable way

• It focuses on design-time aspects: “encourages rapid development and clean, pragmatic design”

• It is responsible for a control flow – developers don’t need to invoke Django code but to follow a Django project structure to get their code invoked in a right moment;

• It enforces multiple architecture patterns (MVC / MVT as a model-view-template, ORM as object-relational mapping, extendable middleware-based request processing pipeline, class-based views, etc)

• It is vendor- and platform-neutral so it can be used on any cloud platform or a virtual machine that can interpret Python programming language;

• It is a “gray box” software that can be redistributed as a package but it also has its source code published in a public GitHub repository under the BSD-3 OSS license

The second one is Node.js, the most popular web full-stack framework based on the JavaScript programming language:

• It has number of standard built-in libraries listed in official documentation and hundreds of pluggable external libraries like listed here

• Org design-time focus is clearly emphasized as a key success factor of this framework; …

  (As Ryan Dahl said): “So for kind of technical reasons, adding a server onto JavaScript worked really well and people who were programming front end websites were able to take those same skills and with just a small amount of additional knowledge were able to program pretty nice web servers that could do long polling or other kinds of real time interactions. And I think there’s just a large base of JavaScript users out there, naturally, it being the language of the web, and so there was a lot of people who were able to take their skills and add on Node to that and suddenly become full stack developers.”

• It is responsible for the control flow

• It enforces architecture patterns like event-driven, microservices, API-first etc.

• It is vendor- and platform-neutral

• It is a “gray box” open-source software

And the final one is React, a modern web front-end framework based on the JavaScript language:

• Dozens of libraries like here

• Design-time focus: most of public sources mention developer-oriented benefits as a key advantage of the framework; this list includes declarative syntaxis, reusable components, community support, detailed documentation, etc.

• Control flow management based on a virtual DOM concept is the core of the framework

• React’s intrinsic architecture patterns include event-driven (hooks), container-based decorators, data repository (provider), etc.

• It is vendor- and platform-neutral

• It is a “gray box” open-source software

As we can see, most ADFs follow the same model which we will explore in more detail in Chapter 3.

Platform

And the final concept to review here is a Platform. The main differentiator of a Platform, in comparison with all the others: APIs, libraries, SDKs and frameworks, is its hosted server-side execution runtime. But it is important to understand that the runtime concern is a “bonus value” here because the fundamental benefits of Platforms are still organizational design-time toolings. Platforms usually combine that hosted execution backend with APIs, libraries, and SDKs; and it is common for modern Platforms to provide developers with more advanced tooling like dev portals, resource management console and UI, cloud IDE, infrastructure-as-a-code definitions support and many other org productivity boosters. However, platforms rarely include application development frameworks and vice versa. We still can see frameworks being part of the platform (like AWS Well-Architected Framework is part of AWS platform value proposition, and Microsoft also has the same one), but these are not ADFs but architecture design frameworks (set of values, viewpoints, blueprints and patterns for cloud-native applications).

Typical taxonomy with examples of the Platforms is the following:

• Cloud platforms like AWS, MS Azure or GCP

• Messaging platforms like Confluent Kafka or Redpanda

• Task execution platforms like Temporal.io

• Robotic process automation (RPA) platforms like WorkFusion or UIPath

• Game Platforms like Heroic Cloud

• Dev platforms like Split.io, Firebase, Launchdarkly

Figure 1.6: Concept diagram of Platform, Framework, SDK, Library, and API

• Platforms are the most complete and mature kind of reusable engineering elements; however, their value comes with a high cost of ownership – this means that only large-scale companies can afford to build and operate an internal software development platform. Please do not confuse internal SDP with commercial PaaS (platform as a service) – we have many examples that successful PaaS can be created and operated with relatively small investment.

Domain Specific Language

Here it is, the bonus addition to the chapter. A Domain Specific Language (DSL) is a “language” with a higher level of abstraction optimized for a specific class of problems. A DSL uses the concepts and rules from the field or domain.

Note

Please note that DSL is not a programming language but rather a “formal domain description” language. In most cases, DSLs implementations are closer to executable configuration in JSON, YAML, XML or similar formats.

In terms of architecture abstractions, extensibility, and control flow, DSL is very similar to ADF – they both imply certain design patterns, employ libraries for extensibility and portability purposes, and invoke necessary code in the right time, defined by DSL creators.

The difference is the level of freedom for software product developers to code the DSL execution. We may consider DSL a “next level” framework suitable for cases when we want to achieve a high grade of standardization at a high level of abstraction.

Typical simplified taxonomy of the DSLs is the following:

• Workflow/BPMN like Camunda, Oracle BPMS, Nikku

• Rules like Drools

• Infra like Terraform

• Development like Gradle

Figure 1.7: Concept diagram with DSL, Platform, Framework, SDK, Library, and API

The diagram demonstrates a DSL primary use case of extending a Framework with limited predefined capabilities.

In this section we focused on key differentiators of Frameworks in comparison with other kinds of “engineering building blocks”. Now we can concentrate on the most important aspects of Frameworks, including the ones we further discuss in the next chapters.

Differentiating ADF and other Types of Frameworks

There are even more sources of confusion: we have Frameworks that could be used in the process of developing software products, but these Frameworks are not ADFs!

In the realm of software development, framework is a broad term encompassing various tools that structure and streamline different aspects of the process. We’ve established that Application Development Frameworks (ADFs) directly assist with coding. Let’s delve deeper into two other crucial categories: Architecture Frameworks and Software Delivery Frameworks.

…

Architecture Frameworks

These frameworks provide a structured approach to designing the overall architecture of a software system. Think of them as organizational architecture design process blueprints or roadmaps that define the foundation upon which your application will be built.

Here’s a breakdown of Architecture Frameworks’ types:

Enterprise Architecture Frameworks (EAFs): Focuses on the high-level structure of an entire organization’s IT infrastructure, including software applications, data, and hardware. Examples: TOGAF, Zachman Framework.

Software Architecture Frameworks (SAFs): Specializes in designing the internal structure of a single software application. Examples: 4+1 View Model, C4 Model.

The following are some of the benefits of adopting such frameworks:

• Consistency: Promotes a standardized approach to design, ensuring all components fit together seamlessly.

• Communication: Provide a common language for stakeholders (architects, developers, etc.) to discuss system design.

• Reduced Complexity: Break down complex systems into manageable components, simplifying design and development.

The following are some of the popular examples of the architecture frameworks:

• TOGAF (The Open Group Architecture Framework): A widely used EAF known for its comprehensive approach to enterprise architecture.

• Zachman Framework: Another EAF, offering a framework for classifying architectural information across different viewpoints (e.g., business, data, application).

• 4+1 View Model: A SAF focusing on five architectural viewpoints (system, application, deployment, container, and code) for designing software applications.

(Another example is DoDAF)

Architecture Frameworks can be very useful, but they are completely out of scope of this book.

Software Delivery Frameworks

These frameworks focus on streamlining the entire software development and delivery process, particularly for large-scale or complex projects. They don’t deal with the specifics of coding or designing the application itself, but rather how to efficiently manage the development lifecycle.

Here’s a closer look at Software Delivery Frameworks:

Core Principles: Emphasize iterative development, continuous integration and continuous delivery (CI/CD), and agile methodologies.

The following are some of the benefits:

• Improved Efficiency: Streamlines workflows and processes to deliver software faster and with fewer errors.

• Enhanced Communication: Fosters collaboration between development teams, product managers, and stakeholders.

• Increased Adaptability: Enables teams to respond to changing requirements and market needs more effectively.

The following are some of the popular examples:

• Scaled Agile Framework (SAFe): A popular framework for scaling agile methodologies to large enterprises.

• Large Scale Scrum (LeSS): An adaptation of the Scrum framework designed for large teams working on complex projects.

• Disciplined Agile Delivery (DAD): A framework that integrates various agile practices with other project management methodologies.

These categories provide a glimpse into the diverse landscape of frameworks beyond ADFs. Architecture Frameworks ensure a well-designed foundation for your software system, while Software Delivery Frameworks guide the overall development journey with efficiency and agility. By understanding and leveraging these frameworks, you can build robust and successful software applications.

But as we decided to focus on ADF, we need to pay special attention to a Software Development Lifecycle topic as it covers the key value of using application development frameworks.

Software Development Lifecycle (SDLC) and Flow

To better understand Application Development Frameworks, we need first to understand key scenarios of ADF adoption.

To set up a context, we need to differentiate the design-time and operations (runtime) aspects of the software product development. It is important to note that the Software Development Lifecycle (SDLC) can be used as a “blueprint” for software product development iteration. Visual diagram of SDLC can help us better understand this aspect:

Figure 1.8: Spiral SDLC model is one of the most advanced SDLC models for software development: image credits to tutorialspoint

We should always assume a repeatable iterative nature of the software product development process not because “agile” is our current state-of-the-art delivery methodology; every lean or efficient delivery approach relies on feedback loops to enable continuous improvement.

While ADF can be defined from both design-time and operations (runtime) perspectives, its primary value is always in design-time and SDLC. We can impact routines as repeatable and reproducible tasks. That’s how it is different from software libraries or APIs that can be employed for a one-time task that might never be repeated again (e.g. library that provides integration to specialized hardware or vendor-locked API that can be used in multiple places in source code, but the use of a library is not about repeatable part of SDLC task).

In systems engineering terminology we can declare that SDLC is a “using system” (or a supersystem) for ADF, and ADF is a “subsystem” of SDLC. This implies not only the fact that ADF is literally part of SDLC, but also enables the following mental model:

• ADF value, success criteria, and metrics are targeting SDLC improvements

• ADF stakeholders’ roles are SDLC participants

• Key ADF architecture viewpoints are defined in SDLC

In addition to SDLC concept, which is focused on engineering and instrumenting the development process, we can apply a “Flow” term to emphasize the systemic aspect of software product development in terms of value throughput and delivery management.

ADF and Flow

…ADF (is) s one of the most influencing ways to optimize a Flow throughput and eliminate bottlenecks related to product and process complexity.

Summary

An Application Development Framework is a software product “skeleton” that offers a fundamental structure for building applications within a specific environment. It acts as a reusable foundation, supplying pre-defined functionalities and promoting code organization through established conventions. This distinguishes it from other ways of reusing code in the form of an SDK, library, or API. It also makes ADF a unique opportunity to streamline development by reducing repetitive coding efforts, decreasing cognitive load, and promoting architecture best practices.

With this knowledge, we can explore the next chapters to find a way to calculate ADF return on investment, meet stakeholders’ expectations, and align the ADF roadmap with a common maturity model.

Further reading

To know more please visit the (following) links:

• Vendor-specific hardware SDKs like Android Studio

• Samsung TV SDK

• MS Xbox

• Product-specific SDKs like PandaDoc SDK

• Hubspot SDK: https://developers.hubspot.com/docs/platform/ui-extensions-sdk

• GetStream

• Platform SDKs like AWS: Amazon Chime SDK & AWS SDK for .NET

• Azure or Google’s Cloud SDK

If you found this chapter insightful, check out Building an Application Development Framework by Ivan Padabed and Roman Voronin (Packt, Sept 2025). It’s a practical, end-to-end playbook for designing, building, and rolling out a custom ADF that measurably improves flow and quality across your SDLC. The first edition includes real-world cases and a downloadable code bundle.

You’ll learn to: design and implement a robust ADF architecture; choose technologies for performance and scalability; manage versioning and packaging for efficient distribution; harden the framework’s security posture; foster collaboration to build an extensible ADF ecosystem; and measure and optimize performance (including DORA-style outcomes) for continuous improvement. Ideal for technical leaders, engineering managers, and developers who want a pragmatic, code-backed path to a reusable framework that scales with their organization.

Here’s what some readers have said:

Why call out AI systems? Unlike traditional software, AI systems are model- and data-driven: behavior depends on training data, live inputs, and evaluation metrics. Architecture is how we make that behavior predictable and operable in production.

Why architecture work matters

Architecture makes prototypes survivable in production by forcing end-to-end thinking about data reality, key non-functional constraints, stakeholder trust, and the documentation that turns design into testable behavior.

• Data reality: The key driver in any AI system is the quality of the data used for training and inference. Data engineering is often not done well or not done with a large enough aperture to appreciate the full end-to-end system. For example: does the prototype assume well-formatted or highly structured data that is not possible in production? Did the prototype suppose that data into the system was ordered but, in a production environment, that is not the case? Was the system overly reliant on a technology that cannot scale to expected production data rates? And so on.

• Partitioning and organization: Software architecture involves design decisions about partitioning and organization. An analysis of the software components of the prototype can identify where choke points may occur, how interfaces into and out of the prototype were handled, and where a component may require more design work in a production environment.

• Non-functional requirements: A robust architecture captures and understands the key non-functional requirements at play in the production environment. For example, will the new AI functionality impact availability requirements that, if not met, would result in a critical error? Does the new model with higher production data rates impact performance requirements that have cascading effects?

• Trust tolerance: Robust architecting should illuminate the trust tolerance and decision making of key stakeholders. For example, does a single off-nominal inference or decision result in a loss of confidence or outright hostility to the new AI functionality? As part of a conceptual design, guard rails or off ramps can be put into the design so there is a sort of “no-harm” guarantee that may cause consternation but not a loss of confidence.

• Documentation to test: The documentation of a conceptual design can aid in defining and illuminating test cases and scenarios that the prototype must satisfy in a production environment.

A fair criticism—and the response

It’s reasonable to say that architecting adds “document-centric” work that slows delivery. But the MIT statistic exists for a reason: no amount of heroic coding fixes a flawed architecture. Water doesn’t flow uphill; a bad architecture produces a bad system. Agile methods work best when a stable architectural frame guides rapid prototyping, integration, and demos. Productive engineering isn’t only writing code—it’s also designing systems that can survive production.

Architecting is critical in guiding engineering decisions and priorities. This is done by laying out the conceptual design of the end system. One of the first drivers of architecture is to have a solid conceptual design that informs prototyping. That same design later anchors the pass/fail criteria that we can categorize into gates.

Five production-readiness gates (with actions)

There are five gates or decision buckets—pass/fail outcomes we must clear before go-live.

Note: the following actions are not exhaustive.

1) Data Engineering (AI inputs and contracts)

• Analysis: Confirm that production inputs and interfaces match what the model expects—data rates, formats, fields, identifiers, consistency, and summary statistics.

• Ingest testing: Exercise the pipeline with current production data and verify ingest correctness and throughput.

• Resilience checks: Prove the prototype tolerates malformed records, off-nominal flows, and misformatted streams without cascading failures.

2) Model Correctness (does the model still earn its keep?)

• Ongoing relevance: Verify the model still solves a real user problem; retire it if it no longer does.

• Performance on live data: Re-evaluate with current production samples; confirm metrics such as AUC, accuracy, and false positive rate meet or exceed targets.

• Execution time: Show end-to-end model latency remains within bounds required by downstream consumers and users.

3) Observability (for data pipelines and model behavior)

• Data flow telemetry: Logging and monitors for inputs, transformations, and handoffs.

• Model execution telemetry: Traces, metrics, and logs for inference paths and failure modes.

• Acceptance bounds: Document thresholds, alert rules, and expected distributions for observability metrics.

4) Operations (release and recovery for AI pipelines)

• Canaries: Implement canary or shadow releases to surface pipeline issues before full rollout.

• Time budgets: Define and monitor time bounds for the full pipeline and each critical stage.

• Update paths: Document steps for model refresh, rollback, and disentanglement from production if needed.

5) Data Governance (quality, compliance, ethics)

• Quality audit: Run a data quality review on the latest production datasets.

• Compliance docs: Prepare artifacts to satisfy regulatory, security, and privacy requirements.

• Ethical filters: Enforce policy-aligned filtering and access controls.

In addition to these gates, the key non-functional requirements for the system need to be monitored, with action plans to ensure they are met—for example, time budgets across the system via specialized logging and canary deployments.

The gates define what must be true to go live; let us go a step further and look at the rubric that translates the actions into concrete checks and artifacts that prove each gate is passed.

Rubric for pre-deployment checks

Here is a rubric you can use as a go/no-go gate: confirm business relevance, data contracts, operational readiness (including rollback), on-call coverage and monitoring, and model quality against explicit SLOs before promoting any prototype to production.

This rubric is the evidence and checks that prove each gate is passed and that you can go live.

• Business Checks

  • Is the model output still relevant to intended users?

  • Is the prototype operating at time scales relevant to users?

  • Is the prototype still within budget and resource limits?

• Data Engineering

  • Verify data sources.

  • Verify interfaces still meet service-level agreements (format, content, time budgets).

  • Confirm computational resources for scale are in place.

• Operations

  • Verify the prototype in a pre-production environment or a simulation that mimics true production.

  • Confirm datastore schemas and queries execute correctly and produce expected results.

  • Ensure error logging exists at multiple points with sufficient detail for troubleshooting.

  • Identify a rollback procedure to take the model offline if needed.

• Pre-deployment

  • Identify a developer or staff resource to support rollout and address issues promptly.

  • Ensure monitoring is in place to capture operations.

• Model

  • Monitor non-functional requirements, SLAs, and errors.

  • Track correctness metrics (AUC, accuracy) and model drift.

  • Provide a user interface (or tools) to visualize execution.

  • Supply guides/documentation for troubleshooting, architecture diagrams, expected outputs at each stage, and accessible source code.

  • Use canaries with injected known datasets for monitoring.

Mini case study

To understand how the 5 gates and the rubric work in cohesion, let us take the example of a new application that recommends technical articles to an engineering audience. Before launch, the team decides to apply actions under each of the 5 gates and the Pre-deployment checks reveal the following:

• Data & Schema

  • Since training began, an extra tag field was added; the date format also changed, breaking metadata parsing.

  • The datastore schema drifted: a key field changed, requiring interface updates.

• Model Correctness: On current production data, AUC dropped below training thresholds; investigation showed an over-represented training subset.

• Observability: Logging failed; output and processed-file logs couldn’t be written due to permission issues.

• Operations & Release: The team planned to launch at peak traffic with no rollback procedure.

• Governance: A compliance check found personal information was being collected and shouldn’t have been.

As a result, a failed release is prevented.

For AI systems, some architecting during prototyping helps cross the chasm from an impressive demonstration to a production system that delivers true value and positive business outcomes.

If you found this article insightful, Richard D Avila and Imran Ahmad’s forthcoming book, Architecting AI Software Systems extends the same ideas with a structured approach to integrating AI into traditional architectures—covering how inference and decision-making shape design, using architectural models to ensure cohesion, mitigating risks like underperformance and cost overruns, and applying patterns and heuristics for scalable, high-performance systems—anchored by real-world case studies, hands-on exercises, and a complete example of an AI-enabled system for architects, engineering leaders, and AI/ML practitioners.

About the Author:

Richard D Avila is a software and systems architect with over two decades of industry experience building complex software systems. He has architected and held leadership roles for building a wide array of software systems from complex simulations, autonomy and data analytic systems. He is also a principal investigator for AI topics. He has published in referred journals and industry publications on command-and-control theory, assurance architectures, multi-agent modeling, and machine learning. He was the first expert instructor on data analytics at University of Maryland Baltimore County – Training Centers. Before working as a software and systems architect, he served in the US Navy as a submarine officer.

The complete “Chapter 14: Quantum Error Correction and Fault Tolerance” from the book A Practical Guide to Quantum Computing by Elías F. Combarro and Samuel González-Castillo (Packt, 2025)

4.55MB ∙ PDF file

Download

Chapter 14, Quantum Error Correction and Fault Tolerance, introduces quantum error correction, which could pave the road towards fault-tolerant computation and thus may be a cornerstone in the development of useful quantum computers. This chapter begins with a discussion on the necessity for quantum error correction, which is followed by the introduction and implementation of a simple quantum error-correcting code. The chapter finishes with some remarks on fault-tolerant computing and how quantum error correction can help enable it.

Download

If you liked this chapter, check out A Practical Guide to Quantum Computing: Hands-on approach to quantum computing with Qiskit by Elías F. Combarro and Samuel González-Castillo (Packt, July 2025). It’s a self-contained introduction that starts from a single qubit and builds to foundational algorithms—Deutsch, Bernstein–Vazirani, Grover, Shor—with runnable code in Qiskit 2.1 on simulators and real hardware. Along the way, it covers applications like quantum money and BB84, and introduces fault tolerance and the road to quantum advantage.

You’ll learn to: understand what makes a quantum computer unique; represent and measure multi-qubit states; use superposition, entanglement, and interference; implement and run quantum algorithms in Qiskit; explain how Grover’s search and Shor’s factoring work; and get familiar with quantum fault tolerance and emerging notions of quantum advantage. Ideal for university-level STEM courses and for professionals or self-learners who want a practical, code-first path into quantum computing.

Here is what some readers have said:

Download

Clean Architecture, introduced by Robert C. Martin in 2012, synthesizes decades of software design wisdom into a cohesive set of principles. It addresses persistent challenges in software development, such as managing complexity and accommodating change. By applying Clean Architecture principles to Python projects, developers can create systems that are not only functional but also maintainable, testable, and adaptable over time.

In this chapter, we’ll explore the essence of Clean Architecture and its relevance to Python development. We’ll examine how Clean Architecture principles align with Python’s philosophy of simplicity and readability, creating a natural synergy that enhances Python’s strengths. You’ll learn how Clean Architecture can help you build Python applications that are easier to understand, modify, and extend, even as they grow in complexity.

By the end of this chapter, you’ll have an overview of Clean Architecture principles and their potential benefits for Python development. You’ll be introduced to how this approach can address common challenges in software development, particularly as Python projects grow in scale and complexity. This foundational understanding of Clean Architecture will be essential as we delve deeper into its implementation and best practices in Python throughout the rest of the book.

In this chapter, we’re going to cover the following main topics:

• Why Clean Architecture in Python: the benefits of balancing planning and agility

• What is Clean Architecture?

• Clean Architecture and Python: a natural fit

Why Clean Architecture in Python: the benefits of balancing planning and agility

In this section, we’ll explore the critical balance between planning and agility in Python development and how Clean Architecture can help achieve this balance. We’ll examine the challenges posed by increasing complexity in modern Python applications and the imperative for agility in today’s fast-paced business environment. We’ll then discuss the trade-offs between planning and flexibility, and how architectural thinking can provide a framework for managing these trade-offs. Finally, we’ll look at the role of architecture in managing complexity and setting the stage for long-term success. Through these discussions, you’ll gain insight into why Clean Architecture is particularly valuable for Python developers striving to create maintainable, adaptable, and efficient applications.

Let’s begin by examining the complex challenges facing modern Python development.

The complexity challenge in modern Python development

As Python’s popularity soars, so do the scale and complexity of applications built with it. From web services to data science pipelines, Python projects are growing larger and more intricate. This growth brings significant challenges that every Python developer must grapple with.

The increasing complexity of systems makes them harder to understand, modify, and maintain. This complexity can severely limit your ability to add new features or respond to changing requirements. The maintenance burden of complex Python systems can overwhelm development teams, slowing down progress and innovation. Even small changes in large, complex systems can have far-reaching consequences, making modifications expensive and risky.

Consider a fictitious large Python-based e-commerce site: PyShop. The business decides to implement a seemingly simple feature: adding gift-wrapping options to orders. However, this straightforward addition quickly cascades into a complex project:

• The order processing module needs updates to include gift-wrapping choices

• The inventory system requires modification to track gift-wrapping supplies

• The pricing engine needs adjustments to calculate additional costs

• The user interface (UI) must be updated to present gift-wrapping options

• The fulfillment system needs changes to include gift-wrapping instructions

What was estimated as a two-week-long task stretches into a multi-month project. Each change potentially impacts other system parts: adjustments in order processing affect reporting, inventory changes influence supply chain management, and UI modifications require extensive user experience testing.

This example highlights how interconnected modules in a complex system can turn a simple feature addition into a significant undertaking, emphasizing the need for an architecture that allows for more isolated changes and easier testing processes.

Moreover, as Python projects grow, developers often struggle with abstractions, a critical aspect that Clean Architecture helps address. Without proper guidance, codebases can suffer from extremes: either becoming a tangled mess of deeply nested class hierarchies that are hard to understand and modify or devolving into monolithic do-everything classes that lack any meaningful abstraction. In the former case, developers may create too-complex inheritance structures to maximize code reuse, resulting in a fragile system where changes in one place have unforeseen consequences elsewhere. In the latter case, the lack of abstraction leads to massive, unwieldy classes and rampant code duplication, making it nearly impossible to maintain consistency or make systemic changes. Both scenarios result in codebases that are difficult to understand, maintain, and extend, which is precisely the issues that a well-planned architecture helps prevent.

Furthermore, in today’s rapidly evolving tech landscape, complex, tightly coupled systems struggle to take advantage of new technologies. This limitation can significantly impact your ability to stay competitive in a field where technological agility is crucial.

The agility imperative

In our fast-paced business environment, agility is not just an advantage—it’s a necessity. With every company essentially becoming a technology company, the pressure to deliver quickly has never been higher. Python’s simplicity and extensive ecosystem make it an excellent choice for rapid development.

However, sustainable agility requires more than just initial speed, it demands architectural decisions that support ongoing evolution. It’s akin to building a high-performance race car: without proper design fundamentals, what starts as impressive acceleration quickly becomes limited by poor handling and maintenance challenges.

In rapidly evolving Python applications, this principle becomes starkly evident. Without a cohesive architecture, quick feature additions can create a tangled web of dependencies. What starts as a nimble codebase can, within months, become rigid and fragile. Developers find themselves spending more time deciphering existing code than writing new features. When it’s not immediately clear where new code should be added or how it should interact with existing components, developers under pressure may make hasty decisions, leading to suboptimal implementations and introducing bugs. These quick fixes further complicate the codebase, making future changes even more challenging. The initial velocity becomes unsustainable, not because of the speed itself, but due to the lack of a sturdy architectural foundation that can guide rapid changes and provide clear pathways for new feature integration.

Requirements change, often unpredictably. Your Python projects need to be structured in a way that allows for easy adaptation to these changes. This adaptability is crucial for long-term success in software development.

Striking a balance: the planning–agility trade-off

Finding the right balance between planning and agility is crucial in Python development. As Dave Thomas wisely said, “Big design up front is dumb. Doing no design up front is even dumber.” The key is finding the middle ground that allows for both structure and flexibility.

Good architecture helps you postpone decisions. It gives you the flexibility to push decisions to later stages when you have more information to make the correct choice. This approach is particularly valuable in Python development, where the language’s flexibility can sometimes lead to decision paralysis.

Introducing architectural thinking in Python development means considering the long-term structure of your project from the start, without over-engineering. It’s about creating a framework that guides development while remaining adaptable to change.

The role of architecture in managing complexity

Effective architecture is your best tool for managing complexity in Python systems. Good architecture simplifies complex systems by providing a clear structure and separation of concerns (SoC). One of the first steps in architecting a new system is determining how to divide it, keeping things that change for the same reason together and things that change for different reasons apart.

Consider two Python-based content management systems (CMSs) for media companies, both tasked with implementing a new AI-powered content tagging feature. In the well-architected system, this feature is implemented as a standalone module with clear interfaces. It integrates smoothly with the existing content creation and search modules through well-defined APIs. Developers can build and test the AI tagging service independently, and then connect it to the content database and UI with minimal disruption. Conversely, in a poorly structured system, adding this feature requires changes across the entire stack—from database schemas to frontend code—leading to unexpected bugs and performance issues. What takes a sprint in the well-architected system becomes a months-long refactoring project in the poorly structured one, demonstrating how thoughtful initial architecture can dramatically improve development efficiency and system adaptability.

The architectural decisions you make early on have a profound impact on the long-term development costs and flexibility of your Python projects. A well-architected system can significantly reduce the cost of change over time, allowing your team to respond more quickly to new requirements or technological changes.

Preparing for Clean Architecture

As we move toward discussing Clean Architecture, it’s important to understand that it offers a systematic approach to balancing planning and agility in Python projects. Architectural principles provide powerful tools for managing and reducing complexity in your Python systems.

At its core, Clean Architecture is about strategic SoC in your Python applications. It advocates for a structure where the essential business logic is insulated from external factors such as UIs, databases, and third-party integrations. This separation creates clear boundaries between different parts of your application, each with its own responsibilities. By doing so, Clean Architecture allows your core business rules to remain pure and unaffected by the implementation details of input/output (I/O) mechanisms or data management systems (DMSs).

By understanding these challenges and principles, you’ll be better prepared to appreciate the benefits that Clean Architecture can bring to your Python projects. In the next sections, we’ll delve into what Clean Architecture is and how it specifically applies to Python development, providing you with the tools to combat complexity and reduce the cost of change in your software systems.

What is Clean Architecture?

Having explored the challenges of managing complexity in Python development and the need for balancing planning with agility, the goal of this section is to give you a high-level overview of Clean Architecture. We’ll be covering several key concepts and principles in quick succession to provide a broad understanding. Don’t worry if you don’t grasp every detail immediately. This is just the beginning of our journey. Each of these topics will be explored in depth in the chapters to come, where we’ll dive into practical Python implementations and real-world scenarios.

Clean Architecture synthesizes many ideas from previous architectural styles, but it is built around a fundamental concept: the separation of software elements into ring levels, with a strict rule that code dependencies can only move inward from outer levels. This principle is formally known as the Dependency Rule, one of the most critical aspects of Clean Architecture. The Dependency Rule states that source code dependencies must only point inward, toward higher-level policies. Inner circles must know nothing about outer circles, while outer circles must depend on and adapt to inner circles. This ensures that changes to external elements (like databases, UI, or frameworks) don’t impact the core business logic. The aim is to create software systems that are not only functional but also maintainable and adaptable over time. To illustrate this, let’s consider a simple Python application for a library management system:

1. At the core, we have the Book class, representing the basic data structure.

2. Moving outward, we have a BookInventory class that manages operations on books.

3. In the outer ring, we have a BookInterface class that handles user interactions related to books.

In this structure, the Book class knows nothing about the BookInventory or BookInterface classes. The BookInventory class might use the Book class but doesn’t know about the interface. This separation ensures that the core logic remains unaffected by external concerns.

Crucially, this structure allows us to modify or even replace outer layers without affecting the inner layers. For instance, we could change the UI from a command-line interface (CLI) to a web interface by modifying the BookInterface class, without needing to alter the Book or BookInventory classes. This flexibility is a key advantage of the Clean Architecture approach.

This structure is designed to produce systems that embody the key principles we introduced earlier:

• SoC

• Independence of external details

• Testability and maintainability

Let’s explore further how Clean Architecture achieves these goals and the benefits it brings to software development.

The onion architecture concept

Let’s visualize the ring levels mentioned earlier and add another level of detail as to the purpose of each ring. Clean Architecture is often visualized as a series of concentric circles, like an onion. Each circle represents a different layer of software, and the Dependency Rule we discussed ensures that dependencies only flow inward across these boundaries. The core layers contain business logic (entities), while the external layers contain interface and implementation details (see Figure 1.1):

Figure 1.1: Clean Architecture: a series of concentric layers

Figure 1.1 demonstrates the separation of inner core business logic progressing out to external interfaces:

• Entities: At the center are entities, which encapsulate enterprise-wide business rules. Entities in this context are the primary nouns of your product, the core business objects that would exist even without software. For example, in an e-commerce system, entities might include Customer, Product, and Order. In a task management application, they could be User, Task, and Project. These entities contain the most basic, universal rules about how these objects behave and interact.

• Use Cases: The next layer contains use cases, which orchestrate the flow of data to and from entities. A use case represents a specific way the system is used. It’s essentially a description of how the system should behave for a particular scenario. For instance, in a task management app, use cases might include Create New Task, Complete Task, or Assign Task. Use cases contain application-specific business rules and control how and when entities are used to fulfill the goals of the application.

• Interface Adapters: Further out, we find interface adapters, which convert data between use cases and external agencies. This layer acts as a set of translators between the inner layers (entities and use cases) and the outer layer. It might include things such as controllers that handle HTTP requests, presenters that format data for display, and gateways that transform data for persistence. In a Python web application, this might include your view functions or classes that handle routing and request processing. A key point of this layer is that it allows us to decouple from frameworks.

• Frameworks and Drivers: The outermost layer contains frameworks and drivers, where the external agencies reside. By drivers, we mean the specific tools, frameworks, and delivery mechanisms that are used to run the system but aren’t core to the business logic. In a Python context, examples might include the following:

  • Web frameworks such as Django or Flask

  • Database drivers such as psycopg2 for PostgreSQL or pymongo for MongoDB

  • External libraries for tasks such as sending emails (for example, smtplib) or processing payments

  • UI frameworks if you’re building a desktop or mobile app (for example, PyQt)

  • System utilities for tasks such as logging or configuration management

This outermost layer is the most volatile, as it’s where we interact with the outside world and where technologies are most likely to change over time. By keeping it separate from our core business logic, we can more easily swap out these external tools without affecting the heart of our application.

This layered structure of Clean Architecture promotes SoC, establishing a clear organizational framework for software systems. Now that we have an idea of the fundamental structure of Clean Architecture, let’s further investigate its broader benefits.

Benefits of Clean Architecture

One of the primary advantages of Clean Architecture is its focus on protecting and isolating your core business logic, the domain objects that represent the foundation of your business. While external details such as web frameworks and persistence engines come and go, the true value to your business lies in the time invested in designing and implementing these core domain objects. Clean Architecture recognizes this and provides a structure that insulates these crucial components from the volatility of external technologies.

This architectural approach protects your investment in domain logic from the need to move away from a given framework or technology. For example, if a framework you’re using moves from an open source model to a proprietary one, Clean Architecture allows you to replace it without rewriting your core business logic. This separation significantly reduces the cost and risk of changes over time, allowing your system to evolve more easily as requirements change or as you need to adapt to new technologies. In essence, Clean Architecture ensures that the most valuable and stable part of your application, your business logic, remains unaffected by the often turbulent world of external technologies and frameworks.

Another key benefit is enhanced testability across all layers of the application. The independence of the core business logic from external details makes it much easier to write comprehensive unit tests. You can test business rules in isolation, without the need to spin up a database or web server or build cumbersome mocks. This leads to more thorough testing and, consequently, more robust software. It also encourages developers to write more tests, as the process becomes simpler and more straightforward.

Clean Architecture also provides flexibility in technology choices. Because the core of the application isn’t dependent on external frameworks or tools, you have the freedom to swap out these elements as needed. This is particularly valuable in the fast-moving world of technology, where today’s popular framework might be obsolete tomorrow. Similarly, you might start with a CLI for internal use, then add a web interface for broader access, all without altering your core business rules’ code. Your core business logic remains stable, while you have the flexibility to adopt new technologies in the outer layers as they emerge. Lastly, Clean Architecture promotes long-term agility in development and leads to what Robert C. Martin calls a Screaming Architecture. Its focus on separating concerns and managing dependencies results in a codebase that’s easier to understand and modify. The concept of Screaming Architecture suggests that when you look at the structure of your system, it should scream its purpose and use cases, not its frameworks or tools. For instance, your architecture should scream online bookstore, not Django application. This clear, purpose-driven structure allows new team members to quickly grasp the system’s intent and make contributions. The architecture itself becomes a form of documentation, revealing the system’s core purpose and functionality at a glance. Such clarity and flexibility translate to increased development speed over the long term, even as the system grows in complexity. It also ensures that your system remains focused on its core business logic, rather than being tied to specific technical implementations.

Clean Architecture in context

To fully appreciate the value of Clean Architecture, it’s important to understand its place in the broader context of software development practices and methodologies.

Clean Architecture represents an evolution from traditional layered architecture. While it builds upon the concept of layers, it places a stronger emphasis on SoC and enforces the Dependency Rule more strictly than traditional architectures. Unlike traditional layered architectures where lower layers often depend on persistence or infrastructure concerns, Clean Architecture keeps the inner layers pure and focused on business logic. This shift in focus allows for greater flexibility and resilience to change.

Clean Architecture complements modern development practices such as Agile and DevOps. It aligns well with Agile methodologies by facilitating continuous delivery (CD) and making it easier to respond to change. The clear SoC supports iterative development and makes it easier to modify or extend functionality in response to changing requirements. In terms of DevOps, Clean Architecture supports practices such as continuous integration and deployment (CI/CD) by making systems more testable and modular. The clear boundaries between components can also help in scaling development across teams, as different teams can work on different layers or components with minimal interference.

In conclusion, Clean Architecture offers a powerful approach to building software systems that are scalable, maintainable, and adaptable to change. By focusing on SoC and managing dependencies, it provides a structure that can withstand the test of time and the pressures of evolving technology and business needs. As we move into the next section, we’ll explore how these principles align particularly well with Python development practices.

Clean Architecture and Python: a natural fit

As we’ve explored the principles and benefits of Clean Architecture, you might be wondering how well these concepts align with Python development. In this section, we’ll discover that Clean Architecture and Python share a natural affinity, making Python an excellent language for implementing Clean Architecture principles.

Python’s philosophy, as embodied in The Zen of Python aligns remarkably well with Clean Architecture principles. Both emphasize simplicity, readability, and the importance of well-structured code. Python’s focus on creating clear, maintainable, and adaptable code provides a strong foundation for implementing Clean Architecture. As we delve deeper into this section, we’ll explore how Python language features can be leveraged to create robust, maintainable systems that adhere to Clean Architecture principles.

Implementing Clean Architecture in Python

Python’s dynamic nature, combined with its strong support for object-oriented programming (OOP) and functional programming paradigms, allows developers to implement Clean Architecture concepts with less boilerplate and greater clarity than many other languages.

Note on code examples

Throughout this book, you’ll notice type annotations in our code examples (e.g., def function(parameter: type) -> return_type). These type hints enhance code clarity and help enforce Clean Architecture boundaries. We’ll explore this powerful feature in depth in Chapter 3.

A key principle of Clean Architecture is the reliance on abstractions rather than concrete implementations. This principle directly supports the Dependency Rule we discussed earlier: dependencies should only point inward. Let’s see how this works in practice using Python’s abstract base classes (ABCs).

Consider the following example, which models a notification system:

from abc import ABC, abstractmethod
class Notifier(ABC):
    @abstractmethod
    def send_notification(self, message: str) -> None:
        pass
class EmailNotifier(Notifier):
    def send_notification(self, message: str) -> None:
        print(f"Sending email: {message}")
class SMSNotifier(Notifier):
    def send_notification(self, message: str) -> None:
        print(f"Sending SMS: {message}")
class NotificationService:
    def __init__(self, notifier: Notifier):
        self.notifier = notifier
    def notify(self, message: str) -> None:
        self.notifier.send_notification(message)
# Usage
email_notifier = EmailNotifier()
email_service = NotificationService(email_notifier)
email_service.notify("Hello via email")

This example demonstrates key concepts of Clean Architecture using Python’s ABCs:

1. ABC: The Notifier class is an ABC, defining an interface that all notifier classes must follow. This represents an inner ring in our Clean Architecture structure.

2. Abstract method: The send_notification method in Notifier is marked with @abstractmethod, enforcing implementation in subclasses.

3. Concrete implementations: EmailNotifier and SMSNotifier are concrete classes in an outer ring. They inherit from Notifier and provide specific implementations.

4. Dependency inversion: The NotificationService class depends on the abstract Notifier class, not on concrete implementations. This adheres to the Dependency Rule, as the abstract Notifier class (inner ring) doesn’t depend on the concrete notifiers (outer ring). We’ll dive deeper into dependency inversion in the next chapter.

This structure embodies the Clean Architecture principles we’ve discussed:

• It respects the Dependency Rule: The abstract Notifier class (inner ring) knows nothing about the concrete notifiers or the NotificationService class (outer rings)

• It allows for easy extension: We can add new types of notifiers (such as PushNotifier) without changing the NotificationService class

• It promotes flexibility and maintainability: The core business logic (sending a notification) is separated from the implementation details (how the notification is sent)

By structuring our code this way, we create a system that’s not only flexible and maintainable but also adheres to the fundamental principles of Clean Architecture. The abstract Notifier class represents our core business rules, while the concrete notifiers and the NotificationService class represent the more volatile outer layers. This separation allows us to easily swap or add new notification methods without affecting the core logic of our application.

So, we’ve seen a simple ABC example, but this is where Python truly shines. We can implement the same Clean Architecture principles without using a class hierarchy, instead relying on Python’s support for duck typing. This flexibility is one of Python’s strengths, allowing developers to choose the approach that best fits their project’s needs while still adhering to Clean Architecture principles.

Duck typing is a programming concept where the suitability of an object is determined by the presence of certain methods or properties, rather than its explicit type. The name comes from the saying, “If it walks like a duck and quacks like a duck, then it must be a duck.” In duck typing, we don’t care about the object’s type; we care about whether it can do what we need it to do.

This approach aligns well with Clean Architecture’s emphasis on abstractions and interfaces. If you’d prefer to stay away from rigid class hierarchies, Python’s Protocol feature, introduced in Python 3.8, offers the best of both worlds: duck typing with type hinting. Here’s an example that implements the same notification system using protocols:

from typing import Protocol
class Notifier(Protocol):
    def send_notification(self, message: str) -> None:
        ...
class EmailNotifier: # Note: no explicit inheritance
    def send_notification(self, message: str) -> None:
        print(f"Sending email: {message}")
class SMSNotifier: # Note: no explicit inheritance
    def send_notification(self, message: str) -> None:
        print(f"Sending SMS: {message}")
class NotificationService:
    # Still able to use type hinting
    def __init__(self, notifier: Notifier):
        self.notifier = notifier
    def notify(self, message: str) -> None:
        self.notifier.send_notification(message)
# Usage
sms_notifier = SMSNotifier()
sms_service = NotificationService(sms_notifier)
sms_service.notify("Hello via SMS")

This example demonstrates the same notification system as before but using Python’s Protocol feature instead of ABCs. Let’s break down the key differences and their implications for Clean Architecture:

• Protocol versus ABC: The Notifier class is now a Protocol class instead of an ABC class. It defines a structural subtyping interface rather than requiring explicit inheritance.

• Implicit conformance: The EmailNotifier and SMSNotifier classes don’t explicitly inherit from the Notifier class, but they conform to its interface by implementing the send_notification method.

• Duck typing with type hinting: This approach combines Python’s duck typing flexibility with the benefits of static type checking, aligning with Clean Architecture’s emphasis on loose coupling.

• Concrete implementations: The NotificationService class still depends on the abstract Notifier protocol, not concrete implementations, adhering to Clean Architecture principles.

This protocol-based approach offers a flexible, Pythonic implementation of Clean Architecture concepts, balancing type safety with reduced class hierarchy rigidity. It demonstrates how to align Clean Architecture principles with Python’s philosophy, promoting adaptable and maintainable code.

We highly recommend the use of type hinting via either ABCs or protocols when implementing Clean Architecture. This approach, as opposed to simple implicit interfaces without type hinting, offers significant advantages:

• Improved code readability

• Enhanced IDE support and earlier error detection

• Better alignment with Clean Architecture goals

In the remaining parts of the book, we’ll primarily use ABCs in our examples as they are in greater use in existing Python codebases. However, the principles discussed are equally applicable to protocol-based implementations, and readers can adapt the examples to use protocols if preferred.

Practical example: a glimpse of Clean Architecture in a Python project

To illustrate the concepts we’ve discussed, let’s examine the basic structure of a Clean Architecture Python project. This structure embodies the principles we’ve covered and demonstrates how they translate into a practical file organization. We’ll stay at a high level here; later chapters will cover real-world examples in full detail:

Figure 1.2: A potential layout for a Clean Architecture Python web application

This file structure in Figure 1.2 exemplifies the Clean Architecture principles we’ve discussed:

1. SoC: Each directory represents a distinct layer of the application, aligning with the concentric circles we saw in Figure 1.1.

2. Dependency Rule: The structure enforces the Dependency Rule we discussed earlier. If we were to investigate the inner layers (entities and use_cases), we would not see any imports from the outer layers.

3. Entities layer: The entities directory contains the core business objects, such as user.py. These are at the center of our Clean Architecture diagram and have no dependencies on outer layers.

4. Use Cases layer: The use_cases directory holds the application-specific business rules. It depends on the entities but is independent of the outer layers.

5. Interface Adapters layer: The interfaces directory contains controllers, presenters, and gateways. These adapt data between use cases and external agencies (such as web frameworks or databases).

6. Frameworks layer: The outermost frameworks directory contains implementations of external interfaces, such as database object-relational mappers (ORMs) or web frameworks.

7. Straightforward testing: The tests directory structure mirrors the application structure, allowing for comprehensive testing at all levels.

This structure supports the key benefits of Clean Architecture we discussed:

• Maintainability: Changes to external components (in the frameworks directory) don’t affect the core business logic in entities and use cases

• Flexibility: We can easily swap out the database or web framework in the frameworks directory without touching the business logic

• Testability: The clear separation allows for easy unit testing of core components and integration testing of interfaces

Remember our discussion about abstractions? The interfaces directory is where we’d implement the ABCs or protocols we talked about. For instance, user_repository.py might define an abstract UserRepository class, which is then implemented concretely in the frameworks/database/orm.py file.

This structure also facilitates the master plan we mentioned earlier. It provides a clear roadmap for where new code should be placed, helping developers make consistent decisions even as the project grows and evolves.

By organizing our Python project this way, we’re setting ourselves up for long-term success, creating a codebase that’s not just functional but also maintainable, flexible, and aligned with Clean Architecture principles.

Python-specific considerations and potential pitfalls

While Clean Architecture and Python are highly compatible, there are some important considerations to be aware of when implementing these principles in Python projects. Throughout this book, we’ll guide you through mitigating these concerns, providing practical solutions and best practices.

Balancing Pythonic code with architectural principles

Python’s batteries included philosophy and an extensive standard library can sometimes tempt developers to bypass architectural boundaries for the sake of convenience. However, maintaining a clean architecture often involves creating abstractions around even standard library functions to maintain SoC. For example, instead of directly using Python’s smtplib library in your use cases, consider creating an abstraction layer for sending notifications.

As we progress through this book, we’ll demonstrate how this effort of creating abstractions pays off in terms of maintainability, flexibility, and testability. You’ll see that the initial investment in Clean Architecture principles yields significant long-term benefits.

Python’s ease of importing can sometimes lead to messy dependency structures, as all packages are effectively public. We’ll show you how to be vigilant about maintaining the Dependency Rule, ensuring that inner layers don’t depend on outer layers. In Chapter 2, we’ll explore techniques and tools to help you maintain clean dependency structures in your Python projects.

Scaling Clean Architecture in Python projects

The application of Clean Architecture principles should be tailored to the size and complexity of your Python project.

For instance, in small projects or quick prototypes, it’s perfectly fine to have a simple, monolithic architecture. However, even in these cases, building in a thoughtful, modular manner can set the stage for future growth. You might start with a simple structure:

Figure 1.3: Quick prototype Python project

In this small project, you can still apply Clean Architecture principles by doing the following:

• Keeping business logic in models.py separate from presentation logic in views.py

• Using dependency injection (DI) to make components more modular and testable

• Defining clear interfaces between modules

As your project grows, you can gradually evolve toward a more comprehensive Clean Architecture structure. This evolution might involve the following:

1. Separating core business logic (entities and use cases) into their own modules

2. Introducing interfaces to abstract away framework-specific code

3. Organizing tests to align with the architectural layers

This book takes a hands-on approach, starting with a basic application and a pragmatic application of Clean Architecture principles. As we progress, the complexity of our example application will increase, demonstrating how to evolve the Clean Architecture approach as the codebase grows.

Clean Architecture is a spectrum, not a binary choice. The patterns we’ll explore represent a comprehensive implementation designed to showcase Clean Architecture’s full capabilities, but in practice, you might choose to implement only the patterns that provide clear value for your specific context. A small API might benefit from clean controller patterns without needing full presenter abstractions, while a data processing script might adopt domain entities while skipping interface adapters entirely. You’ll learn how to apply these principles judiciously, avoiding over-engineering in smaller projects while leveraging the full power of Clean Architecture in larger systems. The key is understanding what each pattern provides so you can make informed decisions about which architectural boundaries matter most for your project.

Leveraging Python’s dynamic nature appropriately

While Python’s dynamic nature is powerful, it can also lead to issues if not used carefully. Chapter 3 is devoted to aspects of Python’s dynamic nature, including duck typing, the use of type hints, and newer features such as protocols. By the end of this chapter, you’ll have a solid foundation on how to best leverage these language features to support a Clean Architecture approach, balancing Python’s flexibility with architectural rigor.

Testing considerations

This book, as with Clean Architecture itself, strongly promotes the use of tests. Tests are essentially first-class clients of your application code, using the codebase and making assertions on the results. The same architectural considerations that apply to your main codebase also apply to your Python tests.

We’ll guide you through writing tests that respect architectural boundaries. You’ll learn to recognize when your tests are indicating potential issues in your architecture, such as when they require excessive setup or mocking. In the test cases for each chapter’s code examples and culminating in Chapter 8, we’ll explore these concepts in depth, showing you how to use tests not just for verification, but as a tool for maintaining and improving your architecture.

By being aware of these considerations and potential pitfalls and following the guidance provided throughout this book, you can create Python systems that are both clean and practical, leveraging the strengths of both Clean Architecture and Python. Remember, the key is to apply these principles thoughtfully, always with an eye toward creating maintainable, testable, and flexible Python code.

Summary

In this chapter, we introduced Clean Architecture at a high level and its relevance to Python development. We gave you context by exploring the evolution of software architecture, from Waterfall to Agile, highlighting persistent challenges in managing complexity, accommodating change, and maintaining long-term productivity.

We introduced Clean Architecture’s core principles:

• Separation of concerns (SoC)

• Independence of external details

• Testability and maintainability

We examined Clean Architecture’s general structure, from core entities and use cases to outer layers of interface adapters, frameworks, and drivers, emphasizing how this structure promotes maintainability and flexibility. We discussed the benefits of Clean Architecture, including improved adaptability, enhanced testability, and long-term development agility, and how it complements modern development practices such as Agile and DevOps.

Furthermore, we explored the natural fit between Clean Architecture and Python, addressing how Python’s features can be leveraged to implement Clean Architecture effectively. We also highlighted Python-specific considerations and potential pitfalls, emphasizing the need to balance Pythonic code with architectural principles and adapt Clean Architecture to different project sizes.

In this chapter, we introduced the primary goals of Clean Architecture and explored its natural fit with Python development. We saw how Clean Architecture principles can be implemented using Python’s features such as ABCs and protocols, providing a foundation for creating maintainable and flexible software systems.

In the next chapter, we’ll build upon this foundation by diving into the SOLID principles. These principles, which form the bedrock of Clean Architecture, will be explored in depth with practical Python examples, showing how they contribute to robust and extensible application design.

Further reading

To learn more about the topics that were covered in this chapter, take a look at the following resources:

• Clean Architecture: A Craftsman’s Guide to Software Structure and Design by Robert C. Martin. This book provides a comprehensive look at Clean Architecture from its originator.

• Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans. While not specific to Clean Architecture, this book provides valuable insights into designing software around business domains.

• Agile Software Development, Principles, Patterns, and Practices by Robert C. Martin. This book covers many of the principles that underpin Clean Architecture in the context of Agile development.

• The Pragmatic Programmer: Your Journey to Mastery by Andrew Hunt and David Thomas. This classic book offers practical advice on software design and development that aligns well with Clean Architecture principles.

To learn Clean Architecture through a series of real-world, code-centric examples and exercises, optimize system componentization, and significantly reduce maintenance burden and overall complexity, check out Clean Architecture with Python by . The book helps you apply Clean Architecture concepts confidently to new Python projects and legacy code refactoring.

Yet, we don't know exactly how and, especially, when to utilize these principles and syntax in practice. In this chapter, we'll discuss some useful applications of the knowledge we've gained, looking at some new topics along the way:

• How to recognize objects

• Data and behaviors, once again

• Wrapping data behaviors using properties

• The Don't Repeat Yourself principle and avoiding repetition

In this chapter, we'll also address some alternative designs for our case study problem. We'll look at ways to partition the sample data into training sets and test sets.

We'll start this chapter with a close look at the nature of objects and their internal state. There are cases when there's no state change, and a class definition isn't desirable.

Treat objects as objects

This may seem obvious; you should generally give separate objects in your problem domain a special class in your code. We've seen examples of this in the case studies in previous chapters: first, we identify objects in the problem, and then model their data and behaviors.

Identifying objects is a very important task in object-oriented analysis and programming. But it isn't always as easy as counting the nouns in short paragraphs that, frankly, the authors have constructed explicitly for that purpose. Remember, objects are things that have both data and behavior. If we are working only with data, we are often better off storing it in a list, set, dictionary, or other Python data structure (which we'll be covering thoroughly in Chapter 7, Python Data Structures). On the other hand, if we are working only with behavior, but no stored data, a simple function is more suitable.

An object, however, has both data and behavior. Proficient Python programmers use built-in data structures unless (or until) there is an obvious need to define a class. There is no reason to add an extra level of complexity if it doesn't help organize our code. On the other hand, the need is not always self-evident.

We can often start our Python programs by storing data in a few variables. As the program expands, we will later find that we are passing the same set of related variables to a set of functions. This is the time to think about grouping both variables and functions into a class.

For example, if we are designing a program to model polygons in two-dimensional space, we might start with each polygon represented as a list of points. The points would be modeled as two tuples (x, y) describing where that point is located. This is all data, stored in a set of nested data structures (specifically, a list of tuples). We can (and often do) start hacking at the command prompt:

>>> square = [(1,1), (1,2), (2,2), (2,1)] 

Now, if we want to calculate the distance around the perimeter of the polygon, we need to sum the distances between each point. To do this, we need a function to calculate the distance between two points. Here are two such functions:

>>> from math import hypot
>>> def distance(p_1, p_2):
...     return hypot(p_1[0]-p_2[0], p_1[1]-p_2[1])
>>> def perimeter(polygon):
...     pairs = zip(polygon, polygon[1:]+polygon[:1])
...     return sum(
...         distance(p1, p2) for p1, p2 in pairs
...     )

We can exercise the functions to check our work:

>>> perimeter(square)
4.0

This is a start, but it's not completely descriptive of the problem domain. We can kind of see what a polygon might be. But we need to read the entire batch of code to see how the two functions work together.

We can add type hints to help clarify the intent behind each function. The result looks like this:

from __future__ import annotations
from math import hypot
from typing import Tuple, List
Point = Tuple[float, float]
def distance(p_1: Point, p_2: Point) -> float:
    return hypot(p_1[0] - p_2[0], p_1[1] - p_2[1])
Polygon = List[Point]
def perimeter(polygon: Polygon) -> float:
    pairs = zip(polygon, polygon[1:] + polygon[:1])
    return sum(distance(p1, p2) for p1, p2 in pairs)

We've added two type definitions, Point and Polygon, to help clarify our intentions. The definition of Point shows how we'll use the built-in tuple class. The definition of Polygon shows how the built-in list class builds on the Point class.

When writing annotations inside method parameter definitions, we can generally use the type name directly, for example, def method(self, values: list[int]) -> None:. For this to work, we need to use from __future__ import annotations. When defining a new type hint, however, we need to use the names from the typing module. That's why the definition of the new Point type uses typing.Tuple in the expression Tuple[float, float].

Now, as object-oriented programmers, we clearly recognize that a polygon class could encapsulate the list of points (data) and the perimeter function (behavior). Further, a Point class, such as we defined in Chapter 2, Objects in Python, might encapsulate the x and y coordinates and the distance method. The question is: is it valuable to do this?

For the previous code, maybe yes, maybe no. With our recent experience in object-oriented principles, we can write an object-oriented version in record time. Let's compare them as follows:

from math import hypot
from typing import Tuple, List, Optional, Iterable
class Point:
    def __init__(self, x: float, y: float) -> None:
        self.x = x
        self.y = y
    def distance(self, other: "Point") -> float:
        return hypot(self.x - other.x, self.y - other.y)
class Polygon:
    def __init__(self) -> None:
        self.vertices: List[Point] = []
    def add_point(self, point: Point) -> None:
        self.vertices.append((point))
    def perimeter(self) -> float:
        pairs = zip(
            self.vertices, self.vertices[1:] + self.vertices[:1])
        return sum(p1.distance(p2) for p1, p2 in pairs)

There seems to be almost twice as much code here as there was in our earlier version, although we could argue that the add_point method is not strictly necessary. We could also try to insist on using _vertices to discourage the use of the attribute, but the use of leading _ variable names doesn't seem to really solve the problem.

Now, to understand the differences between the two classes a little better, let's compare the two APIs in use. Here's how to calculate the perimeter of a square using the object-oriented code:

>>> square = Polygon()
>>> square.add_point(Point(1,1))
>>> square.add_point(Point(1,2))
>>> square.add_point(Point(2,2))
>>> square.add_point(Point(2,1))
>>> square.perimeter()
4.0

That's fairly succinct and easy to read, you might think, but let's compare it to the function-based code:

>>> square = [(1,1), (1,2), (2,2), (2,1)]
>>> perimeter(square)
4.0

Hmm, maybe the object-oriented API isn't so compact! Our first, hacked-in version, without type hints or class definitions, is the shortest. How do we know what the list of tuples is supposed to represent? How do we remember what kind of object we're supposed to pass into the perimeter function? We needed some documentation to explain how the first set of functions should be used.

The functions annotated with type hints were quite a bit easier to understand, as were the class definitions. The relationships among the objects are more clearly defined by hints or classes or both.

Code length is not a good indicator of code complexity. Some programmers get hung up on complicated one-liners that do an incredible amount of work in one line of code. This can be a fun exercise, but the result is often unreadable, even to the original author the following day. Minimizing the amount of code can often make a program easier to read, but do not blindly assume this is the case.

No one wins at code golf. Minimizing the volume of code is rarely desirable.

Luckily, this trade-off isn't necessary. We can make the object-oriented Polygon API as easy to use as the functional implementation. All we have to do is alter our Polygon class so that it can be constructed with multiple points.

Let's give it an initializer that accepts a list of Point objects:

class Polygon_2:
    def __init__(self, vertices: Optional[Iterable[Point]] = None) -> None:
        self.vertices = list(vertices) if vertices else []
    def perimeter(self) -> float:
        pairs = zip(
            self.vertices, self.vertices[1:] + self.vertices[:1])
        return sum(p1.distance(p2) for p1, p2 in pairs)

For the perimeter() method, we've used the zip() function to create pairs of vertices, with items drawn from two lists to create a sequence of pairs. One list provided to zip() is the complete sequence of vertices. The other list of vertices starts from vertex 1 (not 0) and ends with the vertex before 1 (that is, vertex 0). For a triangle, this will make three pairs: (v[0], v[1]), (v[1], v[2]), and (v[2], v[0]). We can then compute the distance between the pairs using Point.distance(). Finally, we sum the sequence of distances. This seems to improve things considerably. We can now use this class like the original hacked-in function definitions:

>>> square = Polygon_2(
... [Point(1,1), Point(1,2), Point(2,2), Point(2,1)]
... )
>>> square.perimeter()
4.0

It's handy to have the details of the individual method definitions. We've built an API that's close to the original, succinct set of definitions. We've added enough formality to be confident the code is likely to work before we even start putting test cases together.

Let's take one more step. Let's allow it to accept tuples too, and we can construct the Point objects ourselves, if needed:

Pair = Tuple[float, float]
Point_or_Tuple = Union[Point, Pair]
class Polygon_3:
    def __init__(self, vertices: Optional[Iterable[Point_or_Tuple]] = None) -> None:
        self.vertices: List[Point] = []
        if vertices:
            for point_or_tuple in vertices:
                self.vertices.append(self.make_point(point_or_tuple))
    @staticmethod
    def make_point(item: Point_or_Tuple) -> Point:
        return item if isinstance(item, Point) else Point(*item)

This initializer goes through the list of items (either Point or Tuple[float, float]) and ensures that any non-Point objects are converted to Point instances.

If you are experimenting with the above code, you should also define these variant class designs by creating subclasses of Polygon and overriding the __init__() method. Extending a class with dramatically different method signatures can raise error flags from mypy.

For an example this small, there's no clear winner between the object-oriented and more data-oriented versions of this code. They all do the same thing. If we have new functions that accept a polygon argument, such as area(polygon) or point_in_polygon(polygon, x, y), the benefits of the object-oriented code become increasingly obvious. Likewise, if we add other attributes to the polygon, such as color or texture, it makes more and more sense to encapsulate that data into a single class.

The distinction is a design decision, but in general, the more important a set of data is, the more likely it is to have multiple functions specific to that data, and the more useful it is to use a class with attributes and methods instead.

When making this decision, it also pays to consider how the class will be used. If we're only trying to calculate the perimeter of one polygon in the context of a much greater problem, using a function will probably be quickest to code and easier to use one time only. On the other hand, if our program needs to manipulate numerous polygons in a wide variety of ways (calculating the perimeter, area, and intersection with other polygons, moving or scaling them, and so on), we have almost certainly identified a class of related objects. The class definition becomes more important as the number of instances increases.

Additionally, pay attention to the interaction between objects. Look for inheritance relationships; inheritance is impossible to model elegantly without classes, so make sure to use them. Look for the other types of relationships we discussed in Chapter 1, Object-Oriented Design, association and composition.

Composition can, technically, be modeled using only data structures – for example, we can have a list of dictionaries holding tuple values – but it is sometimes less complicated to create a few classes of objects, especially if there is behavior associated with the data.

One size does not fit all. The built-in, generic collections and functions work well for a large number of simple cases. A class definition works well for a large number of more complex cases. The boundary is hazy at best.

Adding behaviors to class data with properties

Throughout this book, we've focused on the separation of behavior and data. This is very important in object-oriented programming, but we're about to see that, in Python, the distinction is uncannily blurry. Python is very good at blurring distinctions; it doesn't exactly help us to think outside the box. Rather, it teaches us to stop thinking about the box.

Before we get into the details, let's discuss some bad object-oriented design principles. Many object-oriented developers teach us to never access attributes directly. They insist that we write attribute access like this:

class Color:
    def __init__(self, rgb_value: int, name: str) -> None:
        self._rgb_value = rgb_value
        self._name = name
    def set_name(self, name: str) -> None:
        self._name = name
    def get_name(self) -> str:
        return self._name
    def set_rgb_value(self, rgb_value: int) -> None:
        self._rgb_value = rgb_value
    def get_rgb_value(self) -> int:
        return self._rgb_value

The instance variables are prefixed with an underscore to suggest that they are private (other languages would actually force them to be private). Then, the get and set methods provide access to each variable. This class would be used in practice as follows:

>>> c = Color(0xff0000, "bright red")
>>> c.get_name()
'bright red'
>>> c.set_name("red")
>>> c.get_name()
'red'

The above example is not nearly as readable as the direct access version that Python favors:

class Color_Py:
    def __init__(self, rgb_value: int, name: str) -> None:
        self.rgb_value = rgb_value
        self.name = name

Here's how this class works. It's slightly simpler:

>>> c = Color_Py(0xff0000, "bright red")
>>> c.name
'bright red'
>>> c.name = "red"
>>> c.name
'red'

So, why would anyone insist upon the method-based syntax?

The idea of setters and getters seems helpful for encapsulating the class definitions. Some Java-based tools can generate all the getters and setters automagically, making them almost invisible. Automating their creation doesn't make them a great idea. The most important historical reason for having getters and setters was to make the separate compilation of binaries work out in a tidy way. Without a need to link separately compiled binaries, this technique doesn't always apply to Python.

One ongoing justification for getters and setters is that, someday, we may want to add extra code when a value is set or retrieved. For example, we could decide to cache a value to avoid complex computations, or we might want to validate that a given value is a suitable input.

For example, we could decide to change the set_name() method as follows:

class Color_V:
    def __init__(self, rgb_value: int, name: str) -> None:
        self._rgb_value = rgb_value
        if not name:
            raise ValueError(f"Invalid name {name!r}")
        self._name = name
    def set_name(self, name: str) -> None:
        if not name:
            raise ValueError(f"Invalid name {name!r}")
        self._name = name

If we had written our original code for direct attribute access, and then later changed it to a method like the preceding one, we'd have a problem: anyone who had written code that accessed the attribute directly would now have to change their code to access a method. If they didn't then change the access style from attribute access to a function call, their code would be broken.

The mantra that we should make all attributes private, accessible through methods, doesn't make much sense in Python. The Python language lacks any real concept of private members! We can see the source; we often say "We're all adults here." What can we do? We can make the syntax distinction between attribute and method less visible.

Python gives us the property function to make methods that look like attributes. We can therefore write our code to use direct member access, and if we ever unexpectedly need to alter the implementation to do some calculation when getting or setting that attribute's value, we can do so without changing the interface. Let's see how it looks:

class Color_VP:
    def __init__(self, rgb_value: int, name: str) -> None:
        self._rgb_value = rgb_value
        if not name:
            raise ValueError(f"Invalid name {name!r}")
        self._name = name
    def _set_name(self, name: str) -> None:
        if not name:
            raise ValueError(f"Invalid name {name!r}")
        self._name = name
    def _get_name(self) -> str:
        return self._name
    name = property(_get_name, _set_name)

Compared to the earlier class, we first change the name attribute into a (semi-)private _name attribute. Then, we add two more (semi-)private methods to get and set that variable, performing our validation when we set it.

Finally, we have the property construction at the bottom. This is the Python magic. It creates a new attribute on the Color class called name. It sets this attribute to be a property. Under the hood, a property attribute delegates the real work to the two methods we just created. When used in an access context (on the right side of the = or :=), the first function gets the value. When used in an update context (on the left side of = or :=), the second function sets the value.

This new version of the Color class can be used in exactly the same way as the earlier version, yet it now performs validation when we set the name attribute:

>>> c = Color_VP(0xff0000, "bright red")
>>> c.name
'bright red'
>>> c.name = "red"
>>> c.name
'red'
>>> c.name = ""
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "setting_name_property.py", line 8, in _set_name
    raise ValueError(f"Invalid name {name!r}")
ValueError: Invalid name ''

So, if we'd previously written code to access the name attribute, and then changed it to use our property-based object, the previous code would still work. If it attempts to set an empty property value, this is behavior we wanted to forbid. Success!

Bear in mind that, even with the name property, the previous code is not 100% safe. People can still access the _name attribute directly and set it to an empty string if they want to. But if they access a variable we've explicitly marked with an underscore to suggest it is private, they're the ones that have to deal with the consequences, not us. We established a formal contract, and if they elect to break the contract, they own the consequences.

Properties in detail

Think of the property function as returning an object that proxies any requests to get or set the attribute value through the method names we have specified. The property built-in is like a constructor for such an object, and that object is set as the public-facing member for the given attribute.

This property constructor can actually accept two additional arguments, a delete function and a docstring for the property. The delete function is rarely supplied in practice, but it can be useful for logging the fact that a value has been deleted, or possibly to veto deleting if we have reason to do so. The docstring is just a string describing what the property does, no different from the docstrings we discussed in Chapter 2, Objects in Python. If we do not supply this parameter, the docstring will instead be copied from the docstring for the first argument: the getter method.

Here is a silly example that states whenever any of the methods are called:

class NorwegianBlue:
    def __init__(self, name: str) -> None:
        self._name = name
        self._state: str
    def _get_state(self) -> str:
        print(f"Getting {self._name}'s State")
        return self._state
    def _set_state(self, state: str) -> None:
        print(f"Setting {self._name}'s State to {state!r}")
        self._state = state
    def _del_state(self) -> None:
        print(f"{self._name} is pushing up daisies!")
        del self._state
    silly = property(
        _get_state, _set_state, _del_state, 
        "This is a silly property")

Note that the state attribute has a type hint, str, but no initial value. It can be deleted, and only exists for part of the life of a NorwegianBlue. We need to provide a hint to help mypy understand what the type should be. But we don't assign a default value because that's the job of the setter method.

If we actually use an instance of this class, it does indeed print out the correct strings when we ask it to:

>>> p = NorwegianBlue("Polly")
>>> p.silly = "Pining for the fjords"
Setting Polly's State to 'Pining for the fjords'
>>> p.silly
Getting Polly's State
'Pining for the fjords'
>>> del p.silly
Polly is pushing up daisies!

Further, if we look at the help text for the Silly class (by issuing help(Silly) at the interpreter prompt), it shows us the custom docstring for our silly attribute:

Help on class NorwegianBlue in module colors:
class NorwegianBlue(builtins.object)
 |  NorwegianBlue(name: str) -> None
 |  
 |  Methods defined here:
 |  
 |  __init__(self, name: str) -> None
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  silly
 |      This is a silly property

Once again, everything is working as we planned. In practice, properties are normally only defined with the first two parameters: the getter and setter functions. If we want to supply a docstring for a property, we can define it on the getter function; the property proxy will copy it into its own docstring. The delete function is often left empty because object attributes are so rarely deleted.

Decorators – another way to create properties

We can create properties using decorators. This makes the definitions easier to read. Decorators are a ubiquitous feature of Python syntax, with a variety of purposes. For the most part, decorators modify the function definition that they precede. We'll look at the decorator design pattern more broadly in Chapter 11, Common Design Patterns.

The property function can be used with the decorator syntax to turn a get method into a property attribute, as follows:

class NorwegianBlue_P:
    def __init__(self, name: str) -> None:
        self._name = name
        self._state: str
    @property
    def silly(self) -> str:
        print(f"Getting {self._name}'s State")
        return self._state

This applies the property function as a decorator to the function that follows. It is equivalent to the previous silly = property(_get_state) syntax. The main difference, from a readability perspective, is that we get to mark the silly method as a property at the top of the method, instead of after it is defined, where it can be easily overlooked. It also means we don't have to create private methods with underscore prefixes just to define a property.

Going one step further, we can specify a setter function for the new property as follows:

class NorwegianBlue_P:
    def __init__(self, name: str) -> None:
        self._name = name
        self._state: str
    @property
    def silly(self) -> str:
        """This is a silly property"""
        print(f"Getting {self._name}'s State")
        return self._state
    @silly.setter
    def silly(self, state: str) -> None:
        print(f"Setting {self._name}'s State to {state!r}")
        self._state = state

This syntax, @silly.setter, looks odd compared with @property, although the intent should be clear. First, we decorate the silly method as a getter. Then, we decorate a second method with exactly the same name by applying the setter attribute of the originally decorated silly method! This works because the property function returns an object; this object also has its own setter attribute, which can then be applied as a decorator to other methods. Using the same name for the get and set methods helps to group together the multiple methods that access one common attribute.

We can also specify a delete function with @silly.deleter. Here's what it looks like:

@silly.deleter
def silly(self) -> None:
    print(f"{self._name} is pushing up daisies!")
    del self._state

We cannot specify a docstring using property decorators, so we need to rely on the decorator copying the docstring from the initial getter method. This class operates exactly the same as our earlier version, including the help text. You'll see the decorator syntax in widespread use. The function syntax is how it actually works under the hood.

Deciding when to use properties

With the built-in property blurring the division between behavior and data, it can be confusing to know when to choose an attribute, or a method, or a property. In the Color_VP class example we saw earlier, we added argument value validation to setting an attribute. In the NorwegianBlue class example, we wrote detailed log entries when attributes were set and deleted. There are also other factors to take into account when deciding to use a property.

In Python, data, properties, and methods are all attributes of a class. The fact that a method is callable does not distinguish it from other types of attributes; indeed, we'll see in Chapter 8, The Intersection of Object-Oriented and Functional Programming, that it is possible to create normal objects that can be called like functions. We'll also discover that functions and methods are themselves ordinary objects.

The fact that methods are callable attributes, and properties are also attributes, can help us make this decision. We suggest the following principles:

• Use methods to represent actions; things that can be done to, or performed by, the object. When you call a method, even with only one argument, it should do something. Method names are generally verbs.

• Use attributes or properties to represent the state of the object. These are the nouns, adjectives, and prepositions that describe an object.

  • Default to ordinary (non-property) attributes, initialized in the __init__() method. These must be computed eagerly, which is a good starting point for any design.

  • Use properties for attributes in the exceptional case when there's a computation involved with setting or getting (or deleting) an attribute. Examples include data validation, logging, and access controls. We'll look at cache management in a moment. We can also use properties for lazy attributes, where we want to defer the computation because it's costly and rarely needed.

Let's look at a more realistic example. A common need for custom behavior is caching a value that is difficult to calculate or expensive to look up (requiring, for example, a network request or database query). The goal is to store the value locally to avoid repeated calls to the expensive calculation.

We can do this with a custom getter on the property. The first time the value is retrieved, we perform the lookup or calculation. Then, we can locally cache the value as a private attribute on our object (or in dedicated caching software), and the next time the value is requested, we return the stored data. Here's how we might cache a web page:

from urllib.request import urlopen
from typing import Optional, cast
class WebPage:
    def __init__(self, url: str) -> None:
        self.url = url
        self._content: Optional[bytes] = None
    @property
    def content(self) -> bytes:
        if self._content is None:
            print("Retrieving New Page...")
            with urlopen(self.url) as response:
                self._content = response.read()
        return self._content

We'll only read the website content once, when self._content has the initial value of None. After that, we'll return the value most recently read for the site. We can test this code to see that the page is only retrieved once:

import time
webpage = WebPage("http://ccphillips.net/")
now = time.perf_counter()
content1 = webpage.content
first_fetch = time.perf_counter() - now
now = time.perf_counter()
content2 = webpage.content
second_fetch = time.perf_counter() - now
assert content2 == content1, "Problem: Pages were different"
print(f"Initial Request     {first_fetch:.5f}")
print(f"Subsequent Requests {second_fetch:.5f}")

The output?

% python src/colors.py
Retrieving New Page...
Initial Request     1.38836
Subsequent Requests 0.00001

It took about 1.388 seconds to retrieve a page from the ccphilips.net web host. The second fetch – from a laptop's RAM – takes 0.01 milliseconds! This is sometimes written as 10 μs, 10 microseconds. Since this is the last digit, we can suspect it's subject to rounding, and the time may be only half that, perhaps as little as 5 μs.

Custom getters are also useful for attributes that need to be calculated on the fly, based on other object attributes. For example, we might want to calculate the average for a list of integers:

class AverageList(List[int]):
    @property
    def average(self) -> float:
        return sum(self) / len(self)

This small class inherits from list, so we get list-like behavior for free. We added a property to the class, and – hey, presto! – our list can have an average as follows:

>>> a = AverageList([10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5])
>>> a.average
9.0

Of course, we could have made this a method instead, but if we do, then we ought to call it calculate_average(), since methods represent actions. But a property called average is more suitable, and is both easier to type and easier to read.

We can imagine a number of similar reductions, including minimum, maximum, standard deviation, median, and mode, all being properties of a collection of numbers. This can simplify a more complex analysis by encapsulating these summaries into the collection of data values.

Custom setters are useful for validation, as we've already seen, but they can also be used to proxy a value to another location. For example, we could add a content setter to the WebPage class that automatically logs into our web server and uploads a new page whenever the value is set.

Manager objects

We've been focused on objects and their attributes and methods. Now, we'll take a look at designing higher-level objects; the kind of objects that manage other objects – the objects that tie everything together. These are sometimes called Façade objects because they present a pleasant, easy-to-use façade over some underlying complexity. See Chapter 12, Advanced Design Patterns, for an additional look at the Façade design pattern.

Most of the previous examples tend to model concrete ideas. Management objects are more like office managers; they don't do the actual visible work out on the floor, but without them, there would be no communication between departments, and nobody would know what they are supposed to do (although, this can be true anyway if the organization is badly managed!). Analogously, the attributes on a management class tend to refer to other objects that do the visible work; the behaviors on such a class delegate to those other classes at the right time, and pass messages between them.

A manager relies on composite design. We assemble a manager class by knitting other objects together. The overall behavior of the manager emerges from the interaction of objects. To an extent, a manager is also an Adapter among the various interfaces. See Chapter 12, Advanced Design Patterns, for an additional look at the Adapter design pattern.

As an example, we'll write a program that does a find-and-replace action for text files stored in a compressed archive file, either a ZIP archive or a TAR archive. We'll need objects to represent the archive file as a whole and each individual text file (luckily, we don't have to write these classes, as they're available in the Python standard library).

An overall manager object will be responsible for ensuring the following three steps occur:

1. Unzipping the compressed file to examine each member

2. Performing the find-and-replace action on text members

3. Zipping up the new files with the untouched as well as changed members

Note that we have to choose between an eager and a lazy approach to the three steps of this process. We can eagerly unzip (or untar) the entire archive, process all the files, and then build a new archive. This tends to use a lot of disk space. An alternative is to lazily extract items one at a time from the archive, perform the find-and-replace, and then build a new compressed archive as we go. The lazy approach doesn't require as much storage.

This design will knit together elements of the pathlib, zipfile, and the regular expression (re) module. The initial design will be focused on the job at hand. Later in this chapter, we'll rethink this design as new requirements surface.

The class is initialized with the archive file's name. We don't do anything else upon creation. We'll define a method with a good, clear verb in its name that does any processing:

from __future__ import annotations
import fnmatch
from pathlib import Path
import re
import zipfile
class ZipReplace:
    def __init__(
            self,
            archive: Path,
            pattern: str,
            find: str,
            replace: str
    ) -> None:
        self.archive_path = archive
        self.pattern = pattern
        self.find = find
        self.replace = replace

Given the archive, the filename pattern to match, and the strings to work with, the object will have everything it needs. We might provide arguments like ZipReplace(Path("sample.zip"), "*.md", "xyzzy", "xyzzy").

The overall manager method for the find-and-replace operation revises a given archive. This method of the ZipReplace class (started above) uses two other methods and delegates most of the real work to other objects:

    def find_and_replace(self) -> None:
        input_path, output_path = self.make_backup()
        with zipfile.ZipFile(output_path, "w") as output:
            with zipfile.ZipFile(input_path) as input:
                self.copy_and_transform(input, output)

The make_backup() method will use the pathlib module to rename the old ZIP file so it's obviously the backup copy, untouched. This backup copy is input to the copy_and_transform() method. The original name will be the final output, also. This makes it look like the file was updated "in place." In fact, a new file was created, but the old name will be assigned to the new content.

We create two context managers (a special kind of manager) to control the open files. An open file is entangled with operating system resources. In the case of a ZIP file or TAR archive, there are summaries and checksums that need to be properly written when the file is closed. Using a context manager assures that this additional work is done, and done properly, in spite of any exceptions being raised. All file operations should be wrapped in a with statement to leverage Python's context manager and handle proper cleanup. We'll look at this again in Chapter 9, Strings, Serialization, and File Paths.

The copy_and_transform() method uses methods of the two ZipFile instances and the re module to transform members of the original file. Since a backup was made of the original file, this will build the output file from the backup file. It examines each member of the archive, performing a number of steps, including expanding the compressed data, doing the transformation with a transform() method, and compressing to write to the output file, and then cleaning up the temporary file (and directories).

Obviously, we could do all of these steps in one method of a class, or indeed do the whole thing in one complex script, without ever creating an object. There are several advantages to separating the steps:

• Readability: The code for each step is in a self-contained unit that is easy to read and understand. The method name describes what the method does, and less additional documentation is required to understand what is going on.

• Extensibility: If a subclass wanted to use compressed TAR files instead of ZIP files, it could override the copy_and_transform() method, reusing all the supporting methods because they apply to any file irrespective of the kind of archive.

• Partitioning: An external class could create an instance of this class and use make_backup() or the copy_and_transform() methods directly, bypassing the find_and_replace() manager.

These two methods of the ZipReplace class (started above) make the backup copy and create the new file by reading from the backup and writing new items after they've been modified:

def make_backup(self) -> tuple[Path, Path]:
    input_path = self.archive_path.with_suffix(
        f"{self.archive_path.suffix}.old")
    output_path = self.archive_path
    self.archive_path.rename(input_path)
    return input_path, output_path
def copy_and_transform(
    self, input: zipfile.ZipFile, output: zipfile.ZipFile
) -> None:
    for item in input.infolist():
        extracted = Path(input.extract(item))
        if (not item.is_dir() 
                and fnmatch.fnmatch(item.filename, self.pattern)):
            print(f"Transform {item}")
            input_text = extracted.read_text()
            output_text = re.sub(self.find, self.replace, input_text)
            extracted.write_text(output_text)
        else:
            print(f"Ignore    {item}")
        output.write(extracted, item.filename)
        extracted.unlink()
        for parent in extracted.parents:
            if parent == Path.cwd():
                break
            parent.rmdir()

The make_backup() method applies a common strategy to avoid damaging a file. The original file is renamed to preserve it, and a new file is created that will have the original file's name. This method is designed to be independent of the file type or other processing details.

The copy_and_transform() function method builds the new archive out of members extracted from the original archive. It performs a number of steps for each member of the archive:

• Extract this file from the original archive.

• If the item is not a directory (this is unlikely, but still possible), and the name matches the wild-card pattern, we want to transform it. This is a three-step process.

  1. Read the file's text.

  2. Transform the file, using the sub() function of the re module.

  3. Write the text, replacing the extracted file. This is where we create a copy of the content.

• Compress the file – either an untouched file or a transformed file – into the new archive.

• We unlink the temporary copy. If there are no links left to the file, it will be deleted by the operating system.

• We clean up any temporary directories created by the extraction process.

• The copy_and_transform() method's operations span the pathlib, zipfile, and re modules. Wrapping these operations up into a manager that uses context managers gives us a tidy package with a small interface.

We can create a main script to use the ZipReplace class:

if __name__ == "__main__":
    sample_zip = Path("sample.zip")
    zr = ZipReplace(sample_zip, "*.md", "xyzzy", "plover's egg")
    zr.find_and_replace()

We've provided the archive (sample.zip), the file matching pattern (*.md), the string to replace (xyzzy), and the final replacement (plover's egg). This will perform a complex series of file operations. A more practical approach is to use the argparse module to define the command-line interface (CLI) for this application.

For brevity, the details are sparsely documented. Our current focus is on object-oriented design; if you are interested in the inner details of the zipfile module, refer to the documentation in the standard library, either online or by typing import zipfile and help(zipfile) into your interactive interpreter.

Of course, an instance of the ZipReplace class does not have to be created from the command line; our class could be imported into another module (to perform batch ZIP file processing), or accessed as part of a GUI interface or even a higher-level manager object that knows where to get ZIP files (for example, to retrieve them from an FTP server or back them up to an external disk).

The benefit of the Façade and Adapter design patterns is to encapsulate complexity into a more useful class design. These composite objects tend to be less like physical objects, and enter the realm of conceptual objects. When we step away from objects that have a close parallel with the real world, the methods are actions that change the state of those concepts; care is required because the simple analogies start to disappear in the haze of ideas. It helps when the foundation is a set of concrete data values and well-defined behaviors.

A good example to bear in mind is the World Wide Web. A web server provides content to browsers. The content can include JavaScript that behaves like a desktop application which reaches out to other web servers to present content. These conceptual relationships are implemented by tangible transfers of bytes. It also includes a browser to paint pages of text, images, video, or sound. The foundation is transfers of bytes, a tangible action. In a classroom setting, it's possible to have developers pass sticky notes and rubber balls to each other to represent requests and responses.

This example works nicely. When we're confronted with additional requirements, we need to find a way to build new, related features without duplicating code. We'll talk about this engineering imperative first, then look at the revised design.

Removing duplicate code

Often, the code in management-style classes such as ZipReplace is quite generic and can be applied in a variety of ways. It is possible to use either composition or inheritance to help keep this code in one place, thus eliminating duplicate code. Before we look at any examples of this, let's discuss some design principles. Specifically, why is duplicate code a bad thing?

There are several reasons, but they all boil down to readability and maintainability. When we're writing a new piece of code that is similar to an earlier piece, the easiest thing to do is copy and paste the old code and change whatever needs to be changed (variable names, logic, comments) to make it work in the new location. Alternatively, if we're writing new code that seems similar, but not identical, to code elsewhere in the project, it is often easier to write fresh code with similar behavior, rather than figuring out how to extract the overlapping functionality. We sometimes call this copy-pasta programming because the result is a big mass of tangled noodles of code, like a bowl of spaghetti.

But as soon as someone trying to understand the code comes across duplicate (or nearly duplicate) code blocks, they now have an additional barrier to understanding. There's an intellectual friction created by a number of side-bar questions. Are they truly identical? If not, how is one section different from the other? What parts are the same? Under what conditions is one section called? When do we call the other? You might argue that you're the only one reading your code, but if you don't touch that code for eight months, it will be as incomprehensible to you as it is to a fresh coder. When we're trying to read two similar pieces of code, we have to understand why they're different, as well as how they're different. This wastes the reader's time; code should always be written to be readable first.

[Dusty here, stepping out of formal author mode] I once had to try to understand someone's code that had three identical copies of the same 300 lines of very poorly written code. I had been working with the code for a month before I finally comprehended that the three identical versions were actually performing slightly different tax calculations. Some of the subtle differences were intentional, but there were also obvious areas where someone had updated a calculation in one function without updating the other two. The number of subtle, incomprehensible bugs in the code could not be counted. I eventually replaced all 900 lines with an easy-to-read function of 20 lines or so.

As the preceding story suggests, keeping two similar pieces of code up to date can be a nightmare. We have to remember to update both sections whenever we update one of them, and we have to remember how multiple sections differ so we can modify our changes when we are editing each of them. If we forget to update all sections, we will end up with extremely annoying bugs that usually manifest themselves as: "But I fixed that already, why is it still happening?"

The key factor here is the time spent in troubleshooting, maintenance, and enhancement compared with the time spent initially creating the code. Software that's in use for more than a few weeks will have a lot more eyeballs on it than the time spent creating it. The tiny bit of time we "save" by copying and pasting existing code is more than wasted when we have to maintain it.

One of the author's personal bests was an application that was in use for almost seventeen years. If other developers and users wasted one extra day each year trying to sort out some confusing part of the code, it means the author should have spent at least two more weeks improving the code to head off this future maintenance cost.

Code is both read and modified many more times and much more often than it is written. Comprehensible code should always be a priority.

This is why programmers, especially Python programmers (who tend to value elegant code more than average developers), follow what is known as the Don't Repeat Yourself (DRY) principle. Our advice for beginner programmers is to never use the copy-and-paste feature of their editor. To intermediate programmers: think thrice before hitting Ctrl + C.

But what should we do instead of code duplication? The simplest solution is often to move the code into a function that accepts parameters to account for whatever parts are different. This isn't a strictly object-oriented solution, but it is frequently optimal.

For example, if we have two pieces of code that unzip a ZIP file into two different directories, we can easily replace it with a function that accepts a parameter for the directory to which it should be unzipped. This may make the function itself slightly longer. The size of a function – measured as lines of code – isn't a good metric for readability. No one wins at code golf.

Good names and docstrings are essential. Each class, method, function, variable, property, attribute, module, and package name should be chosen thoughtfully. When writing docstrings, don't explain how the code works (the code should do that). Be sure to focus on what the code's purpose is, what the preconditions are for using it, and what will be true after the function or method has been used.

The moral of the story is: always make the effort to refactor your code to be easier to read, instead of writing bad code that may seem easier to write. Now we can look at the revised design to the ZipReplace class definition.

In practice

Let's explore two ways we can reuse our existing code. After writing our code to replace strings in a ZIP file full of text files, we are later contracted to scale all the images in a ZIP file to a size suitable for mobile devices. While resolutions vary, 640 x 960 is about the smallest we need. It looks like we could use a very similar paradigm to what we used in ZipReplace.

Our first impulse might be to save a copy of that module and change the find_replace method to scale_image or something similar in the copy.

This processing will rely on the Pillow library to open an image file, scale it, and save it. The Pillow image processing tools can be installed with the following command:

% python -m pip install pillow

This will provide some great image-processing tools.

As we noted above in the Removing duplicate code section of this chapter, this copy-and-paste programming approach is suboptimal. What if someday we want to change the unzip and zip methods to also open TAR files? Or maybe we'll want to use a guaranteed unique directory name for temporary files. In either case, we'd have to change it in two different places!

We'll start by demonstrating an inheritance-based solution to this problem. First, we'll modify our original ZipReplace class into a superclass for processing ZIP files in a variety of ways:

from abc import ABC, abstractmethod
class ZipProcessor(ABC):
    def __init__(self, archive: Path) -> None:
        self.archive_path = archive
        self._pattern: str
    def process_files(self, pattern: str) -> None:
        self._pattern = pattern
        input_path, output_path = self.make_backup()
        with zipfile.ZipFile(output_path, "w") as output:
            with zipfile.ZipFile(input_path) as input:
                self.copy_and_transform(input, output)
    def make_backup(self) -> tuple[Path, Path]:
        input_path = self.archive_path.with_suffix(
            f"{self.archive_path.suffix}.old")
        output_path = self.archive_path
        self.archive_path.rename(input_path)
        return input_path, output_path
    def copy_and_transform(
        self, input: zipfile.ZipFile, output: zipfile.ZipFile
    ) -> None:
        for item in input.infolist():
            extracted = Path(input.extract(item))
            if self.matches(item):
                print(f"Transform {item}")
                self.transform(extracted)
            else:
                print(f"Ignore    {item}")
            output.write(extracted, item.filename)
            self.remove_under_cwd(extracted)
    def matches(self, item: zipfile.ZipInfo) -> bool:
        return (
            not item.is_dir() 
            and fnmatch.fnmatch(item.filename, self._pattern))
    def remove_under_cwd(self, extracted: Path) -> None:
        extracted.unlink()
        for parent in extracted.parents:
            if parent == Path.cwd():
                break
            parent.rmdir()
    @abstractmethod
    def transform(self, extracted: Path) -> None:
        ...

We dropped the three parameters to __init__(), pattern, find, and replace, that were specific to ZipReplace. Then, we renamed the find_replace() method to process_files(). We decomposed the complex copy_and_transform() method and made it call several other methods to do the real work. This includes a placeholder for a transform() method. These name changes help demonstrate the more generalized nature of our new class.

This new ZipProcessor class is a subclass of ABC, an abstract base class, allowing us to provide placeholders instead of methods. (More on ABCs to come in Chapter 6, Abstract Base Classes and Operator Overloading.) This abstract class doesn't actually define a transform() method. If we try to create an instance of the ZipProcessor class, the missing transform() method will raise an exception. The formality of an @abstractmethod decoration makes it clear that there's a piece missing, and the piece must have the expected shape.

Now, before we move on to our image processing application, let's create a version of our original ZipReplace class. This will be based on the ZipProcessor class to make use of this parent class, as follows:

class TextTweaker(ZipProcessor):
    def __init__(self, archive: Path) -> None:
        super().__init__(archive)
        self.find: str
        self.replace: str
    def find_and_replace(self, find: str, replace: str) -> "TextTweaker":
        self.find = find
        self.replace = replace
        return self
    def transform(self, extracted: Path) -> None:
        input_text = extracted.read_text()
        output_text = re.sub(self.find, self.replace, input_text)
        extracted.write_text(output_text)

This code is shorter than the original version, since it inherits its ZIP processing abilities from the parent class. We first import the base class we just wrote and make TextTweaker extend that class. Then, we use super() to initialize the parent class.

We need two extra parameters, and we've used a technique called a fluent interface to provide the two parameters. The find_and_replace() method updates the state of the object, then returns the self object. This lets us use the class with a line of code like the following:

TextTweaker(zip_data)\
.find_and_replace("xyzzy", "plover's egg")\
.process_files("*.md")

We've created an instance of the class, used the find_and_replace() method to set some of the attributes, then used the process_files() method to start the processing. This is called a "fluent" interface because a number of methods are used to help clarify the parameters and their relationships.

We've done a fair amount of work to recreate a program that is functionally not different from the one we started with! But having done that work, it is now much easier for us to write other classes that operate on files in a ZIP archive, such as the (hypothetically requested) photo scaler.

Further, if we ever want to improve or bug fix the ZIP functionality, we can do it for all subclasses at once by changing only the one ZipProcessor base class. Therefore maintenance will be much more effective.

See how simple it is now to create a photo scaling class that takes advantage of the ZipProcessor functionality:

from PIL import Image  # type: ignore [import]
class ImgTweaker(ZipProcessor):
    def transform(self, extracted: Path) -> None:
        image = Image.open(extracted)
        scaled = image.resize(size=(640, 960))
        scaled.save(extracted)

Look how simple this class is! All that work we did earlier paid off. All we do is open each file, scale it, and save it back. The ZipProcessor class takes care of the zipping and unzipping without any extra work on our part. This seems to be a huge win.

Creating reusable code isn't easy. It generally requires more than one use case to make it clear what parts are generic and what parts are specific. Because we need concrete examples, it pays to avoid over-engineering to strive for imagined reuse. This is Python and things can be very flexible. Rewrite as needed to cover the cases as they arrive on the scene.

Case study

In this chapter, we'll continue developing elements of the case study. We want to explore some additional features of object-oriented design in Python. The first is what is sometimes called "syntactic sugar," a handy way to write something that offers a simpler way to express something fairly complex. The second is the concept of a manager for providing a context for resource management.

In Chapter 4, Expecting the Unexpected, we built an exception for identifying invalid input data. We used the exception for reporting when the inputs couldn't be used.

Here, we'll start with a class to gather data by reading the file with properly classified training and test data. In this chapter, we'll ignore some of the exception-handling details so we can focus on another aspect of the problem: partitioning samples into testing and training subsets.

Input validation

The TrainingData object is loaded from a source file of samples, named bezdekIris.data. Currently, we don't make a large effort to validate the contents of this file. Rather than confirm the data contains correctly formatted samples with numeric measurements and a proper species name, we simply create Sample instances, and hope nothing goes wrong. A small change to the data could lead to unexpected problems in obscure parts of our application. By validating the input data right away, we can focus on the problems and provide a focused, actionable report back to the user. Something like "Row 42 has an invalid petal_length value of '1b.25'" with the line of data, the column, and the invalid value.

A file with training data is processed in our application via the load() method of TrainingData. Currently, this method requires an iterable sequence of dictionaries; each individual sample is read as a dictionary with the measurements and the classification. The type hint is Iterable[dict[str, str]]. This is one way the csv module works, making it very easy to work with. We'll return to additional details of loading the data in Chapter 8, The Intersection of Object-Oriented and Functional Programming, and Chapter 9, Strings, Serialization, and File Paths.

Thinking about the possibility of alternative formats suggests the TrainingData class should not depend on the dict[str, str] row definition suggested by CSV file processing. While expecting a dictionary of values for each row is simple, it pushes some details into the TrainingData class that may not belong there. Details of the source document's representation have nothing to do with managing a collection of training and test samples; this seems like a place where object-oriented design will help us disentangle the two ideas.

In order to support multiple sources of data, we will need some common rules for validating the input values. We'll need a class like this:

class SampleReader:
    """
    See iris.names for attribute ordering in bezdekIris.data file
    """
    target_class = Sample
    header = [
        "sepal_length", "sepal_width", 
        "petal_length", "petal_width", "class"
    ]
    def __init__(self, source: Path) -> None:
        self.source = source
    def sample_iter(self) -> Iterator[Sample]:
        target_class = self.target_class
        with self.source.open() as source_file:
            reader = csv.DictReader(source_file, self.header)
            for row in reader:
                try:
                    sample = target_class(
                        sepal_length=float(row["sepal_length"]),
                        sepal_width=float(row["sepal_width"]),
                        petal_length=float(row["petal_length"]),
                        petal_width=float(row["petal_width"]),
                    )
                except ValueError as ex:
                    raise BadSampleRow(f"Invalid {row!r}") from ex
                yield sample

This builds an instance of the Sample superclass from the input fields read by a CSV DictReader instance. The sample_iter() method uses a series of conversion expressions to translate input data from each column into useful Python objects. In this example, the conversions are simple, and the implementation is a bunch of float() functions to convert CSV string data into Python objects. We can imagine more complex conversions might be present for other problem domains.

The float() functions – when confronted with bad data – will raise a ValueError. While this is helpful, a bug in a distance formula may also raise a ValueError, leading to possible confusion. It's slightly better for our application to produce unique exceptions; this makes it easier to identify a root cause for a problem.

The target type, Sample, is provided as a class-level variable, target_class. This lets us introduce a new subclass of Sample by making one relatively visible change. This isn't required, but a visible dependency like this provides a way to disentangle classes from each other.

We'll follow Chapter 4, Expecting the Unexpected, and define a unique exception definition. This is a better way to help disentangle our application's errors from ordinary bugs in our Python code:

class BadSampleRow(ValueError):
    pass

To make use of this, we mapped the various float() problems signaled by ValueError exceptions to our application's BadSampleRow exception. This can help someone distinguish between a bad CSV source file and a bad computation due to a bug in a k-NN distance computation. While both can raise ValueError exceptions, the CSV processing exception is wrapped into an application-specific exception to disambiguate the context.

We've done the exception transform by wrapping the creation of an instance of the target class in a try: statement. Any ValueError that's raised here will become a BadSampleRow exception. We've used a raise...from... so that the original exception is preserved to help with the debugging.

Once we have valid input, we have to decide whether the object should be used for training or testing. We'll turn to that problem next.

Input partitioning

The SampleReader class we just introduced uses a variable to identify what kind of objects to create. The target_class variable provides a class to use. Note that we need to be a little careful in the ways we refer to SampleReader.target_class or self.target_class.

A simple expression like self.target_class(sepal_length=, ... etc.) looks like a method evaluation. Except, of course, self.target_class is not a method; it's another class. To make sure Python doesn't assume that self.target_class() refers to a method, we've assigned it to a local variable called target_class. Now we can use target_class(sepal_length=, … etc.) and there's no ambiguity.

This is pleasantly Pythonic. We can create subclasses of this reader to create different kinds of samples from the raw data.

This SampleReader class definition exposes a problem. A single source of raw sample data needs to be partitioned into two separate subclasses of KnownSample; it's either a TrainingSample or a TestingSample. There's a tiny difference in behavior between these two classes. A TestingSample is used to confirm the k-NN algorithm works, and is used to compare an algorithmic classification against the expert Botanist-assigned species. This is not something a TrainingSample needs to do.

Ideally, a single reader would emit a mixture of the two classes. The design so far only allows for instances of a single class to be created. We have two paths forward to provide the needed functionality:

• A more sophisticated algorithm for deciding what class to create. The algorithm would likely include an if statement to create an instance of one object or another.

• A simplified definition of KnownSample. This single class can handle immutable training samples separately from mutable testing samples that can be classified (and reclassified) any number of times.

Simplification seems to be a good idea. Less complexity means less code and fewer places for bugs to hide. The second alternative suggests we can separate three distinct aspects of a sample:

• The "raw" data. This is the core collection of measurements. They are immutable. (We'll address this design variation in Chapter 7, Python Data Structures.)

• The Botanist-assigned species. This is available for training or testing data, but not part of an unknown sample. The assigned species, like the measurements, is immutable.

• An algorithmically assigned classification. This is applied to the testing and unknown samples. This value can be seen as mutable; each time we classify a sample (or reclassify a test sample), the value changes.

This a profound change to the design created so far. Early in a project, this kind of change can be necessary. Way back in Chapters 1 and 2, we decided to create a fairly sophisticated class hierarchy for various kinds of samples. It's time to revisit that design. This won't be the last time we think through this. The essence of good design is to create and dispose of a number of bad designs first.

The sample class hierarchy

We can rethink our earlier designs from several points of view. One alternative is to separate the essential Sample class from the additional features. It seems like we can identify four additional behaviors for each Sample instance, shown in the following table.

We've omitted a detail from the Classified row. Each time we do a classification, a specific hyperparameter is associated with the resulting classified sample. It would be more accurate to say it's a sample classified by a specific Hyperparameter object. But this might be too much clutter.

The distinction between the two cells in the Unknown column is minute. The distinction is so minor as to be essentially irrelevant to most processing. An Unknown sample will be waiting to be classified for – at most – a few lines of code.

If we rethink this, we may be able to create fewer classes and still reflect the object state and behavior changes correctly.

There can be two subclasses of Sample with a separate Classification object. Here's a diagram.

Figure 5.1: Sample class diagram

We've refined the class hierarchy to reflect two essentially different kinds of samples:

• A KnownSample instance can be used for testing or training. The difference between other classes is implemented in the method that does classification. We can make this depend on a purpose attribute, shown with a small square (or sometimes a "-") as a prefix. Python doesn't have private variables, but this marker can be helpful as a design note. The public attributes can be shown with a small circle (or a "+" to save space) as a prefix.

  • When the purpose has a value of Training, the classify() method will raise an exception. The sample cannot be re-classified; that would invalidate the training.

  • When the purpose has a value of Testing, the classify() method will work normally, applying a given Hyperparameter to compute a species.

• An UnknownSample instance can be used for user classification. The classification method here does not depend on the value of the purpose attribute, and always performs classification.

Let's look at implementing these behaviors with the @property decorator we learned about in this chapter. We can use @property to fetch computed values as if they were simple attributes. We can also use @property to define attributes that cannot be set.

The purpose enumeration

We'll start by enumerating a domain of purpose values:

class Purpose(enum.IntEnum):
    Classification = 0
    Testing = 1
    Training = 2

This definition creates a namespace with three objects we can use in our code: Purpose.Classification, Purpose.Testing, and Purpose.Training. For example, we can use if sample.purpose == Purpose.Testing: to identify a testing sample.

We can convert to Purpose objects from input values using Purpose(x) where x is an integer value, 0, 1, or 2. Any other value will raise a ValueError exception. We can convert back to numeric values, also. For example, Purpose.Training.value is 1. This use of numeric codes can fit well with external software that doesn't deal well with an enumeration of Python objects.

We'll decompose the KnownSample subclass of the Sample class into two parts. Here's the first part. We initialize a sample with the data required by the Sample.__init__() method plus two additional values, the purpose numeric code, and the assigned species:

class KnownSample(Sample):
    def __init__(
        self,
        sepal_length: float,
        sepal_width: float,
        petal_length: float,
        petal_width: float,
        purpose: int,
        species: str,
    ) -> None:
        purpose_enum = Purpose(purpose)
        if purpose_enum not in {Purpose.Training, Purpose.Testing}:
            raise ValueError(
                f"Invalid purpose: {purpose!r}: {purpose_enum}"
            )
        super().__init__(
            sepal_length=sepal_length,
            sepal_width=sepal_width,
            petal_length=petal_length,
            petal_width=petal_width,
        )
        self.purpose = purpose_enum
        self.species = species
        self._classification: Optional[str] = None
    def matches(self) -> bool:
        return self.species == self.classification

We validate the purpose parameter's value to be sure it decodes to either Purpose.Training or Purpose.Testing. If the purpose value isn't one of the two allowed values, we'll raise a ValueError exception because the data is unusable.

We've created an instance variable, self._classification, with a leading _ name. This is a convention that suggests the name is not for general use by clients of this class. It's not "private," since there's no notion of privacy in Python. We could call it "concealed" or perhaps "watch out for surprises here."

Instead of a large, opaque wall available in some languages, Python uses a low, decorative floral border that sets this variable apart from the others. You can march right through the floral _ character to look at the value closely, but you probably shouldn't.

Here's the first @property method:

    @property
    def classification(self) -> Optional[str]:
        if self.purpose == Purpose.Testing:
            return self._classification
        else:
            raise AttributeError(f"Training samples have no classification")

This defines a method that will be visible as an attribute name. Here's an example of creating a sample for testing purposes:

>>> from model import KnownSample, Purpose
>>> s2 = KnownSample(
...     sepal_length=5.1, 
...     sepal_width=3.5, 
...     petal_length=1.4, 
...     petal_width=0.2, 
...     species="Iris-setosa", 
...     purpose=Purpose.Testing.value)
>>> s2
KnownSample(sepal_length=5.1, sepal_width=3.5, petal_length=1.4, petal_width=0.2, purpose=1, species='Iris-setosa')
>>> s2.classification is None
True

When we evaluate s2.classification, this will call the method. This function makes sure this is a sample to be used for testing, and returns the value of the "concealed" instance variable self._classification.

If this is a Purpose.Training sample, the property will raise an AttributeError exception because any application that checks the value of the classification for a training sample has a bug in it that needs to be fixed.

Property setters

How do we set the classification? Do we really execute the statement self._classification= h.classify(self)? The answer is no – we can create a property that updates the "concealed" instance variable. This is a bit more complex than the example above:

    @classification.setter
    def classification(self, value: str) -> None:
        if self.purpose == Purpose.Testing:
            self._classification = value
        else:
            raise AttributeError(
                f"Training samples cannot be classified")

The initial @property definition for classification is called a "getter." It gets the value of an attribute. (The implementation uses the __get__() method of a descriptor object that was created for us.) The @property definition for classification also creates an additional decorator, @classification.setter. The method decorated by the setter is used by assignment statements.

Note that the method names for these two properties are both classification. This is the attribute name to be used.

Now a statement like s2.classification = h.classify(self) will change the classification from a particular Hyperparameter object. This assignment statement will use the method to examine the purpose of this sample. If the purpose is testing, the value will be saved. If the purpose is not Purpose.Testing, then attempting to set a classification raises an AttributeError exception, and identifies a place where something's wrong in our application.

Repeated if statements

We have a number of if statements checking for specific Purpose values. This is a suggestion that this design is not optimal. The variant behavior is not encapsulated in a single class; instead, multiple behaviors are combined into a class.

The presence of a Purpose enumeration and if statements to check for the enumerated values is a suggestion that we have multiple classes. The "simplification" here isn't desirable.

In the Input partitioning section of this case study, we suggested there were two paths forward. One was to try and simplify the classes by setting the purpose attribute to separate testing from training data. This seems to have added if statements, without really simplifying the design.

This means we'll have to search for a better partitioning algorithm in a later chapter's case study. For now, we have the capability of creating valid data, but we also have code that's cluttered with if statements. The reader is encouraged to try alternative designs to examine the resulting code to see what seems simpler and easier to read.

Recall

Here are some of the key points in this chapter:

• When we have both data and behavior, this is the sweet spot for object-oriented design. We can leverage Python's generic collections and ordinary functions for many things. When it becomes complex enough that we need to be sure that pieces are all defined together, then we need to start using classes.

• When an attribute value is a reference to another object, the Pythonic approach is to allow direct access to the attribute; we don't write elaborate setter and getter functions. When an attribute value is computed, we have two choices: we can compute it eagerly or lazily. A property lets us be lazy and do the computation just in time.

• We'll often have cooperating objects; the behavior of the application emerges from the cooperation. This can often lead to manager objects that combine behaviors from component class definitions to create an integrated, working whole.

Exercises

We've looked at various ways that objects, data, and methods can interact with each other in an object-oriented Python program. As usual, your first thoughts should be how you can apply these principles to your own work. Do you have any messy scripts lying around that could be rewritten using an object-oriented manager? Look through some of your old code and look for methods that are not actions. If the name isn't a verb, try rewriting it as a property.

Think about code you've written in any language. Does it break the DRY principle? Is there any duplicate code? Did you copy and paste code? Did you write two versions of similar pieces of code because you didn't feel like understanding the original code? Go back over some of your recent code now and see whether you can refactor the duplicate code using inheritance or composition. Try to pick a project you're still interested in maintaining, not code so old that you never want to touch it again. That will help to keep you interested when you do the improvements!

Now, look back over some of the examples we looked at in this chapter. Start with the cached web page example that uses a property to cache the retrieved data. An obvious problem with this example is that the cache is never refreshed. Add a timeout to the property's getter, and only return the cached page if the page has been requested before the timeout has expired. You can use the time module (time.time() - an_old_time returns the number of seconds that have elapsed since an_old_time) to determine whether the cache has expired.

Also look at the inheritance-based ZipProcessor. It might be reasonable to use composition instead of inheritance here. Instead of extending the class in the ZipReplace and ScaleZip classes, you could pass instances of those classes into the ZipProcessor constructor and call them to do the processing part. Implement this.

Which version do you find easier to use? Which is more elegant? What is easier to read? These are subjective questions; the answer varies for each of us. Knowing the answer, however, is important. If you find you prefer inheritance over composition, you need to pay attention that you don't overuse inheritance in your daily coding. If you prefer composition, make sure you don't miss opportunities to create an elegant inheritance-based solution.

Finally, add some error handlers to the various classes we created in the case study. How should one bad sample be handled? Should the model be inoperable? Or should the row be skipped? There are profound data science and statistical consequences to a seemingly small technical implementation choice. Can we define a class that permits either alternative behavior?

In your daily coding, pay attention to the copy and paste commands. Every time you use them in your editor, consider whether it would be a good idea to improve your program's organization so that you only have one version of the code you are about to copy.

Summary

In this chapter, we focused on identifying objects, especially objects that are not immediately apparent; objects that manage and control. Objects should have both data and behaviors, but properties can be used to blur the distinction between the two. The DRY principle is an important indicator of code quality, and inheritance and composition can be applied to reduce code duplication.

In the next chapter, we'll look at Python's methods for defining abstract base classes. This lets us define a class that's a kind of template; it must be extended with subclasses that add narrowly-defined implementation features. This lets us build families of related classes, confident that they will work together properly.

About the Author:

Steven Lott has been programming since computers were large, expensive, and rare. Working for decades in high tech has given him exposure to a lot of ideas and techniques, some bad, but most are helpful to others. Since the 1990s, Steven has been engaged with Python, crafting an array of indispensable tools and applications. His profound expertise has led him to contribute significantly to Packt Publishing, penning notable titles like Mastering Object-Oriented Python (Packt, 2019), Modern Python Cookbook (Packt, 2024), and Python Real-World Projects (Packt, 2023). A self-proclaimed technomad, Steven's unconventional lifestyle sees him residing on a boat, often anchored along the vibrant east coast of the US. He tries to live by the words “Don’t come home until you have a story.”

Lott is currently working on the 5th Ed. of Python Object-Oriented Programming due later this year. If you liked this chapter and want more right now, you can check out Python Object-Oriented Programming, 4th Ed. (Packt, 2021).

Here is what some readers have said:

Everything in Go is executed as a goroutine, either transparently or consciously. Each executable Go program has at least one goroutine, which is used for running the main() function of the main package. Each goroutine is executed on a single OS thread according to the instructions of the Go scheduler, which is responsible for the execution of goroutines—the developer has no control over the amount of memory allocated to a goroutine. The OS scheduler does not dictate how many threads the Go runtime is going to create because the Go runtime will spawn enough threads to ensure that GOMAXPROCS threads are available to run the Go code.

However, goroutines cannot communicate with each other directly. Data sharing in Go is implemented using either channels, local sockets, or shared memory. Channels act as the glue that connects multiple goroutines. On the other hand, channels cannot process data or execute code but they can send data to and receive data from goroutines and have a special purpose like acting as signals or specifying the order of execution for goroutines.

When I first learned about channels, I thought that they were a great idea, much better than shared memory, and I wanted to use channels everywhere! However, nowadays I only use channels when I have no other alternative. Look at the implementation of the concurrent statistical application at the end of the chapter to realize that there exist designs that do not require the use of channels.

Although the use of channels to communicate and synchronize between goroutines is very typical and expected, channels might introduce deadlocks, overhead, and complexity to the design, as well as performance considerations, especially when low-latency communication is a priority.

When you combine multiple channels and goroutines, you can create data flows, which, in Go terminology, are called pipelines. So, you might have a goroutine that reads data from a database and sends it to a channel and a second goroutine that reads from that channel, processes that data, and sends it to another channel to be read from another goroutine, before making modifications to the data and storing it in another database.

This chapter covers:

• Processes, threads, and goroutines

• The Go scheduler

• Goroutines

• Channels

• Race conditions are bad

• The select keyword

• Timing out a goroutine

• Go channels revisited

• Handling UNIX signals

• Shared memory and shared variables

• Closured variables and the go statement

• The context package

• The semaphore package

• Making the statistics application concurrent

Go Concurrency

The key component of the Go concurrency model is the goroutine, which is the minimum executable entity in Go. To create a new goroutine, we must use the go keyword followed by a function call or an anonymous function—the two methods are equivalent. For a goroutine or a function to terminate the entire Go application, it should call os.Exit() instead of return. However, most of the time, we exit a goroutine or a function using return because what we really want is to exit that specific goroutine or function and not stop the entire application.

Everything in Go is executed as a goroutine, either transparently or consciously. Each executable Go program has at least one goroutine, which is used for running the main() function of the main package. Each goroutine is executed on a single OS thread according to the instructions of the Go scheduler, which is responsible for the execution of goroutines—the developer has no control over the amount of memory allocated to a goroutine. The OS scheduler does not dictate how many threads the Go runtime is going to create because the Go runtime will spawn enough threads to ensure that GOMAXPROCS threads are available to run the Go code.

However, goroutines cannot communicate with each other directly. Data sharing in Go is implemented using either channels, local sockets, or shared memory. Channels act as the glue that connects multiple goroutines. On the other hand, channels cannot process data or execute code but they can send data to and receive data from goroutines and have a special purpose like acting as signals or specifying the order of execution for goroutines.

When I first learned about channels, I thought that they were a great idea, much better than shared memory, and I wanted to use channels everywhere! However, nowadays I only use channels when I have no other alternative. Look at the implementation of the concurrent statistical application at the end of the chapter to realize that there exist designs that do not require the use of channels.

Although the use of channels to communicate and synchronize between goroutines is very typical and expected, channels might introduce deadlocks, overhead, and complexity to the design, as well as performance considerations, especially when low-latency communication is a priority.

When you combine multiple channels and goroutines, you can create data flows, which, in Go terminology, are called pipelines. So, you might have a goroutine that reads data from a database and sends it to a channel and a second goroutine that reads from that channel, processes that data, and sends it to another channel to be read from another goroutine, before making modifications to the data and storing it in another database.

This chapter covers:

• Processes, threads, and goroutines

• The Go scheduler

• Goroutines

• Channels

• Race conditions are bad

• The select keyword

• Timing out a goroutine

• Go channels revisited

• Handling UNIX signals

• Shared memory and shared variables

• Closured variables and the go statement

• The context package

• The semaphore package

• Making the statistics application concurrent

Processes, threads, and goroutines

A process is an OS representation of a running program, while a program is a binary file on a disk that contains all the information necessary for creating an OS process. The binary file is written in a specific format and contains all the instructions the CPU is going to run, as well as a plethora of other required sections. That program is loaded into memory and the instructions are executed, creating a running process. So, a process carries with it additional resources such as memory, opened file descriptions, and user data, as well as other types of resources that are obtained during runtime.

A thread is a smaller and lighter entity than a process. Processes consist of one or more threads that have their own flow of control and stack. A quick and simplistic way to differentiate a thread from a process is to consider a process as the running binary file and a thread as a subset of a process.

A goroutine is the minimum Go entity that can be executed concurrently. The use of the word minimum is very important here, as goroutines are not autonomous entities like UNIX processes—goroutines live in OS threads that live in OS processes. The good thing is that goroutines are lighter than threads, which, in turn, are lighter than processes—running thousands or hundreds of thousands of goroutines on a single machine is not a problem. Among the reasons that goroutines are lighter than threads are that they have a smaller stack that can grow, they have a faster startup time, and they can communicate with each other through channels with low latency. In practice, this means that a process can have multiple threads and lots of goroutines, whereas a goroutine needs the environment of a process to exist. So, to create a goroutine, you need to have a process with at least one thread. The OS takes care of the process and thread scheduling, while Go creates the necessary threads and the developer creates the desired number of goroutines.

Now that you know the basics of processes, programs, threads, and goroutines, let us talk a little bit about the Go scheduler.

The Go scheduler

The OS kernel scheduler is responsible for the execution of the threads of a program. Similarly, the Go runtime has its own scheduler, which is responsible for the execution of the goroutines using a technique known as m:n scheduling, where m goroutines are executed using n OS threads using multiplexing. The Go scheduler is the Go component responsible for the way and the order in which the goroutines of a Go program get executed. This makes the Go scheduler a really important part of the Go programming language. The Go scheduler is also executed as a goroutine.

Be aware that as the Go scheduler only deals with the goroutines of a single program, its operation is much simpler, cheaper, and faster than the operation of the OS kernel scheduler.

Go uses the fork-join concurrency model. The fork part of the model, which should not be confused with the fork(2) system call, states that a child branch can be created at any point of a program. Analogously, the join part of the Go concurrency model is where the child branch ends and joins with its parent. Keep in mind that both sync.Wait() statements and channels that collect the results of goroutines are join points, whereas each new goroutine creates a child branch.

The fair scheduling strategy shares all load evenly among the available processors. At first, this might look like the perfect strategy because it does not have to take many things into consideration while keeping all processors equally occupied. However, it turns out that this is not exactly the case because most distributed tasks usually depend on other tasks. Therefore, some processors are underutilized or, equivalently, some processors are utilized more than others.

A goroutine is a task, whereas everything after the calling statement of a goroutine is a continuation. In the work-stealing strategy used by the Go scheduler, a (logical) processor that is underutilized looks for additional work from other processors. When it finds such jobs, it steals them from the other processor or processors, hence the name. Additionally, the work-stealing algorithm of Go queues and steals continuations. A stalling join, as is suggested by its name, is a point where a thread of execution stalls at a join and starts looking for other work to do.

Although both task stealing and continuation stealing have stalling joins, continuations happen more often than tasks; therefore, the Go scheduling algorithm works with continuations rather than tasks. The main disadvantage of continuation stealing is that it requires extra work from the compiler of the programming language. Fortunately, Go provides that extra help and, therefore, uses continuation stealing in its work-stealing algorithm. One of the benefits of continuation stealing is that you get the same results when using function calls instead of goroutines or a single thread with multiple goroutines. This makes perfect sense, as only one thing is executed at any given point in both cases.

The Go scheduler works using three main kinds of entities: OS threads (M), which are related to the OS in use, goroutines (G), and logical processors (P). The number of processors that can be used by a Go program is specified by the value of the GOMAXPROCS environment variable—at any given time, there are, at most, GOMAXPROCS processors. Now, let us return to the m:n scheduling algorithm used in Go. Strictly speaking, at any time, you have m goroutines that are executed and, therefore, scheduled to run, on n OS threads using, at most, GOMAXPROCS number of logical processors. You will learn more about GOMAXPROCS shortly.

Each goroutine can be in one of the following three stages: executing, runnable, or waiting. In the executing stage, the instructions of the goroutine are executed on an OS thread. In the runnable stage, the goroutine waits to be assigned to an OS thread for execution. Finally, in the waiting stage, the goroutine is blocked for some reason like waiting for a resource or a mutex to become available to go into one of the other two stages.

The following figure shows that there are two different kinds of queues—a global run queue and a local run queue—attached to each logical processor. Goroutines from the global queue are assigned to the queue of a logical processor in order to get executed at some point in the future.

Figure 8.1: The operation of the Go scheduler

Each logical processor can have multiple threads, and the stealing occurs between the local queues of the available logical processors. Finally, keep in mind that the Go scheduler is allowed to create more OS threads when needed. OS threads are expensive in terms of resources and going from one status to another (context switching), which means that dealing too much with OS threads might slow down your Go applications.

Next, we discuss the meaning and the use of GOMAXPROCS.

The GOMAXPROCS environment variable

The GOMAXPROCS environment variable allows you to set the number of OS threads that can execute user-level Go code simultaneously; this does not limit the number of threads created but it does limit the number of threads that are actively running. Starting with Go version 1.5, the default value of GOMAXPROCS should be the number of logical cores available in your machine. There is also the runtime.GOMAXPROCS() function, which allows you to set and get the value of GOMAXPROCS programmatically.

If you decide to assign a value to GOMAXPROCS that is smaller than the number of cores in your machine, you might affect the performance of your program. However, using a GOMAXPROCS value that is larger than the number of available cores does not necessarily make your Go programs run faster due to the context switching of threads.

As mentioned earlier in this subsection, you can programmatically set and get the value of the GOMAXPROCS environment variable—this is illustrated in maxprocs.go, which will also show additional capabilities of the runtime package. The main() function is implemented as follows:

func main() {
    fmt.Print("You are using ", runtime.Compiler, " ")
    fmt.Println("on a", runtime.GOARCH, "machine")
    fmt.Println("Using Go version", runtime.Version())

The runtime.Compiler variable holds the compiler toolchain used for building the running binary. The two most well-known values are gc and gccgo. The runtime.GOARCH variable holds the current architecture and runtime.Version() returns the current version of the Go compiler. This information is not necessary for using runtime.GOMAXPROCS() but it is good to have a better knowledge of your system.

    fmt.Printf("GOMAXPROCS: %d\n", runtime.GOMAXPROCS(0))
}

What happens with the runtime.GOMAXPROCS(0) call? It always returns the previous value of the maximum number of CPUs that can be executed simultaneously. When the parameter of runtime.GOMAXPROCS() is equal to or bigger than 1, then runtime.GOMAXPROCS() also changes the current setting. As we are using 0, our call does not alter the current setting.

Running maxprocs.go produces the following output:

You are using gc on a arm64 machine
Using Go version go1.21.0
GOMAXPROCS: 10

You can change the value of GOMAXPROCS on the fly using the following technique:

$ GOMAXPROCS=100; go run maxprocs.go
You are using gc on a amd64 machine
Using Go version go1.21.0
GOMAXPROCS: 100

The previous command temporarily changes the value of GOMAXPROCS to 100 and runs maxprocs.go.

Apart from testing the performance of your code using fewer cores, you will most likely not need to change GOMAXPROCS. The next subsection will explain the similarities and differences between concurrency and parallelism.

Concurrency and parallelism

It is a common misconception that concurrency is the same thing as parallelism—this is not true! Parallelism is the simultaneous execution of multiple entities of some kind, whereas concurrency is a way of structuring your components so that they can be executed independently when possible.

It is only when you build software components concurrently that you can safely execute them in parallel, when and if your OS and your hardware permit it. The Erlang programming language did this a long time ago—long before CPUs had multiple cores and computers had lots of RAM.

In a valid concurrent design, adding concurrent entities makes the whole system run faster because more things can be executed in parallel. So, the desired parallelism comes from a better concurrent expression and implementation of the problem. The developer is responsible for taking concurrency into account during the design phase of a system and will benefit from a potential parallel execution of the components of the system. So, the developer should not think about parallelism but about breaking things into independent components that solve the initial problem when combined.

Even if you cannot run your functions in parallel on your machine, a valid concurrent design still improves the design, data flow, and maintainability of your programs. In other words, concurrency is better than parallelism! Let us now talk about goroutines before looking into channels, which are the main components of the Go concurrency model.

Goroutines

You can define, create, and execute a new goroutine using the go keyword followed by a named function or an anonymous function call. The go keyword makes the function call return immediately, while the function starts running in the background as a goroutine and the rest of the program continues its execution. You cannot control or make any assumptions about the order in which your goroutines are going to be executed because that depends on the scheduler of the OS, the Go scheduler, and the load of the OS.

Creating a goroutine

In this subsection, you will learn how to create goroutines. The program that illustrates the technique is called create.go. The implementation of the main() function is as follows:

func main() {
    go func(x int) {
        fmt.Printf("%d ", x)
    }(10)

This is how you run an anonymous function as a goroutine. The (10) at the end is how you pass a parameter to an anonymous function. The previous anonymous function just prints a value. In general, it is more readable to pass parameters explicitly than to have the function close over the variables it uses.

    go printme(15)

This is how you execute a function as a goroutine. As a rule of thumb, the functions that you execute as goroutines should not return any values directly. Exchanging data with goroutines happens via the use of shared memory or channels or some other mechanism.

    time.Sleep(time.Second)
    fmt.Println("Exiting...")
}

As a Go program does not wait for its goroutines to end before exiting, we need to delay it manually, which is the purpose of the time.Sleep() call. We correct that shortly in order to wait for all goroutines to finish before exiting.

Running create.go produces the following output:

$ go run create.go 
10 * 15
Exiting...

The 10 part in the output is from the anonymous function, whereas the * 15 part is from the go printme(15) statement. However, if you run create.go more than once, you might get a different output because the two goroutines are not always executed in the same order, as this depends on the Go scheduler:

$ go run create.go
* 15
10 Exiting...

The next subsection shows how to run a variable number of goroutines.

Creating multiple goroutines

In this subsection, you will learn how to create a variable number of goroutines. The program that illustrates the technique is called multiple.go. The number of goroutines is given as a command line argument to the program. The important code from the implementation of the main() function is the following:

fmt.Printf("Going to create %d goroutines.\n", count)
for i := 0; i < count; i++ {

There is nothing prohibiting you from using a for loop to create multiple goroutines, especially when you want to create lots of them.

    go func(x int) {
        fmt.Printf("%d ", x)
    }(i)
}
time.Sleep(time.Second)
fmt.Println("\nExiting...")

Once again, time.Sleep() prevents the main() function from exiting immediately.

Running multiple.go generates the following kind of output:

$ go run multiple.go 15
Going to create 15 goroutines.
3 0 8 4 5 6 7 11 9 12 14 13 1 2 10 
Exiting...

If you run multiple.go many times, you are going to get different outputs. So, there is still room for improvement. The next subsection shows how to remove the call to time.Sleep() and make your programs wait for the goroutines to finish.

Waiting for all goroutines to finish

It is not enough to create multiple goroutines—you also need to wait for them to finish before the main() function ends. Therefore, this subsection shows a very popular technique that improves the code of multiple.go—the improved version is called varGoroutines.go. But first, we need to explain how this works.

The synchronization process begins by defining a sync.WaitGroup variable and using the Add(), Done(), and Wait() methods. If you look at the source code of the sync Go package, and more specifically at the waitgroup.go file, you see that the sync.WaitGroup type is nothing more than a structure with two fields:

type WaitGroup struct {
    noCopy noCopy
    state1 [3]uint32
}

Each call to sync.Add() increases a counter in the state1 field, which is an array with three uint32 elements. Notice that it is really important to call sync.Add() before the go statement in order to prevent any race conditions—we will learn about race conditions in the Race conditions are bad section. When each goroutine finishes its job, the sync.Done() function should be executed in order to decrease the same counter by one. Behind the scenes, sync.Done() runs an Add(-1) call. The Wait() method waits until that counter becomes 0 in order to return. The return of Wait() inside the main() function means that main() is going to return and the program ends.

You can call Add() with a positive integer value other than 1 in order to avoid calling Add(1) multiple times. This can be handy when you know the number of goroutines you are going to create in advance. Done() does not support that functionality.

The important part of varGoroutines.go is the following:

var waitGroup sync.WaitGroup
fmt.Printf("%#v\n", waitGroup)

This is where you create a sync.WaitGroup variable that you are going to use. The fmt.Printf() call prints the contents of the sync.WaitGroup structure—you do not usually do that but it is good for learning more about the sync.WaitGroup structure.

for i := 0; i < count; i++ {
    waitGroup.Add(1)

We call Add(1) just before we create the goroutine in order to avoid race conditions.

    go func(x int) {
        defer waitGroup.Done()

The Done() call is going to be executed just before the anonymous function returns because of the defer keyword.

        fmt.Printf("%d ", x)
    }(i)
}
fmt.Printf("%#v\n", waitGroup)
waitGroup.Wait()

The Wait() function waits for the counter in the waitGroup variable to become 0 before it returns, which is what we want to achieve.

fmt.Println("\nExiting...")

When the Wait() function returns, the fmt.Println() statement is executed. No need to call time.Sleep() any more!

Running varGoroutines.go produces the following output:

$ go run varGoroutines.go 15
Going to create 10 goroutines.
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0x0}, sema:0x0}
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0xa00000000}, sema:0x0}
14 8 9 10 11 5 0 4 1 2 3 6 13 12 7 
Exiting...

Remember that using more goroutines in a program is not a panacea for performance, as more goroutines, in addition to the various calls to sync.Add(), sync.Wait(), and sync.Done(), might slow down your program due to the extra housekeeping that needs to be done by the Go scheduler and the Go garbage collector.

What if the number of Add() and Done() calls differ?

When the number of sync.Add() calls and sync.Done() calls are equal, everything is going to be fine in your programs. However, this subsection tells you what happens when these two numbers do not agree with each other.

Without giving any command line parameters to addDone.go, the number of Add() calls is going to be smaller than the number of Done() calls. With at least one command line argument, the number of Done() calls is going to be smaller than the number of Add() calls. You can look at the Go code of addDone.go on your own. What is important is the output it generates. Running addDone.go without any command line arguments produces the following error message:

$ go run addDone.go
Going to create 20 goroutines.
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0x0}, sema:0x0}
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0x1300000000}, sema:0x0}
19 3 4 5 6 7 8 9 10 11 12 13 14 15 16 2 1 17 18 0
Exiting...
panic: sync: negative WaitGroup counter
goroutine 20 [running]:
sync.(*WaitGroup).Add(0x1?, 0x1?)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/waitgroup.go:62 +0x108
sync.(*WaitGroup).Done(0x0?)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/waitgroup.go:87 +0x20
main.main.func1(0x0?)
    ~/go/src/github.com/mactsouk/mGo4th/ch08/addDone.go:26 +0x9c
created by main.main in goroutine 1
    ~/go/src/github.com/mactsouk/mGo4th/ch08/addDone.go:23 +0xec
exit status 2

The cause of the error message can be found in the output: panic: sync: negative WaitGroup counter.

Sometimes, addDone.go does not produce any error messages and terminates just fine—this mainly happens when the system is already busy. This is an issue with concurrent programs in general—they do not always crash or misbehave as the order of execution can change, which might change the behavior of the program. This makes the debugging of concurrent software even more difficult.

Running addDone.go with one command line argument produces the following error message:

$ go run addDone.go 1
Going to create 20 goroutines.
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0x0}, sema:0x0}
sync.WaitGroup{noCopy:sync.noCopy{}, state:atomic.Uint64{_:atomic.noCopy{}, _:atomic.align64{}, v:0x1500000000}, sema:0x0}
19 1 2 11 12 13 14 15 16 17 18 6 3 4 5 8 7 9 0 10 fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x0?)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/runtime/sema.go:62 +0x2c
sync.(*WaitGroup).Wait(0x14000128030)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/waitgroup.go:116 +0x78
main.main()
    ~/go/src/github.com/mactsouk/mGo4th/code/ch08/addDone.go:38 +0x230
exit status 2

Once again, the reason for the crash is printed on the screen: fatal error: all goroutines are asleep - deadlock!. This means that the program should wait indefinitely for a goroutine to finish—that is, for a Done() call that is never going to happen.

Creating multiple files with goroutines

As a practical example of the use of goroutines, this subsection presents a command line utility that creates multiple files populated with randomly generated data—such files can be used for testing file systems or generating data for testing. The crucial code of randomFiles.go is the following:

var waitGroup sync.WaitGroup
for i := start; i <= end; i++ {
    waitGroup.Add(1)
    
    go func(n int) {
        filepath := filepath.Join(path, fmt.Sprintf("%s%d", filename, n))
        defer waitGroup.Done()
        createFile(filepath)
    }(i)
}
waitGroup.Wait()

We first create a sync.WaitGroup variable in order to wait for all goroutines to finish in the right way. Each file is created by a single goroutine only. What is important here is that each file has a unique filename—this is implemented with the filepath variable that contains the value of the for loop counter. Multiple createFile() functions executed as goroutines create the files. This is a simple yet very efficient way of creating multiple files. Running randomFiles.go generates the following output:

$ go run randomFiles.go 
Usage: randomFiles firstInt lastInt filename directory

So, the utility requires four parameters, which are the first and last values of the for loop as well as the filename and the directory in which the files are going to be written. So, let us run the utility with the correct number of parameters:

$ go run randomFiles.go 3 5 masterGo /tmp
/tmp/masterGo3 created!
/tmp/masterGo5 created!
/tmp/masterGo4 created!

Everything looks fine, and four files have been created according to our instructions! Now that we know about goroutines, let us continue with channels.

Channels

A channel is a communication mechanism that, among other things, allows goroutines to exchange data. Firstly, each channel allows the exchange of a particular data type, which is also called the element type of the channel, and secondly, for a channel to operate properly, you need someone to receive what is sent via the channel. You should declare a new channel using make() and the chan keyword (make(chan int)), and you can close a channel using the close() function. You can declare the size of a channel by writing something similar to make(chan int, 1). This statement creates a buffered channel that has a different use—buffered channels are explained later in this chapter.

Just because we can use channels, it does not mean that we should. If a simpler solution exists that allows goroutines to get executed and save the generated information, we should also consider that. The purpose of every developer should be to create a simple design, not to use all the features of a programming language.

A pipeline is a virtual method for connecting goroutines and channels so that the output of one goroutine becomes the input of another goroutine using channels to transfer your data. One of the benefits that you get from using pipelines is that there is a constant data flow in your program, as no goroutine or channel has to wait for everything to be completed in order to start their execution. Additionally, you use fewer variables and, therefore, less memory space because you do not have to save everything as a variable. Finally, the use of pipelines simplifies the design of the program and improves its maintainability.

Writing to and reading from a channel

Writing a value (val) to a channel (ch) is as easy as writing ch <- val. The arrow shows the direction of the value, and you will have no problem with this statement as long as both var and ch are of the same data type.

You can read a single value from a channel named c by executing <-c. In this case, the direction is from the channel to the outer world. You can save that value into a new variable using aVar := <-c.

Both channel reading and writing are illustrated in channels.go, which comes with the following code:

package main
import (
    "fmt"
    "sync"
)
func writeToChannel(c chan int, x int) {
    c <- x
    close(c)
}

This function just writes a value to the channel and immediately closes it.

func printer(ch chan bool) {
    ch <- true
}

This function just sends the true value to a bool channel.

func main() {
    c := make(chan int, 1)

This channel is buffered with a size of 1. This means that as soon as we fill that buffer, we can close the channel and the goroutine is going to continue its execution and return. A channel that is unbuffered has a different behavior: when you try to send a value to that channel, it blocks forever because it is waiting for someone to fetch that value. In this case, we definitely want a buffered channel in order to avoid any blocking.

    var waitGroup sync.WaitGroup
    waitGroup.Add(1)
    go func(c chan int) {
        defer waitGroup.Done()
        writeToChannel(c, 10)
        fmt.Println("Exit.")
    }(c)
    fmt.Println("Read:", <-c)

Here, we read from the channel and print the value without storing it in a separate variable.

    _, ok := <-c
    if ok {
        fmt.Println("Channel is open!")
    } else {
        fmt.Println("Channel is closed!")
    }

The previous code shows a technique for determining whether a channel is closed or not. In this case, we are ignoring the read value—if the channel was open, then the read value would be discarded.

    waitGroup.Wait()
    var ch chan bool = make(chan bool)
    for i := 0; i < 5; i++ {
        go printer(ch)
    }

Here, we make an unbuffered channel and create five goroutines without any synchronization as we do not use any Add() calls.

    // Range on channels
    // IMPORTANT: As the channel ch is not closed,
    // the range loop does not exit on its own.
    n := 0
    for i := range ch {

The range keyword works with channels! However, a range loop on a channel only exits when the channel is closed or uses the break keyword.

        fmt.Println(i)
        if i == true {
            n++
        }
        if n > 2 {
            fmt.Println("n:", n)
            close(ch)
            break
        }
    }

We close the ch channel when a condition is met and exit the for loop using break. Note that it is never a good idea to close a channel on the receiving end—this is presented here for the sake of the example. You are going to see the consequences of this decision in a while.

    for i := 0; i < 5; i++ {
        fmt.Println(<-ch)
    }
}

When trying to read from a closed channel, we get the zero value of its data type, so this for loop works just fine and does not cause any issues.

Running channels.go generates the following output:

Exit.
Read: 10

After writing the value 10 to the channel using writeToChannel(c, 10), we read that value back.

Channel is closed!
true
true
true

The for loop with the range exits after three iterations—each iteration prints true on screen.

n: 3
false
false
false
false
false

These five false values are printed by the last for loop of the program.

Although everything looks fine with channels.go, there is a logical issue with it, which we will explain and resolve in the Race conditions are bad section. Additionally, if we run channels.go multiple times, it might crash. However, most of the time, it does not, which makes debugging even more challenging.

Receiving from a closed channel

Reading from a closed channel returns the zero value of its data type. However, if you try to write to a closed channel, your program is going to crash in a bad way (panic). These two situations are explored in readCloseCh.go and, more specifically, in the implementation of the main() function:

func main() {
    willClose := make(chan complex64, 10)

If you make that an unbuffered channel, the program is going to crash.

    // Write some data to the channel
    willClose <- -1
    willClose <- 1i

We write two values to the willClose channel.

    // Read data and empty channel
    <-willClose
    <-willClose
    close(willClose)

Then, we read and discard these two values and we close the channel.

    // Read again - this is a closed channel
    read := <-willClose
    fmt.Println(read)
}

The last value that we read from the channel is the zero value of a complex64 data type. Running readCloseCh.go generates the following output:

(0+0i)

So, we got back the zero value of the complex64 data type. Now let us continue and discuss how to work with functions that accept channels as parameters.

Channels as function parameters

When using a channel as a function parameter, you can specify its direction—that is, whether it is going to be used for sending or receiving data only. In my opinion, if you know the purpose of a channel in advance, you should use this capability because it makes your programs more robust. You will not be able to send data accidentally to a channel from which you should only receive data, or receive data from a channel to which you should only be sending data.

If you declare that a channel function parameter is going to be used for reading only and you try to write to it, you are going to get a compile error message that will most likely save you from nasty bugs in the future. This is the major benefit of this approach!

All these are illustrated in channelFunc.go—the implementation of the functions that accept channel parameters are the following:

func printer(ch chan<- bool) {
    ch <- true
}

The above function accepts a channel parameter that is available for writing only.

func writeToChannel(c chan<- int, x int) {
    fmt.Println("1", x)
    c <- x
    fmt.Println("2", x)
}

The channel parameter of the above function is available for reading only.

func f2(out <-chan int, in chan<- int) {
    x := <-out
    fmt.Println("Read (f2):", x)
    in <- x
    return
}

The last function accepts two channel parameters. However, out is available for reading, whereas in is offered for writing. If you try to perform an operation on a channel parameter that is not allowed, the Go compiler is going to complain. This happens even if the function is not being used.

The subject of the next section is race conditions—read it carefully in order to avoid undefined behaviors and unpleasant situations when working with multiple goroutines.

Race conditions are bad

A data race condition is a situation where two or more running elements, such as threads and goroutines, try to take control of or modify a shared resource or shared variable of a program. Strictly speaking, a data race occurs when two or more instructions access the same memory address, where at least one of them performs a write (change) operation. If all operations are read operations, then there is no race condition. In practice, this means that you might get different output if you run your program multiple times, and that is a bad thing.

Using the -race flag when running or building Go source files executes the Go race detector, which makes the compiler create a modified version of a typical executable file. This modified version can record all accesses to shared variables as well as all synchronization events that take place, including calls to sync.Mutex and sync.WaitGroup, which are presented later on in this chapter. After analyzing the relevant events, the race detector prints a report that can help you identify potential problems so that you can correct them.

The Go race detector

You can run the race detector tool with go run -race. If we test channels.go using go run -race, we are going to get the following output:

$ go run -race channels.go 
Exit.
Read: 10
Channel is closed!
true
true
true
n: 3
==================
WARNING: DATA RACE
Write at 0x00c000094010 by main goroutine:
  runtime.recvDirect()
      /opt/homebrew/Cellar/go/1.21.0/libexec/src/runtime/chan.go:348 +0x7c
  main.main()
      ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:54 +0x444
Previous read at 0x00c000094010 by goroutine 10:
  runtime.chansend1()
      /opt/homebrew/Cellar/go/1.21.0/libexec/src/runtime/chan.go:146 +0x2c
  main.printer()
      ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:14 +0x34
  main.main.func3()
      ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:40 +0x34
Goroutine 10 (running) created at:
  main.main()
      ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:40 +0x2b8
==================
false
false
false
false
false
panic: send on closed channel
goroutine 36 [running]:
main.printer(0x0?)
    ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:14 +0x38
created by main.main in goroutine 1
    ~/go/src/github.com/mactsouk/mGo4th/ch08/channels.go:40 +0x2bc
exit status 2

Therefore, although channels.go looks fine at first, there is a race condition waiting to happen. Let us now discuss where the problem with channels.go lies based on the previous output. There is a closing of a channel at channels.go on line 54, and there is a write to the same channel on line 14 that looks to be the root of the race condition situation.

Line 54 is close(ch), whereas line 14 is ch <- true. The issue is that we cannot be sure about what is going to happen and in which order—this is the race condition. If you execute channels.go without the race detector, it might work, but if you try it multiple times, you might get a panic: send on closed channel error message—this mainly has to do with the order in which the Go scheduler is going to run the goroutines of the program. So, if the closing of the channel happens first, then writing to that channel is going to fail—race condition!

Fixing channels.go requires changing the code and, more specifically, the implementation of the printer() function. The corrected version of channels.go is named chRace.go and comes with the following code:

func printer(ch chan<- bool, times int) {
    for i := 0; i < times; i++ {
        ch <- true
    }
    close(ch)
}
func main() {
    // This is an unbuffered channel
    var ch chan bool = make(chan bool)
    // Write 5 values to channel with a single goroutine
    go printer(ch, 5)
    // IMPORTANT: As the channel ch is closed,
    // the range loop is going to exit on its own.
    for val := range ch {
        fmt.Print(val, " ")
    }
    fmt.Println()
    for i := 0; i < 15; i++ {
        fmt.Print(<-ch, " ")
    }
    fmt.Println()
}

The first thing to notice is that instead of using multiple goroutines for writing to the desired channel, we use a single goroutine. A single goroutine writing to a channel followed by the closing of that channel cannot create any race conditions because things happen sequentially.

Running go run -race chRace.go produces the following output, which means that there is no longer a race condition:

true true true true true 
false false false false false false false false false false false false false false false

The next section is about the important and powerful select keyword.

The select keyword

The select keyword is really important because it allows you to listen to multiple channels at the same time. A select block can have multiple cases and an optional default case, which mimics the switch statement. It is good for select blocks to have a timeout option just in case. Lastly, a select without any cases (select{}) waits forever.

In practice, this means that select allows a goroutine to wait on multiple communication operations. So, select gives you the power to listen to multiple channels using a single select block. Consequently, you can have non-blocking operations on channels, provided that you have implemented your select blocks appropriately.

A select statement is not evaluated sequentially, as all of its channels are examined simultaneously. If none of the channels in a select statement are ready, the select statement blocks (waits) until one of the channels is ready. If multiple channels of a select statement are ready, then the Go runtime makes a random selection from the set of these ready channels.

The code in select.go presents a simple use of select running in a goroutine that has three cases. But first, let us see how the goroutine that contains select is executed:

    wg.Add(1)
    go func() {
        gen(0, 2*n, createNumber, end)
        wg.Done()
    }()

The previous code tells us that for wg.Done() to get executed, gen() should return first. So, let us see the implementation of gen():

func gen(min, max int, createNumber chan int, end chan bool) {
    time.Sleep(time.Second)
    for {
        select {
        case createNumber <- rand.Intn(max-min) + min:
        case <-end:
            fmt.Println("Ended!")
            // return

The right thing to do here is to add the return statement for gen() to finish. Let us imagine that you have forgotten to add the return statement. This means that the function is not going to finish after the select branch associated with the end channel parameter is executed—createNumber is not going to end the function as it has no return statement. Therefore, the select block keeps waiting for more. The solution can be found in the code that follows:

        case <-time.After(4 * time.Second):
            fmt.Println("time.After()!")
            return
        }
    }
}

So, what is really happening in the code of the entire select block? This particular select statement has three cases. As stated earlier, select does not require a default branch. You can consider the third branch of the select statement as a clever default branch. This happens because time.After() waits for the specified duration (4 * time.Second) to elapse and then prints a message and properly ends gen() with return. This unblocks the select statement in case all of the other channels are blocked for some reason. Although omitting return from the second branch is a bug, this shows that having an exit strategy is always a good thing.

Running select.go produces the following output:

$ go run select.go 10
Going to create 10 random numbers.
13 0 2 8 12 4 13 15 14 19 Ended!
time.After()!
Exiting...

We are going to see select in action in the remainder of the chapter, starting from the next section, which discusses how to time out goroutines. What you should remember is that select allows us to listen to multiple channels from a single point.

Timing out a goroutine

There are times when goroutines take more time than expected to finish—in such situations, we want to time out the goroutines so that we can unblock the program. This section presents two such techniques.

Timing out a goroutine inside main()

This subsection presents a simple technique for timing out a goroutine. The relevant code can be found in the main() function of timeOut1.go:

func main() {
    c1 := make(chan string)
    go func() {
        time.Sleep(3 * time.Second)
        c1 <- "c1 OK"
    }()

The time.Sleep() call is used for emulating the time it normally takes for a function to finish its operation. In this case, the anonymous function that is executed as a goroutine takes about three seconds before writing a message to the c1 channel.

    select {
    case res := <-c1:
        fmt.Println(res)
    case <-time.After(time.Second):
        fmt.Println("timeout c1")
    }

The purpose of the time.After() call is to wait for the desired time before being executed—if another branch is executed, the waiting time resets. In this case, we are not interested in the actual value returned by time.After() but in the fact that the time.After() branch was executed, which means that the waiting time has passed. In this case, as the value passed to the time.After() function is smaller than the value used in the time.Sleep() call that was executed previously, you will most likely get a timeout message. The reason for saying “most likely” is that Linux is not a real-time OS and, sometimes, the OS scheduler plays strange games, especially when it has to deal with a high load and has to schedule lots of tasks—this means that you should not make any assumptions about the operation of the OS scheduler.

    c2 := make(chan string)
    go func() {
        time.Sleep(3 * time.Second)
        c2 <- "c2 OK"
    }()
    select {
    case res := <-c2:
        fmt.Println(res)
    case <-time.After(4 * time.Second):
        fmt.Println("timeout c2")
    }
}

The preceding code executes a goroutine that takes about three seconds to execute because of the time.Sleep() call and defines a timeout period of four seconds in select using time.After(4 * time.Second). If the time.After(4 * time.Second) call returns after you get a value from the c2 channel found in the first case of the select block, then there will be no timeout; otherwise, you will get a timeout. However, in this case, the value of the time.After() call provides enough time for the time.Sleep() call to return, so you will most likely not get a timeout message here.

Let us now verify our thoughts. Running timeOut1.go produces the following output:

$ go run timeOut1.go 
timeout c1
c2 OK

As expected, the first goroutine timed out, whereas the second one did not. The subsection that follows presents another timeout technique.

Timing out a goroutine outside main()

This subsection illustrates another technique for timing out goroutines. The select statement can be found in a separate function. Additionally, the timeout period is given as a command line argument.

The interesting part of timeOut2.go is the implementation of timeout():

func timeout(t time.Duration) {
    temp := make(chan int)
    go func() {
        time.Sleep(5 * time.Second)
        defer close(temp)
    }()
    select {
    case <-temp:
        result <- false
    case <-time.After(t):
        result <- true
    }
}

In timeout(), the time duration that is used in the time.After() call is a function parameter, which means that it can vary. Once again, the select block supports the logic of the timeout. Any timeout period longer than 5 seconds will most likely give the goroutine enough time to finish. If timeout() writes false to the result channel, then there is no timeout, whereas if it writes true, there is a timeout. Running timeOut2.go produces the following output:

$ go run timeOut2.go 100
Timeout period is 100ms
Time out!

The timeout period is 100 milliseconds, which means that the goroutine did not have enough time to finish, hence the timeout message.

$ go run timeOut2.go 5500 
Timeout period is 5.5s
OK

This time, the timeout is 5,500 milliseconds, which means that the goroutine had enough time to finish.

The next section revisits and presents advanced concepts related to channels.

Go channels revisited

So far, we have seen the basic usages of channels—this section presents the definition and the usage of nil channels, signal channels, and buffered channels.

Although channels seem like an interesting concept, they are not the answer to every concurrency problem as there exist times when they can be replaced by mutexes and shared memory. So, do not force the use of channels.

It helps to remember that the zero value of the channel type is nil, and if you send a message to a closed channel, the program panics. However, if you try to read from a closed channel, you get the zero value of the type of that channel. So, after closing a channel, you can no longer write to it but you can still read from it. To be able to close a channel, the channel must not be receive-only.

Additionally, a nil channel always blocks, which means that both reading and writing from nil channels block. This property of channels can be very useful when you want to disable a branch of a select statement by assigning the nil value to a channel variable. Finally, if you try to close a nil channel, your program is going to panic. This is best illustrated in the closeNil.go program:

package main
func main() {
    var c chan string

The previous statement defines a nil channel named c of the type string.

    close(c)
}

Running closeNil.go generates the following output:

panic: close of nil channel
goroutine 1 [running]:
main.main()
    ~/go/src/github.com/mactsouk/mGo4th/ch08/closeNil.go:5 +0x20
exit status 2

The previous output shows the message you are going to get if you try to close a nil channel. Let us now discuss buffered channels.

Buffered channels

Unlike unbuffered channels, which have a capacity of 0 and require a sender to have a corresponding receiver ready at the other end, buffered channels allow a certain number of values to be sent into the channel before a receiver is needed.

These channels allow us to put jobs in a queue quickly to be able to deal with more requests and process requests later on. Moreover, you can use buffered channels as semaphores to limit the throughput of your application.

The presented technique works as follows: all incoming requests are forwarded to a channel, which processes them one by one. When the channel is done processing a request, it sends a message to the original caller saying that it is ready to process a new one. So, the capacity of the buffer of the channel restricts the number of simultaneous requests that it can keep. Bear in mind that it is not the channel that processes the requests or sends the messages.

Also, bear in mind that a buffered channel keeps accepting data until it blocks due to its limited capacity. However, in the presented example, the implementation is what cancels remaining requests after the channel buffer is full due to the select statement, not the channel itself. The source file that implements the technique is named bufChannel.go and contains the following code:

package main
import (
    "fmt"
)
func main() {
    numbers := make(chan int, 5)

The numbers channel can store up to five integers because it is a buffer channel with a capacity of 5.

    counter := 10
    for i := 0; i < counter; i++ {
        select {
        // This is where the processing takes place
        case numbers <- i * i:
            fmt.Println("About to process", i)
        default:
            fmt.Print("No space for ", i, " ")
        }

We begin putting data into numbers—however, when the channel is full, it is not going to store more data and the default branch is going to be executed. This is not because of the way channels work but because of the specific implementation with select.

    }
    fmt.Println()
    for {
        select {
        case num := <-numbers:
            fmt.Print("*", num, " ")
        default:
            fmt.Println("Nothing left to read!")
            return
        }
    }
}

Similarly, we try to read data from numbers using a for loop. When all data from the channel is read, the default branch is going to be executed and will terminate the program with its return statement—when main() returns, the entire program will be terminated.

Running bufChannel.go produces the following output:

$ go run bufChannel.go 
About to process 0
. . .
About to process 4
No space for 5 No space for 6 No space for 7 No space for 8 No space for 9 
*0 *1 *4 *9 *16 Nothing left to read!

Let us now discuss nil channels.

nil channels

nil channels always block! Therefore, you should use them when you want that behavior on purpose! The code that follows illustrates nil channels:

package main
import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)
var wg sync.WaitGroup

We are making wg a global variable in order to be available from anywhere in the code and avoid passing it to every function that needs it as a parameter. This is not idiomatic Go and some people might dislike that approach, despite its simpler implementation. An alternative would be to declare wg in main() and pass a pointer to each function that needs it—you can implement that as an exercise.

func add(c chan int) {
    sum := 0
    t := time.NewTimer(time.Second)
    for {
        select {
        case input := <-c:
            sum = sum + input
        case <-t.C:
            c = nil
            fmt.Println(sum)
            wg.Done()
        }
    }
}

The send() function keeps sending random numbers to channel c. Do not confuse channel c, which is a (channel) function parameter, with channel t.C, which is part of timer t—you can change the name of the c variable but not the name of the C field of a timer. When the time of timer t expires, the timer sends a value to the t.C channel.

This triggers the execution of the relevant branch of the select statement, which assigns the value nil to channel c and prints the value of the sum variable, and wg.Done() is executed, which is going to unblock wg.Wait() found in the main() function. Additionally, as c becomes nil, it stops/blocks send() from sending more data to it.

func send(c chan int) {
    for {
        c <- rand.Intn(10)
    }
}
func main() {
    c := make(chan int)
    rand.Seed(time.Now().Unix())
    wg.Add(1)
    go add(c)
    go send(c)
    wg.Wait()
}

Running nilChannel.go produces the following output:

$ go run nilChannel.go 
11168960

Since the number of times that the first branch of the select statement in add() is going to be executed is not fixed, you get different results each time you execute nilChannel.go.

The next subsection discusses worker pools.

Worker pools

A worker pool is a set of threads that process jobs assigned to them. The Apache web server and the net/http package of Go more or less work this way: the main process accepts all incoming requests, which are forwarded to worker processes to get served. Once a worker process has finished its job, it is ready to serve a new client.

As Go does not have threads, the presented implementation is going to use goroutines instead of threads. Additionally, threads do not usually die after serving a request because the cost of ending a thread and creating a new one is too high, whereas goroutines do die after finishing their job. Worker pools in Go are implemented with the help of buffered channels, as they allow you to limit the number of goroutines running at the same time.

The presented utility implements a simple task: it processes integers and prints their square values using a single goroutine for serving each request. The code of wPools.go is as follows:

package main
import (
    "fmt"
    "os"
    "runtime"
    "strconv"
    "sync"
    "time"
)
type Client struct {
    id      int
    integer int
}

The Client structure is used for keeping track of the requests that the program is going to process.

type Result struct {
    job    Client
    square int
}

The Result structure is used for keeping the data of each Client as well as the results generated by the client. Put simply, the Client structure holds the input data of each request, whereas Result holds the results of a request—if you want to process more complex data, you should modify these structures.

var size = runtime.GOMAXPROCS(0)
var clients = make(chan Client, size)
var data = make(chan Result, size)

The clients and data buffered channels are used to get new client requests and write the results, respectively. If you want your program to run faster, you can increase the value of size.

func worker(wg *sync.WaitGroup) {
    for c := range clients {
        square := c.integer * c.integer
        output := Result{c, square}
        data <- output
        time.Sleep(time.Second)
    }
    wg.Done()
}

The worker() function processes requests by reading the clients channel. Once the processing is complete, the result is written to the data channel. The delay that is introduced with time.Sleep() is not necessary but it gives you a better sense of the way that the generated output is printed.

func create(n int) {
    for i := 0; i < n; i++ {
        c := Client{i, i}
        clients <- c
    }
    close(clients)
}

The purpose of the create() function is to create all requests properly and then send them to the clients buffered channel for processing. Note that the clients channel is read by worker().

func main() {
    if len(os.Args) != 3 {
        fmt.Println("Need #jobs and #workers!")
        return
    }
    nJobs, err := strconv.Atoi(os.Args[1])
    if err != nil {
        fmt.Println(err)
        return
    }
    nWorkers, err := strconv.Atoi(os.Args[2])
    if err != nil {
        fmt.Println(err)
        return
    }

In the preceding code, you read the command line parameters that define the number of jobs and workers. If the number of jobs is greater than the number of workers, the jobs are served in smaller chunks.

    go create(nJobs)

The create() call mimics the client requests that you are going to process.

    finished := make(chan interface{})

The finished channel is used for blocking the program and, therefore, needs no particular data type.

    go func() {
        for d := range data {
            fmt.Printf("Client ID: %d\tint: ", d.job.id)
            fmt.Printf("%d\tsquare: %d\n", d.job.integer, d.square)
        }
        finished <- true

The finished <- true statement is used for unblocking the program as soon as the for range loop ends. The for range loop ends when the data channel is closed, which happens after wg.Wait(), which means after all workers have finished.

    }()
    var wg sync.WaitGroup
    for i := 0; i < nWorkers; i++ {
        wg.Add(1)
        go worker(&wg)
    }
    wg.Wait()
    close(data)

The purpose of the previous for loop is to generate the required number of worker() goroutines to process all requests.

    fmt.Printf("Finished: %v\n", <-finished)
}

The <-finished statement in fmt.Printf() blocks until the finished channel is closed.

Running wPools.go creates the following kind of output:

$ go run wPools.go 8 5
Client ID: 0    int: 0    square: 0
Client ID: 1    int: 1    square: 1
Client ID: 2    int: 2    square: 4
Client ID: 3    int: 3    square: 9
Client ID: 4    int: 4    square: 16
Client ID: 5    int: 5    square: 25
Client ID: 6    int: 6    square: 36
Finished: true

The previous output shows that all requests were processed. This technique allows you to serve a given number of requests, which saves you from server overload. The price you pay for that is having to write more code.

The next subsection introduces signal channels and shows a technique for using them to define the order of execution for a small number of goroutines.

Signal channels

A signal channel is one that is used just for signaling. Put simply, you can use a signal channel when you want to inform another goroutine about something. Signal channels should not be used for data transferring. You are going to see signal channels in action in the next subsection where we specify the order of execution of goroutines.

Specifying the order of execution for your goroutines

This subsection presents a technique for specifying the order of execution of goroutines using signal channels. However, keep in mind that this technique works best when you are dealing with a small number of goroutines. The presented code example has four goroutines that we want to execute in the desired order—first, the goroutine for function A(), then function B(), then C(), and finally, D().

The code of defineOrder.go without the package statement and import block is the following:

var wg sync.WaitGroup
func A(a, b chan struct{}) {
    <-a
    fmt.Println("A()!")
    time.Sleep(time.Second)
    close(b)
}

Function A() is going to be blocked until channel a, which is passed as a parameter, is closed. Just before it ends, it closes channel b, which is passed as a parameter. This is going to unblock the next goroutine, which is going to be function B().

func B(a, b chan struct{}) {
    <-a
    fmt.Println("B()!")
    time.Sleep(3 * time.Second)
    close(b)
}

Similarly, function B() is going to be blocked until channel a, which is passed as a parameter, is closed. Just before B() ends, it closes channel b, which is passed as a parameter. As before, this is going to unblock the following function:

func C(a, b chan struct{}) {
    <-a
    fmt.Println("C()!")
    close(b)
}

As it happened with functions A() and B(), the execution of function C() is blocked by channel a. Just before it ends, it closes channel b.

func D(a chan struct{}) {
    <-a
    fmt.Println("D()!")
    wg.Done()
}

This is the last function that is going to be executed. Therefore, although it is blocked, it does not close any channels before exiting. Additionally, being the last function means that it can be executed more than once, which is not true for functions A(), B(), and C() because a channel can be closed only once.

func main() {
    x := make(chan struct{})
    y := make(chan struct{})
    z := make(chan struct{})
    w := make(chan struct{})

We need to have as many channels as the number of functions we want to execute as goroutines.

    wg.Add(1)
    go func() {
        D(w)
    }()

This proves that the order of execution dictated by the Go code does not matter as D() is going to be executed last.

    wg.Add(1)
    go func() {
        D(w)
    }()
    go A(x, y)
    wg.Add(1)
    go func() {
        D(w)
    }()
    go C(z, w)
    go B(y, z)

Although we run C() before B(), C() is going to finish after B() has finished.

    wg.Add(1)
    go func() {
        D(w)
    }()
    // This triggers the process
    close(x)

The closing of the first channel is what triggers the execution of the goroutines because this unblocks A().

    wg.Wait()
}

Running defineOrder.go produces the following output:

$ go run defineOrder.go
A()!
B()!
C()!
D()! D()! D()! D()!

So, the four functions, which are executed as goroutines, are executed in the desired order, and, in the case of the last function, the desired number of times.

Handling UNIX signals

UNIX signals offer a very handy way of interacting asynchronously with applications and server processes. UNIX signal handling in Go requires the use of channels that are used exclusively for this task. The presented program handles SIGINT (which is called syscall.SIGINT in Go) and SIGINFO separately and uses a default case in a switch block for handling the remaining signals. The implementation of that switch block allows you to differentiate between the various signals according to your needs.

There exists a dedicated channel that receives all signals, as defined by the signal.Notify() function. Go channels can have a capacity—the capacity of this particular channel is 1 in order to be able to receive and keep one signal at a time. This makes perfect sense as a signal can terminate a program and there is no need to try to handle another signal at the same time. There is usually an anonymous function that is executed as a goroutine and performs the signal handling and nothing else. The main task of that goroutine is to listen to the channel for data. Once a signal is received, it is sent to that channel, read by the goroutine, and stored into a variable—at this point, the channel can receive more signals. That variable is processed by a switch statement.

Some signals cannot be caught, and the operating system cannot ignore them. So, the SIGKILL and SIGSTOP signals cannot be blocked, caught, or ignored; the reason for this is that they allow privileged users as well as the UNIX kernel to terminate any process they desire.

Create a text file by typing the following code—a good filename for it would be signals.go.

package main
import (
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "time"
)
func handleSignal(sig os.Signal) {
    fmt.Println("handleSignal() Caught:", sig)
}

handleSignal() is a separate function for handling signals. However, you can also handle signals inline, in the branches of a switch statement.

func main() {
    fmt.Printf("Process ID: %d\n", os.Getpid())
    sigs := make(chan os.Signal, 1)

We create a channel with data of the type os.Signal because all channels must have a type.

    signal.Notify(sigs)

The previous statement means handling all signals that can be handled.

    start := time.Now()
    go func() {
        for {
            sig := <-sigs

Wait until you read data (<-) from the sigs channel and store it in the sig variable.

            switch sig {

Depending on the read value, act accordingly. This is how you differentiate between signals.

            case syscall.SIGINT:
                duration := time.Since(start)
                fmt.Println("Execution time:", duration)

For the handling of syscall.SIGINT, we calculate the time that has passed since the beginning of the program execution and print it on the screen.

            case syscall.SIGINFO:
                handleSignal(sig)

The code of the syscall.SIGINFO case calls the handleSignal() function—it is up to the developer to decide on the details of the implementation.

On Linux machines, you should replace syscall.SIGINFO with another signal such as syscall.SIGUSR1 or syscall.SIGUSR2 because syscall.SIGINFO is not available on Linux (https://github.com/golang/go/issues/1653).

                // do not use return here because the goroutine exits
                // but the time.Sleep() will continue to work!
                os.Exit(0)
            default:
                fmt.Println("Caught:", sig)
            }

If there is no match, the default case handles the rest of the values and just prints a message.

        }
    }()
    for {
        fmt.Print("+")
        time.Sleep(10 * time.Second)
    }
}

The endless for loop at the end of the main() function is for emulating the operation of a real program. Without an endless for loop, the program exits almost immediately.

Running signals.go and interacting with it creates the following kind of output:

$ go run signals.go
Process ID: 70153
+^CExecution time: 631.533125ms
+Caught: user defined signal 1
+Caught: urgent I/O condition
+signal: killed

The second line of output was generated by pressing Ctrl + C on the keyboard, which, on UNIX machines, sends the syscall.SIGINT signal to the program. The third line of output was caused by executing kill -USR1 74252 on a different terminal. The last line of the output was generated by the kill -9 74252 command. As the KILL signal (which is also represented by the number 9) cannot be handled, it terminates the program, and the shell prints the killed message.

Handling two signals

If you want to handle a limited number of signals instead of all of them, you should replace the signal.Notify(sigs) statement with a statement like the following:

signal.Notify(sigs, syscall.SIGINT, syscall.SIGINFO)

After that, you need to make the appropriate changes to the code of the goroutine responsible for signal handling in order to identify and handle syscall.SIGINT and syscall.SIGINFO—the current version (signals.go) already handles both of them.

The next section talks about shared memory and shared variables, which is a very handy way of making goroutines communicate with each other with the use of channels.

Shared memory and shared variables

Shared memory and shared variables are huge topics in concurrent programming and the most common ways for UNIX threads to communicate with each other. The same principles apply to Go and goroutines, which is what this section is about. A mutex variable, which is the abbreviation for a mutual exclusion variable, is mainly used for thread synchronization and for protecting shared data when multiple writes or a write and a read can occur at the same time. A mutex works like a buffered channel with a capacity of one, which allows, at most, one goroutine to access a shared variable at any given time. This means that there is no way for two or more goroutines to be able to update that variable simultaneously. Go offers the sync.Mutex and sync.RWMutex data types.

A critical section of a concurrent program is the code that cannot be executed simultaneously by all processes, threads, or, in this case, goroutines. It is the code that needs to be protected by mutexes. Therefore, identifying the critical sections of your code makes the whole programming process so much simpler that you should pay particular attention to this task. A critical section cannot be embedded into another critical section when both critical sections use the same sync.Mutex or sync.RWMutex variable. However, avoid at almost any cost the spreading of mutexes across functions because that makes it really hard to see whether you are embedding or not.

The sync.Mutex type

The sync.Mutex type is the Go implementation of a mutex. Its definition, which can be found in the mutex.go file of the sync directory, is as follows (you do not need to know the definition of sync.Mutex in order to use it):

type Mutex struct {
    state int32
    sema  uint32
}

The definition of sync.Mutex is nothing special. All of the interesting work is done by the sync.Lock() and sync.Unlock() functions, which can lock and unlock a sync.Mutex variable, respectively. Locking a mutex means that nobody else can lock it until it has been released using the sync.Unlock() function. All these are illustrated in mutex.go, which contains the following code:

package main
import (
    "fmt"
    "os"
    "strconv"
    "sync"
    "time"
)
var m sync.Mutex
var v1 int
func change() {
    m.Lock()
    defer m.Unlock()

This function makes changes to the value of v1. The critical section begins here.

    time.Sleep(time.Second)
    v1 = v1 + 1
    if v1 == 10 {
        v1 = 0
        fmt.Print("* ")
    }

This is the end of the critical section. Now, another goroutine can lock the mutex.

}
func read() int {
    m.Lock()
    a := v1
    defer m.Unlock()
    return a
}

This function is used for reading the value of v1—therefore, it should use a mutex to make the process concurrently safe. Most specifically, we want to make sure that nobody is going to change the value of v1 while we are reading it. The rest of the program contains the implementation of the main() function—feel free to see the entire code of mutex.go in the GitHub repository of the book.

Running mutex.go produces the following output:

$ go run -race mutex.go 10
0 -> 1-> 2-> 3-> 4-> 5-> 6-> 7-> 8-> 9* -> 0-> 0

The previous output shows that due to the use of a mutex, goroutines cannot access shared data and, therefore, there are no hidden race conditions.

The next subsection shows what could happen if we forget to unlock a mutex.

What happens if you forget to unlock a mutex?

Forgetting to unlock a sync.Mutex mutex creates a panic situation, even in the simplest kind of a program. The same applies to the sync.RWMutex mutex, which is presented in the next section. Let us now see a code example to understand this unpleasant situation a lot better—this is part of forgetMutex.go:

var m sync.Mutex
var w sync.WaitGroup
func function() {
    m.Lock()
    fmt.Println("Locked!")
}

Here, we lock a mutex without releasing it afterward. This means that if we run function() as a goroutine more than once, all instances after the first one are going to be blocked waiting to Lock() the shared mutex. In our case, we run two goroutines—feel free to see the entire code of forgetMutex.go for more details. Running forgetMutex.go generates the following output:

Locked!
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x140000021a0?)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/runtime/sema.go:62 +0x2c
sync.(*WaitGroup).Wait(0x100fa1710)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/waitgroup.go:116 +0x74
main.main()
    ~/go/src/github.com/mactsouk/mGo4th/ch08/forgetMutex.go:29 +0x5c
goroutine 34 [sync.Mutex.Lock]:
sync.runtime_SemacquireMutex(0x0?, 0x0?, 0x0?)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/runtime/sema.go:77 +0x28
sync.(*Mutex).lockSlow(0x100fa1520)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/mutex.go:171 +0x174
sync.(*Mutex).Lock(...)
    /opt/homebrew/Cellar/go/1.21.0/libexec/src/sync/mutex.go:90
main.function()
    ~/go/src/github.com/mactsouk/mGo4th/ch08/forgetMutex.go:12 +0x84
main.main.func1()
    ~/go/src/github.com/mactsouk/mGo4th/ch08/forgetMutex.go:20 +0x50
created by main.main in goroutine 1
    ~/go/src/github.com/mactsouk/mGo4th/ch08/forgetMutex.go:18 +0x34
exit status 2

As expected, the program crashes because of the deadlock. To avoid such situations, always remember to unlock any mutexes created in your program as soon as possible.

Let us now discuss sync.RWMutex, which is an improved version of sync.Mutex.

The sync.RWMutex type

The sync.RWMutex data type is an improved version of sync.Mutex and is defined in the rwmutex.go file of the sync directory of the Go Standard library as follows:

type RWMutex struct {
    w           Mutex
    writerSem   uint32
    readerSem   uint32
    readerCount int32
    readerWait  int32
}

In other words, sync.RWMutex is based on sync.Mutex with the necessary additions and improvements. So, you might ask, how does sync.RWMutex improve sync.Mutex? Although a single function is allowed to perform write operations with a sync.RWMutex mutex, you can have multiple readers owning a sync.RWMutex mutex—this means that read operations are usually faster with sync.RWMutex. However, there is one important detail that you should be aware of: until all of the readers of a sync.RWMutex mutex unlock it, you cannot lock it for writing, which is the small price you have to pay for the performance improvement you get for allowing multiple readers.

The functions that can help you work with sync.RWMutex are RLock() and RUnlock(), which are used for locking and unlocking the mutex for reading purposes, respectively. The Lock() and Unlock() functions used in sync.Mutex should still be used when you want to lock and unlock a sync.RWMutex mutex for writing purposes. Finally, it should be apparent that you should not make changes to any shared variables inside an RLock() and RUnlock() block of code.

All these are illustrated in rwMutex.go—the important code is the following:

var Password *secret
var wg sync.WaitGroup
type secret struct {
    RWM      sync.RWMutex
    password string
}