Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member.

📅 Online: Apr 18 & Apr 19. Limited seats.

Register now

✍️ From the editor’s desk,

Welcome to the 43rd issue of Deep Engineering!

On April 15, Neo4j ran NODES AI 2026, a full-day virtual conference dedicated to AI with knowledge graphs, with tracks covering GraphRAG, graph-based agent memory, and context engineering. Interestingly, the opening keynote framed the central challenge directly around how most AI systems behave as if they have never seen the data that the organisations deploying them have spent years accumulating. The gap between what a system has access to and what it can actually use at query time is, in the view of the practitioners at NODES, the defining engineering problem of this moment.

And so, in today’s issue we want to address exactly that gap. David Knickerbocker, founder of Verdant Intelligence and author of Network Science with Python (Packt), has spent years building knowledge graph systems that stay current with a different design constraint than most teams start from: not how to make retrieval faster, but how to make the system aware of what changed a minute ago. His system (Verdant Eye) treats knowledge as a continuous stream of claims rather than a static store of facts, and that distinction has significant consequences for how you handle freshness, hallucination, temporal drift, and testing.

This is the first of two features from our conversation with David, the second will cover what building AI on top of messy, adversarial, real-world data teaches you that clean-dataset engineers never have to confront. You can read the full interview and watch the conversation here.

Let’s get started.

30k+ DevOps Engineers Read EverythingDevOps

The cloud-native ecosystem moves on its own schedule. New tooling, shifting practices, team structures that look nothing like they did two years ago. Therefore, keeping up is not optional when your job is to build and run reliable systems.

EverythingDevOps‘ weekly digest is built specifically for engineers who want the signal without the scroll.

• What’s changing in the ecosystem — and what it actually means for your work

• Career thinking for engineers moving from senior to staff to leadership

• Events, opportunities, and community conversations worth your Frida

Join 30,000+ engineers

AI That Sees the World as It Is, Not as It Was

by with David Knickerbocker

Most AI systems are answering questions about a world that no longer quite exists. They draw from training data frozen at a point in time, from retrieval systems that return whatever was most recently indexed, from pipelines that treat knowledge as a stable object rather than a moving one. For David Knickerbocker, founder of Verdant Intelligence, this is not a limitation to work around, but a design error to avoid from the start.

The problem starts at problem definition

Building a knowledge graph that stays fresh does not require a different database. It requires a different question. For Knickerbocker the starting point is always the same, “what is the system actually trying to know, and at what resolution of time?”

He explains, “If you want to build a world AI and be able to answer questions about things that happened a minute ago, then that is your problem statement. And then you think about how to get that data into the database so that it is there and it is fresh. But then you also have to get AI to be able to use that data. There are two sides to this coin.”

The engineering philosophy he applies is KISS (Keep it Super Simple), and YAGNI (You Aren't Gonna Need It"). Both push in the same direction, he reasons “build the minimum thing that works, validate that it works, and expand from there.” His system (Verdant Eye) maintains data fresh up to a minute old, not by doing something architecturally exotic, but by treating freshness as a first-class constraint from day one rather than an optimization to add later.

“The AI industry feels very shiny and very new, but there is a lot of old school discipline that is still extremely useful. You start with the idea, you go through the ideation, from ideation you create your spec, from the spec you do your project management, you assign tasks and do the work. It feels like vanilla old school engineering to me.”

Awareness before truth

The instinct when building a real-time knowledge system is to treat freshness and accuracy as a tension to manage: newer data is less verified, older data is more reliable, and the system needs some formula for weighing one against the other. But Knickerbocker rejects this frame entirely.

He explains from his engineering experience that why his systems are not trying to determine what is true. But instead trying to capture what is being claimed, and those are fundamentally different engineering problems.

“In the world of open source intelligence, it has less to do with right and wrong. It has less to do with facts. What I am looking for is really claims of what is going on in the world. I do not make that decision, and I do not allow my AI to make the decision about what is true or false either.”

In adversarial open source intelligence environments, two sources in direct conflict with each other are not a problem to resolve. But they are both signal. And what matters is that both claims are captured, clustered, and surfaced. Knickerbocker describes this structure using a frame he developed from years of NLP work: not clusters in the geometric sense, but ribbons, layered bands of related information that emerge from the data.

“You have a whole bunch of information and this top ribbon might be this bad thing happened. The next ribbon might be this event is happening at the library. The next ribbon might be a punk rock show is happening at this nightclub.”

The consequence of this design is that awareness comes before truth, and that is by intention. For the applications the Verdant Eye is built for, knowing what is being said and where is actionable before you know whether any of it is correct. Adjudicating truth is a downstream problem, one that can be handled by human judgment or purpose-built downstream systems, not by the ingestion layer.

Snapshot versus movie

What separates a real-time knowledge graph from an agent that runs searches is not a matter of degree. It is a structural difference in what kind of object the system is.

An agent with search tools takes a snapshot. It queries for something, gets back the most relevant indexed results, and presents them. The system only knows what it was asked to look for, and it only looks when prompted. A knowledge graph that continuously ingests from the open web is more like a movie: it is always recording, and when you query it, you are not taking a new photograph but pulling a frame from a film that has been running the whole time.

“If you use a tool to do a search to find out something, you are getting a snapshot of time. My systems capture the heartbeat of the Internet themselves, and they are always listening. It would be closer to a movie compared to a photograph.”

The practical difference shows up clearly in demos with clients who need urgent situational information. Running a calibration query against the Verdant Eye, asking what is the latest information, returns results anywhere from a few seconds old to a minute and a half old. That behaviour is not possible with a search-tool approach at any level of optimization.

An empty result matters too. If nothing has been reported in the last minute on a given topic, the system returns nothing. That discipline is load-bearing.

“An empty dataset,” Knickerbocker underscores, “is better than a hallucination.”

The rush to use a tool before understanding it

Knickerbocker spent years watching teams fail with graph databases in 2020 and 2021, and he sees the same pattern repeating now with agents and AI. Teams adopt the tool before building the understanding, and then cannot tell whether it is working.

“There is a rush to use agents before even understanding AI. And if the understanding is not there, then it is just wishful. If you do not know how it works, you cannot tell the difference between it ran and it ran correctly. Those are very different things.”

His own relationship with graphs predates the graph database trend by years. At Intel, from 2015 to 2019, he was using graph theory for data flow mapping, tracing inputs to outputs across thousands of scripts and hundreds of servers, using centralities to find important nodes and shortest paths to understand flow. He was never using graph databases. He was using the underlying science.

“I was never invited to the cool kid graph database parties. I was always just doing stuff with graphs and using it to map out data flows and fix production outages. Dead serious stuff.”

That gap between tool and understanding is what drove him to write Network Science with Python. Teams were populating graph databases and then stopping, treating the populated database as the deliverable. The value, in his view, starts the moment the graph is populated and you begin running network science on top of it.

“If you do not know about the basics of network science, then what are you gonna do with the graph database? You have put your data in the graph database. Whereas if you come to graph databases and you have learned about centralities, community detection, shortest paths, simulations, then it can really have an impact on your network thinking.”

Similarity is not the same as identical

The central limitation of vector-based retrieval is one that Knickerbocker has framed the same way since at least 2017. Semantic similarity is a probabilistic measure, not a logical one. Two pieces of text can be highly similar in embedding space while referring to entirely different things. A graph replaces probabilistic similarity with structural traversal.

“Similarity in language is not equal to same. I will say that one more time. Similarity is not equal to same. Similar sounding things can be very, very different from each other. A graph anchors things into a piece of context.”

In a GraphRAG system, the query does not ask what is most like this input. It traverses the graph: if you are asking about jazz events in Portland, Oregon, you are connected to the Oregon node, to the Portland node, to the jazz node. The hallucination space collapses because the answer space is bounded by structure rather than probability.

“In a GraphRAG system, if there is no match then the output is that there is no match. There is no hallucination opportunity. With a similarity-based system, there could be similarity even if it is only a single word in a paragraph.”

This intuition was clear to him during his data operations years at Intel. Log files across hundreds of servers written by dozens of engineers contain enormous amounts of natural language. Working with that language to understand what a production system was doing made it obvious that the structure, the graph, and the meaning, the language, were inseparable.

“Graphs show you where things go,” Knickerbocker adds. “But all of the context about what that node even is is often carried by language itself.”

Deliberate forgetting

Temporal drift is usually framed as a correctness problem: facts become outdated, relationships change, the graph silently goes wrong. The standard response is validation pipelines, contradiction detection, freshness scores. Knickerbocker builds around a different principle.

Sharing from his experience building the Verdant Eye, Knickerbocker explains that the Verdant Eye system is not called the Verdant Brain because the metaphor is not storage. It is perception.

“The Verdant Eye sees, and it does not contain eternal memory, because that is not what an eye does. An eye sees. When the scene changes, the scene changes. Your eyes do not need to be recalibrated. The thing has just changed.”

The design principle is biological: living systems do not maintain eternal memory, and a knowledge system that tracks a living world should not either. The practical implementation mirrors transactional database patterns he worked with throughout his career in data operations. Data that is no longer operationally relevant gets archived. The live layer runs on what is current, and the cost of maintaining it stays proportional to what the system actually needs to know.

“In a transactional database, you operate off of what you need, and data that is not needed eventually gets archived. I want to build AI that does not boil the ocean, that can be bootstrapped by individuals. Infinite memory is not just philosophically wrong for this kind of system. It is economically unworkable.”

The pass butter philosophy

Knickerbocker shares how he currently runs three production GraphRAG systems, each built for a distinct purpose with its own testing criteria. Verdant Intelligence operates at high altitude, tracking events across states and regions. Grooveseeker (which he created to put his AI system to a different and specific kind of use) operates at street level, finding events in specific cities on specific nights. A third system contains thirty years of AI research, useful for tracing the intellectual lineage of ideas when building new things.

The principle connecting all three comes from a scene in Rick and Morty. Rick builds a robot. The robot asks what its purpose is. Rick says: you pass butter.

“There is no testing framework anybody else can give me that is going to be fit for purpose for what I am trying to build, because I am not trying to build general intelligence. I am trying to build intelligence that serves a specific purpose.”

Each system is therefore tested against the specific failures that would make it useless for its purpose. For Grooveseeker, that means verifying the right city, not a city of the same name in a different state; the right date, not a historical recurrence of the same event; and a URL that actually leads somewhere you can buy a ticket.

The final test of the Grooveseeker system was to stop writing articles proving it worked and simply use it. The system returned events in Portland between March 10 and March 13. Knickerbocker even attended one of them, bought a ticket at the door, saw the band, and spoke to one of the musicians.

His AI did not send him to a venue that did not exist. “That is how I know it works,” Knickerbocker affirms.

🔍 In case you missed it…

🛠️ Tool of the Week

Graphiti — open-source temporal knowledge graph engine for AI agents

Highlights:

• Temporal fact management: Facts carry explicit validity windows — invalidated when superseded, never deleted, queryable at any point in time.

• Incremental ingestion: New data integrates immediately without batch recomputation, keeping the graph current as the world changes.

• Hybrid retrieval: Combines semantic search, BM25, and graph traversal in a single query, typically under 100ms.

• MCP-native: Ships an MCP server for direct integration with Claude, Cursor, and other MCP clients.

Learn more about Graphiti

📎 Tech Briefs

• Neo4j NODES AI 2026 closing keynote: From Data to Knowledge to Action - Sudhir Hasbe makes the case that most enterprise AI behaves as if it has never seen the organisation's own data, and how graphs help close that gap.

• Graph memory for LLM agents: the relational blind spots that flat vectors miss - Zep's LongMemEval evaluation shows 18.5% accuracy gains over vector baselines, context tokens dropping from 115,000 to 1,600, and latency falling from 29 seconds to under 3 seconds.

• FalkorDB Browser ships April 2026 update for GraphRAG developers - Adds favourite query saving, one-click connection string copying, enhanced graph statistics, and configurable table views..

• Neo4j Aura Agent Hackathon opens - Developers building knowledge-graph-grounded AI agents can register for cloud credits and managed GraphRAG platform access.

• Four GraphRAG papers accepted at ACL 2026 - PolyU’s Awesome-GraphRAG repo confirmed acceptances including ProbeRAG for retrieval faithfulness, LegalGraphRAG for legal reasoning, and LinearRAG for efficient graph construction.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and support my work.

Hone your skills to design, implement, and ship an AI-powered internal developer platform from scratch with Ajay Chankramath.

📅 25th and 26th April 2026 | 11:00 AM to 3:00 PM ET

Use code DEEPENG50 for 50% off, or grab the 2-for-1 deal

Register now

Includes access to The Platform Engineer’s Handbook by Ajay Chankramath upon release (eBook worth $35.99)

✍️ From the editor’s desk,

Welcome to the 42nd issue of Deep Engineering!

Multi-agent AI is widely being adopted in production, but the engineering reality is messier than the demos suggest. GitHub Engineering published a post by Gwen Davis laying out the most common ways multi-agent workflows fail in practice. Most failures come down to missing structure, not model capability.

Agents make implicit assumptions about state, ordering, and validation at the points where they hand off work to one another, and without explicit data formats, typed interfaces, and defined action schemas at those boundaries, the system breaks in ways that are hard to reproduce and harder to debug.

The architectural decisions you make around the model matter more than the model itself. That is precisely what today’s issue covers. Fitz Nowlan, VP of AI and Architecture at SmartBear, has spent years building agentic systems in production.

Let’s get started.

Featured Newsletter: The Cloud Playbook

If you own reliability, cost, and compliance in production on AWS, The Cloud Playbook newsletter is worth your attention.

It is a weekly read for engineering leaders building platforms where incidents, audits, and surprise AWS bills are real problems. It covers AWS architecture, platform engineering, FinOps, security, observability, and compliance frameworks like FedRAMP, HIPAA, and ISO 27001.

Subscribe to The Cloud Playbook

Multi-Agent Systems Need Rules to Stay Reliable in Production

by with Fitz Nowlan

As agents pass tasks to one another, reason autonomously, and compose solutions on the fly, it is easy to construct a version of multi-agent AI that looks technically sound on a whiteboard. In practice, however, that version tends to fall apart when context gets lost between handoffs, when loops run without terminating, and when open-ended systems become progressively harder to test, debug, or trust.

Fitz Nowlan, VP of AI and Architecture at SmartBear, has spent the past two years building agentic systems in production, and his view, drawn from that engineering experience, is that the reliability of a multi-agent system is determined less by the model you choose and more by the architectural decisions you make around it.

Preserving facts across the handoff boundary

The first place multi-agent systems break down is at the handoff. When agent A passes context to agent B, the question is not just what to pass but how to pass it. Raw string prompts are the path of least resistance, but Nowlan argues they are also the path most likely to introduce errors.

“We are almost always using some form of structured data,” he points out. “It’s generally going to be JSON. What we’ll often try to do on top of that JSON is define, effectively, a domain-specific language, or DSL, or our own schema, to cache, kind of, to preserve truth that we’ve already determined.”

The distinction matters because of what happens when you do not preserve that truth. Suppose an agent is working within the context of a browser and identifies that two elements appear inside the same container on a page. That relationship has been syntactically proven through a screenshot, the HTML, or the DOM. If you pass that context as plain text to the next agent, that agent has to decompose the text and then probabilistically reconstruct whether the relationship exists at all.

“That other agent has to effectively decompose and then probabilistically, i.e., potentially hallucinate, that relationship is there,” Nowlan notes. “So what we try to do is communicate over JSON and lock in to the JSON in our custom schema the facts that we’ve identified or extracted from one context or world to another.”

The practical principle is simple enough. Anything that has been established as a fact should be encoded as a fact, not as prose. Structured contracts at the handoff boundary are not just a formatting preference. They are a mechanism for preventing the downstream agent from having to guess at things the upstream agent already knew.

When MCP outperforms a static API wrapper

Model Context Protocol (MCP) has become a standard topic in agentic architecture discussions, and anyone planning to use it should understand where it actually adds value and where it does not.

“If you knew all five workflows that your customers ever did in your application, well, then you should just statically code them up,” Nowlan reasons. “Make it a single API endpoint, take the inputs at the outset, chain them all together, and spit the outputs back out to your user. That would be highly efficient. You wouldn’t then need MCP or agents at all.”

The benefit of MCP is the emergent use case. That is the workflow your users need that you did not anticipate, in the sequence they need it, composed on the fly from the tools you have exposed. By giving the model a set of tools rather than a fixed pipeline, you allow the system to compose solutions for problems you never explicitly programmed for.

“If you have an application that’s very diverse, with a diverse set of data, with a wide-ranging type or class of data, and maybe even different user roles that are coming in and using your product, that’s where MCP can really be a massive unlock, because the composition artifacts are effectively infinite,” Nowlan observes.

The corollary is equally important. If your application is relatively rigid and your users reliably need the same five things, MCP introduces complexity without adding value. The decision to use it should follow directly from the diversity of your use cases, not from the fact that it is available.

The 80-20 rule for DAG versus autonomous orchestration

One of the more practically useful frameworks Nowlan describes is how his team decides when to use a fixed Directed Acyclic Graph (DAG)-based workflow versus an open-ended MCP-style loop, and how that decision evolves over time.

The starting point is openness. When a new feature or product area is introduced, the team tends to begin with the full MCP loop. The premise is straightforward. Here is the set of tools. Let the system compose whatever the user needs. This preserves maximum flexibility while the team learns how customers actually use the system.

Over time, patterns emerge. Certain workflows appear repeatedly. Those are the candidates for promotion into a DAG.

“There’s kind of an 80-20 rule here, where 80% of the time your customers are looking to solve their problems with 20% of these key workflows that you’ve identified,” Nowlan notes. “Those should then be translated into DAGs, into workflows, into a little bit stricter information flow architecture.”

The payoff is significant, Nowlan affirms. Once a workflow is on a DAG, you can optimise aggressively. You can use a cheaper model for nodes that do not require frontier-level reasoning. You can shed context that is irrelevant to a particular step rather than packing everything into the window. You can bound latency because the execution path is known in advance.

“When you get on that track, that predefined workflow, that’s where you can save cost, you can potentially use a cheaper model for a particular node that you know doesn’t need the most expensive model. And you can also shed information from context in that workflow when you know it’s not necessary for the outcome.”

The open-ended MCP loop remains available for everything else, for the 20% of use cases that do not fit a known pattern. The architecture supports both, and the team actively monitors usage to identify when a new workflow has crossed the threshold and is ready to be promoted. There is no firm cutoff, just the observation that a workflow is appearing often enough to justify the investment in making it faster and cheaper.

The infinite loop problem

Bounding execution is one of the more fundamental challenges in agentic systems, and it is one that the industry is still working through. The early solutions were basic by necessity. At the most fundamental level, Nowlan explains, it comes down to two questions: has the system hit a timeout, and has it exceeded a set number of attempts.

The deeper issue is that the problem did not originate with modern agents. In the early days of GPT-3-era pipelines, a common approach was to ask the model for one action at a time, execute it, and return for the next. The loop would never terminate because the model had no reliable sense of when it was done.

“We quickly realized they would literally go on forever. The loop would just never terminate, would never know that it was done, or it would know that it was done way too early.”

The response was to shift from reactive step-by-step prompting to upfront scoping. Rather than asking the model what to do next after each action, you ask it at the outset to decompose the overall task into a bounded set of subtasks, and then you execute those subtasks in sequence and stop.

“We pivoted from the early days toward putting more bounds on the problem space. What are the things you think I should do? Break this down into a set of tasks, a set of subtasks, and then I’ll take each of those subtasks in turn, but when I get to that last subtask, I’m done. I’m not going back and asking you for more.”

As models have improved, this has relaxed somewhat. The team now allows a final evaluation pass, a double or triple check at the end, before closing the loop. But the underlying principle has not changed. Get the model to scope the work before execution begins, and treat that scope as a constraint rather than a suggestion.

“You want to get the AI to put bounds and scope around the overall work that you’ll be doing to solve or complete some task, and try to stick to that as a guide so that you don’t run off into the infinite space of querying forever.”

Testing non-deterministic flows

Testing a system where the execution path changes based on the model’s output requires a different approach from conventional integration testing. Nowlan and his team lean toward trace-based evaluation, grounded in a close understanding of realistic user inputs.

“All of our evaluations are, we think, we hope, reasonably close to the reality of those inputs that we’re going to get from our end users.”

The logic is that if you understand the domain well enough, you can anticipate the shape of the tasks users will bring to the system even if you cannot predict the exact inputs. In a web testing context, for example, that means understanding that users are going to log in, fill out forms, scroll through lists, and navigate between pages. Those logical actions become the basis for evaluations, and the evaluations can then check whether the agent followed a reasonable path to completion rather than whether it followed an exact predetermined path.

At the same time, Nowlan acknowledges that no evaluation suite fully covers the space of real-world inputs. The complement to pre-built evaluations is comprehensive logging and tracing of every prompt and response the system exchanges with the model.

“We log and trace all of our inputs and outputs that we exchange to the LLMs, and then we can go back and debug those, and examine those, and we can obviously use AI to probabilistically parse and understand those inputs and outputs.”

This creates a feedback loop that pre-built evaluations cannot replicate. When a user reports a bad experience or churns, the team can go back to the traces for that user, examine the prompt and response sequences, and use a model to evaluate whether the quality of the AI outputs degraded at any point. The evaluation happens after the fact, using the actual production inputs rather than synthetic ones.

Nowlan’s broader point is that evaluation in non-deterministic systems is not a gate you run before deployment. It is an ongoing process that runs in parallel with production, using real data to surface quality issues that no pre-deployment test suite would have caught.

🛠️ Tool of the Week

smolagents — Hugging Face’s open-source library for building agents that think in code

Highlights:

• Code-first agents: Generates and executes Python code instead of structured tool calls, which can reduce back-and-forth with the model in some workflows.

• Model-agnostic: Works with local Transformers models, Hugging Face Inference API, and providers via LiteLLM.

• Supports sandboxing: Can run code in environments like Docker, E2B, or Pyodide, but requires proper setup for safety.

• Hub integration: Tools and agents can be shared and reused via the Hugging Face Hub.

Learn more about smolagents

📎 Tech Briefs

• Anthropic moves Claude Mythos into controlled early access under Project Glasswing - Limited rollout to partners including Amazon, Apple, and Microsoft for defensive security work, with early reports pointing to strong capability in identifying software vulnerabilities

• Microsoft Copilot Studio ships multi-agent orchestration to general availability - Now supports A2A protocol for agent-to-agent delegation and cross-app agent reuse via the Microsoft 365 Agents SDK.

• Google releases Gemma 4 under Apache 2.0 - Ranges from lightweight edge models to a 31B-parameter variant with strong performance on reasoning, agentic workflows, and multilingual tasks.

• LangGraph ships async subagents in latest Deep Agents update - Adds non-blocking background subagents alongside type-safe streaming and Pydantic coercion in the new v2 API.

• Microsoft releases open-source Agent Governance Toolkit - A seven-package system that intercepts every agent action before execution at sub-millisecond latency, with native integrations for LangChain, CrewAI, Google ADK, and Microsoft Agent Framework.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and support my work.

Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member. Online: Sat Apr 11, 10:30 AM–Sun Apr 12, 4:00 PM ET. Limited seats.

Register now

✍️ From the editor’s desk,

Welcome to the 41^st issue of Deep Engineering!

As AI scaling hits the memory wall, system performance is increasingly limited by data movement rather than raw compute power. This shifts the optimization focus toward data locality because adding cores provides no benefit when bandwidth bottlenecks the system. Writing fast code now requires precise control over memory layout and concurrency, yet these techniques often introduce complexity that becomes unmanageable as codebases grow.

That specific tension defines this week’s feature with Sam Morley, Research Software Engineer and Mathematician at the University of Oxford and author of The C++ Programmer’s Mindset.

Building on our earlier discussion in Part 1 around decomposition and abstraction costs, today's issue digs deeper into the guiding principles for scaling C++ systems. It covers how teams can keep complexity under control as codebases grow, why template metaprogramming deserves extreme caution, and why thinking with the machine is non-negotiable when optimising cache layout.

Let’s get started.

Stop Building Vault

Secrets, PKI, & PAM in one platform. Postgres-backed. No custom orchestration. Flexible deployment.

Start for free today →

Adopting the C++ Programmer’s Mindset (Part 2) with Sam Morley

Before listing any specific practices, Morley introduces a concept he returns to repeatedly. He calls it the future you.

“Future you is your future self and for all intents and purposes this is a different person,” Morley says. “When you’re writing some code, you understand things the way they are in the context of what you’re doing at the moment. Future you will have lost this context. So, when you come back to your code in a month, six months, a year’s time and you look at it and you think what was I thinking — almost surely the answer to that is: I don’t know.”

The practical implication is easy to skip under deadline pressure, but it could be costly to ignore. Comments are not just for teammates. They are messages to yourself, written at the moment of maximum understanding, for a future reader who no longer has that understanding. Morley is explicit that this does not mean narrating the obvious. What he does instead, particularly on mathematically intricate work, is write large block comments that describe where the process is, how the next section works, and what the algorithm is intended to achieve. “These comments save me so much pain when I jump off the project for a week and then go back and have to remember exactly what I was trying to do.”

Beyond comments, Morley points to three C++ specific practices that matter most for scalability. The first is a strict separation of concerns. Numerical computation should not live in the same place as user-facing code. Components should be modular enough to be tested in isolation. This also pays dividends when distributing computation across clusters, because reusable tight-loop routines can be dropped into different distribution mechanisms without rework.

The second practice is thinking about thread safety earlier than you think you need to. Designing class members with that future in mind costs almost nothing early and can be enormously expensive to retrofit later. The third is keeping your build system clean. Morley uses CMake and is emphatic about this. “Having a broken build system is far worse than having broken code. It’s much harder to figure out what exactly has gone wrong if your build system is broken.” Build systems accumulate debt quietly and the consequences arrive at the worst possible moment, usually when you need to extract a component into its own library under time pressure.

When metaprogramming becomes the problem

“I’ve seen some horrendous template metaprogramming in my life,” Morley reflects. “I’ve written some horrendous template metaprogramming in my life. I’m going to be the first one to admit that it’s never worth it.”

The target here is the elaborate kind: deeply nested type machinery and SFINAE-heavy enable_if chains. Instantiating a complex template metaprogram can easily double compile time for a single translation unit. At scale, across thousands of files, the cost becomes structural. Morley notes this is exactly why Google kept metaprogramming to an absolute minimum when writing Abseil. Modern C++ has reduced the genuine need for TMP significantly. Concepts and constexpr functions cover much of what engineers used to reach for TMP to solve, with readable syntax and without the compile time penalty.

Lambdas are a different conversation. Used correctly, Morley thinks they are one of the most readability-enhancing tools in the language. Used carelessly, they introduce exactly the kind of invisible coupling that makes the future you miserable. His specific warning is about lambdas that capture and modify values defined far away from where the lambda is used. “Every time you think what is this lambda doing, it’s modifying something that you’ve not looked at for a long time because your screen has been further down the page.” In cases where a lambda is the only place that reads or writes a particular value, Morley argues the value almost certainly belongs inside a class, where the ownership and mutation path are explicit.

Thinking with the machine

Morley uses a road analogy to introduce hardware-aware programming. If you are driving down an unfamiliar road in the dark, you slow down because you cannot see what is ahead. Knowing the road means you can go faster without risk. The system you are running the code on is the road, the code you are writing is the car, and understanding the road conditions means you can drive faster with confidence.

The most practically important piece of hardware to understand is the cache hierarchy. Cache behavior is invisible in the code but determines a large part of actual runtime performance. The canonical example comes from the games industry: the debate between Array of Structs (AoS) and Struct of Arrays (SoA).

If you represent each game entity as a large struct and store those structs in a vector, then iterating over position data means striding across the full width of each struct to reach the next position value. “Big strides are bad for the cache. What you really want is all of the position data to be close together.”

The alternative is to separate data by type, storing all position values together and all velocity values together, so that each field is laid out contiguously in memory rather than interleaved across structs. As Morley puts it: “This transformation basically doubles or quadruples your throughput because now you don’t have to step over all of the useless data in order to update your position.”

Matrix multiplication illustrates the same principle at a more concrete level. In one direction, data access is sequential and cache-friendly. In the other direction, moving between rows in a column means jumping across memory in steps as wide as the entire matrix, which is exactly what the cache is designed to avoid. The standard mitigation is tiling: load a small tile of the matrix, do as much computation as possible on that tile, then move to the next one. Morley walks through a tiled implementation in the book and notes roughly a four-times improvement over the naive approach from tiling alone, before any SIMD or pipelining optimisations are applied.

Branch prediction and SIMD matter too, but Morley is careful about context. For general-purpose application code, the compiler will usually make reasonable decisions, and the latency bottleneck is more often a network call or a disk write than an instruction pipeline stall. These details start to matter seriously when throughput is the primary constraint: training large models, running physics simulations, anything where a microsecond saved per operation compounds into meaningful wall-clock time across billions of invocations. “Taking an extra microsecond to do a calculation is devastating when you have to do that a billion times.”

Concurrency: two models, two risk profiles

Morley distinguishes two fundamentally different multi-threaded scenarios that require different mental models.

The first is data parallelism. A large dataset split across threads, each thread operating on its own independent range with no shared state. “There’s never any overlap. Each thread goes away, does its work, and the results are put in the buffer.” This is relatively safe territory. Parallel algorithms and OpenMP handle much of the machinery, and as long as the data partition is clean and there is no self-referential access, data races are structurally prevented.

The second scenario is harder. Multiple worker threads operating on shared state within a larger system. A dispatch queue is the typical pattern, where a main thread stacks work items and worker threads pull from the queue. Here, ownership discipline becomes non-negotiable.

Morley’s design goal is simple to state and difficult to achieve: only one thread, one function, one whatever should be able to modify a value at any given time. There are two ways to achieve this. The first is architectural: design the program so that each thread exclusively owns its data and never touches another thread’s values. The second is synchronisation: use atomics, mutex-locked values, or other thread-safe mechanisms to control access.

Morley urges engineers to read the synchronisation documentation carefully before reaching for any of these tools. Deadlocks happen when engineers use them without fully understanding their semantics. The architectural solution is always preferable when achievable: if you can design the program so that each thread never touches another thread’s data, you eliminate the entire class of problem. When you cannot, synchronisation primitives are available, but the burden of correctness falls entirely on the engineer.

Memory safety and the Rust argument

The issue closes on a topic that provokes strong reactions in C++ communities. With around 70% of serious security vulnerabilities attributed to memory safety failures in C and C++ code, the question of whether the language itself is the problem is not academic.

Morley’s answer is direct. “Go and learn some Rust.”

He anticipates the objection. “A lot of C++ programmers turn their nose up when Rust is mentioned. Generally, the feeling is: we don’t need Rust, we can do all of this in C++. But that’s not the point.” The point is that the Rust compiler forces a specific kind of thinking about ownership and lifetime that C++ leaves entirely to the developer. Rust’s sync and send traits enforce thread safety at the type system level. Unsafe code is marked explicitly, which means the developer makes a deliberate choice to step outside safe boundaries rather than doing so accidentally. “Learning a bit of Rust will make you better at writing safe C++. The reverse is not true.”

For engineers who are not yet ready to invest in Rust, Morley's immediate C++ recommendations are practical and specific. Prefer std::array over C-style arrays. Use smart pointers instead of manual memory management; writing operator new directly in application code is an antipattern at this point. Use std::span rather than raw pointers when passing data around. Avoid the C standard library IO functions entirely: gets, puts, sprintf, and their relatives have documented vulnerabilities and safer equivalents exist.

“The best-case scenario for a bad memory access is a crash. If it goes unnoticed, it could happen for months before you notice that this has been producing garbage the entire time. By which point you’ve wasted months.”

The throughline from Part 1 holds here. Whether the subject is decomposition, abstraction, maintainability, or memory safety, Morley keeps returning to the same discipline: be conscious of what you are doing, understand the costs of your choices, write for the person who will read the code next, and treat the machine as a collaborator rather than a black box.

🔍 In case you missed it…

The complete Chapter 1 from Sam Morley’s book, The C++ Programmer’s Mindset 👇

🛠️ Tool of the Week

Google Benchmark — A microbenchmark support library for C++

Google Benchmark is an open-source framework for measuring the performance of isolated C++ code so you can test cache-aware designs and tight-loop optimizations with high precision.

Learn more about Google Benchmark

📎 Tech Briefs

• C++26 Finalizes Technical Work — The ISO committee completed the C++26 draft in London, locking in features like compile-time reflection, memory safety improvements, and contracts ahead of final approval.

• Rust 1.94.1 Shipped — A point release that fixes compiler regressions, resolves clippy false positives on match arms, and updates Cargo dependencies to address critical CVEs.

• RustRover 2026.1 Released — JetBrains introduced cargo-nextest integration, a call hierarchy view, LLDB 21 upgrades, and improved DWARF indexing for procedural macros.

• CppCon 2026 Opens Registration — Registration is open for the Colorado event, with sessions covering AI infrastructure, low-latency design, and hardware-aware performance.

• Linux Kernel 7.0 Approaches — After Linux 6.19, the merge window for 7.0 is now open, pointing to a major version release expected later this April.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

A full-lifecycle workshop on designing, securing, evaluating, and optimizing agent systems that hold up in production.

Online: March 29 | 10:30 AM - 4:00 PM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for the price of 1.

Register now

✍️ From the editor’s desk,

Welcome to the 40th issue of Deep Engineering!

The conversation around AI infrastructure is heating up. Nvidia this week pointed to more than $1 trillion in AI chip revenue opportunity by 2027, explicitly tying that outlook to growing demand for inference. And when inference at scale becomes a boardroom number, model efficiency stops being a niche engineering concern and starts shaping cost control, product design, and build versus buy decisions. That is the context in which small language models are becoming harder to ignore.

The case is not that general-purpose LLMs are going away – it is that a large share of production workloads, classification, structured extraction, and domain-specific reasoning, do not need an LLM. Serving a 7-billion parameter SLM runs 10 to 30 times cheaper than a 70-billion parameter LLM, and that gap compounds across millions of requests. For engineering leaders, the question is no longer whether SLMs belong in the stack – it is how to train them to reason well, how to route across multiple specialized models at runtime, and how to govern correctness when AI is accelerating the pace of change.

This week’s feature is based on a conversation with Karun Thankachan, Senior Data Scientist at Walmart. We dig deep into how to treat reasoning as a budgeted resource, when routing across specialized models beats relying on a single large one, and why context engineering is winning over fine-tuning for many teams right now.

And we also have a thought leadership piece by Lee Peterson, VP of Secure WAN Product Management at Cisco, on why the network layer underneath agentic systems matters just as much as the models running on top of it.

Let’s get started.

Stop Building Vault

Secrets, PKI, & PAM in one platform. Postgres-backed. No custom orchestration. Flexible deployment.

Start for free today →

The Case for Small Language Models in Real Systems

by with Karun Thankachan.

Karun Thankachan’s conversation with us keeps circling back to one production truth:

The “best” model is the one that meets your accuracy target inside your cost and latency budget.

In early phases of a product, teams often use large general-purpose LLMs to explore what’s possible. But once the user experience stabilizes, the engineering focus shifts – toward predictable latency, controllable spend, and repeatable quality. That shift is where small language models (SLMs) start to matter more than giant LLMs, because many real systems don’t need universal reasoning – they need reliable, narrow reasoning at scale.

Recent research and releases support this trajectory. For example, Zihao An et al. (Jan 20, 2026) show that a 0.6B-parameter reasoning model (ReasonLite) can reach 75.2% accuracy on AIME 2024, and they attribute it to distillation and training recipe design rather than sheer parameter count. This is exactly the kind of result that makes SLMs a serious production option once you know what task you need to solve.

Cost-effective reasoning is a budget problem

Thankachan’s core argument is not that LLMs are bad. It’s that general-purpose reasoning is expensive overkill for many high-volume workflows (customer support classification, catalog normalization, routing, anomaly triage, internal copilots for one domain). When a feature goes from thousands of calls to millions, the economics change.

A key point from the research community is that reasoning often becomes expensive because teams implicitly pay for long reasoning traces even when a request is simple. Kun Liang et al. (Jan 13, 2026) describe this problem directly: long-form chain-of-thought reasoning can improve accuracy, but applying “overlong reasoning” uniformly at inference time creates “substantial and often unnecessary computational cost.” They propose ORBIT, which aims to make reasoning behavior controllable across multiple “budgets.”

That maps cleanly to production reality: you don’t want a single mode of intelligence. You want a system that can do “fast and cheap” when it can, and “slow and deep” only when it must.

Teaching SLMs to reason without bloating latency

The approach Thankachan describes in the interview treats SLM training like an engineering workflow: combine distillation techniques, program-aided verification, and evaluation probes to close the gap between a teacher model and a smaller student.

What’s notable is that the same idea is showing up in recent public work: reasoning performance in small models is increasingly a function of distillation data quality, training curriculum, and trace control, not only scale.

In the AMD ReasonLite write-up, the authors describe a two-stage distillation curriculum: first fine-tune on short chain-of-thought for efficiency, then fine-tune on long chain-of-thought for peak accuracy. They also report that the “Turbo” variant tends to generate shorter outputs and offers a better efficiency/accuracy balance, while the long-trace variant reaches higher peak performance. This lines up with the production emphasis Thankachan describes: if your model is trained to be verbose, it will be slow and costly; if you want predictable serving, you must treat thinking length as a first-class knob.

A practical pattern for senior engineers is to separate training-time compute from serving-time compute:

• Use heavier techniques (like generating multiple traces, voting, or deeper teacher reasoning) during dataset creation and training, where batching and offline pipelines reduce the pain.

• Enforce strict budgets at inference (max reasoning tokens, max latency, max retries) and treat budget violations as signals to improve training data or shorten traces.

The leadership takeaway: budgeting belongs in the contract, not just in infrastructure dashboards after costs explode.

Orchestrating many small models beats one giant model

The SLM-Fusion framing Thankachan describes – routing across specialized models, merging where it helps, and wrapping the whole thing behind an OpenAI-compatible gateway – reflects a broader trend:

Modern systems are becoming mixtures, not monoliths.

Recent work in retrieval-heavy QA shows why. Yasaman Zarrinkia, Venkatesh Srinivasan, and Alex Thomo (Mar 18, 2026; v2) report a striking result when evaluating a Graph-RAG system: 77%–91% of questions contain the gold answer in the retrieved context, yet end-to-end accuracy is only 35%–78%, and 73%–84% of errors are reasoning failures.

That finding matters for architecture decisions. It implies that “better retrieval” alone isn’t enough; you need better reasoning structure and better context shaping. The same paper proposes two augmentations – structured prompting plus graph-walk compression – and then shows something that should make engineering leaders pay attention: with question-type routing, an open-weight Llama-8B configuration can match or exceed an unaugmented Llama-70B baseline across multiple benchmarks at about 12× lower cost.

This is the pragmatic case for SLM-centric production:

• Route by intent/domain and cost budget, not just by “best model available.”

• Specialize models for stable tasks (where fine-tuning or distillation pays off).

• Escalate to a larger model only when uncertainty is high or stakes demand it.

Why context engineering is beating fine-tuning right now

Thankachan argues that industry attention is shifting toward RAG and context engineering because it can be cheaper and easier to operationalize than constant fine-tuning – especially when domain knowledge changes frequently.

The Graph-RAG results above reinforce this. Even when the answer is present in retrieved context, the failure mode is often reasoning over context, not retrieval itself. That pushes teams toward two concrete engineering priorities:

• Structure the prompt so the model reasons in a way that matches the data representation (for Graph-RAG, they use SPARQL-style decomposition).

• Compress context so the model sees less noise and wastes fewer tokens, while still preserving the minimal information needed to answer.

For architects, the key decision lens is: fine-tuning changes the model; context engineering changes the problem presented to the model. When requirements or source documents change weekly, the second approach often has a faster operational loop.

What’s next – budget-aware reasoning and diffusion models

Thankachan flags diffusion models as a trend to watch. The key reason is not hype – it’s that diffusion-based language models may offer new efficiency controls that differ from token-by-token autoregressive decoding.

A concrete example: Vittorio Rossi et al. (Mar 6, 2026) argue that diffusion language models (DLMs) can waste compute because they run denoising steps over a fixed maximum length even when the desired response is short. They propose a zero-shot mechanism to estimate required output length and crop the context window before generation, reporting large FLOP reductions without statistically significant performance loss (and improvements in 2 of 4 benchmarks).

This matters because “right-sized reasoning” is becoming an explicit research goal across paradigms – whether you do it with controllable budgets in autoregressive models (as in ORBIT) or with length control in diffusion models.

Finally, distillation itself is getting more modular. Shaoxiong Yang et al. (Feb 1, 2026) propose FutureMind, which combines adaptive distillation with a multi-stage reasoning pipeline and retrieval guidance – explicitly acknowledging that SLMs are attractive for low-latency settings but often struggle on knowledge-intensive tasks without structured reasoning and retrieval help.

Key Takeaways

• Treat reasoning as a budgeted resource. Put token limits and latency targets into your API contract, not just your monitoring dashboards, and design “fast vs deep” modes intentionally.

• Distillation makes small models competitive when the recipe is right. Evidence from subbillion models shows that curriculum design and trace supervision can close surprising amounts of the gap.

• RAG failures are often reasoning failures, not retrieval failures. Even when the answer is in-context, models may fail to use it – so prompt structure and context compression matter.

• Routing is a cost lever. With the right augmentations and routing, smaller open-weight models can rival much larger baselines at large cost reductions.

• Watch diffusion LMs for new efficiency primitives. Length-aware diffusion decoding suggests a different path to controlling wasted compute, which may matter for production workloads dominated by short answers.

🔍 In case you missed it…

💡 Industry Perspective

Agentic AI Is Redefining Edge Infrastructure

Thought leadership piece by with Lee Peterson, VP of Secure WAN Product Management at Cisco.

As agentic systems move closer to the edge, the network underneath them matters just as much as the models running on top. Peterson makes the case that organizations still designing around centralized control will hit a wall at exactly the wrong moment, and that getting edge compute and networking right is not an upgrade but a rethink from the ground up.

Packt Deep Engineering

Agentic AI Is Redefining Edge Infrastructure

Artificial intelligence is entering a new phase with agentic AI, where autonomous systems perceive, decide, act, and learn without constant human oversight, operating independently across distributed environments while collaborating with other agents in real time…

Read more

a month ago · Saqib Jan

🛠️ Tool of the Week

DistillKit — open-source knowledge distillation toolkit for language models

Highlights:

• Two distillation methods in one toolkit: Supports both logit-based distillation and hidden states-based distillation, which aligns intermediate layer representations and allows distillation across different model architectures.

• Offline distillation at scale: An advanced logit compression system using polynomial approximation, error-diffusion quantization, and bit-packing makes it practical to train from pre-captured teacher outputs without running the teacher model live during every training step.

• Composable loss functions: Mix and match KL divergence, JSD, TVD, ranking losses, and hidden state alignment to control exactly what the student model learns and how aggressively it is pushed toward the teacher’s behaviour.

Learn more about Distillkit

📎 Tech Briefs

• Hugging Face Spring 2026 open source report released: The platform now has 13 million users and over 2 million models, with Chinese models accounting for 41% of downloads and robotics datasets growing 23x to become the largest category.

• vLLM 0.18.0 released with 30.8% throughput improvement and new Realtime API: The release ships async scheduling with Pipeline Parallelism delivering a 30.8% end-to-end throughput gain, a WebSocket-based Realtime API for streaming audio, and expanded hardware support across NVIDIA, AMD, Intel, and TPU.

• OpenAI launches GPT-5.4 mini and GPT-5.4 nano: Two smaller models optimized for speed and efficiency in coding, subagents, and high-volume tool use, continuing the industry shift toward task-specific models over general-purpose frontier inference.

• NVIDIA’s AI Grid reference design cuts edge inference costs by 76% in early benchmarks: The architecture distributes AI compute across telco network nodes, with Comcast benchmarks showing sub-500ms latency at P99 burst traffic and 80.9% throughput gains over a centralized deployment.

• Anthropic announces the Anthropic Institute: A new research body focused on the economic, societal, and security impacts of advanced AI, signalling that frontier labs are formalizing impact analysis as a structured part of model development and evaluation.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

A full-lifecycle workshop on designing, securing, evaluating, and optimizing agent systems that hold up in production.

Online: March 29 | 10:30 AM - 4:00 PM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for the price of 1.

Register now

✍️ From the editor’s desk,

Welcome to the 39th issue of Deep Engineering!

In Issue #34 we spoke with José Dimas Luján Castillo and Ron Veen, authors of Kotlin for Java Developers, to unpack practical decision lenses for modernizing JVM systems. In the past week, JetBrains released Kotlin 2.3.20 with a simpler Maven setup and compiler-plugin improvements, and OpenAI added GPT-5.4 mini to Codex as a faster, lighter model for coding tasks. The language stack is evolving, and so are the AI tools teams now use to change production code at speed.

But how can a large Java codebase be migrated without turning modernization into a rewrite, a slowdown, or a source of subtle semantic risk? We are collaborating again with Ron Veen, whose article answers that by treating migration as a controlled, layer-by-layer transformation—one where Kotlin’s gains in null safety and expressiveness matter, but where the harder engineering work lies in framework compatibility, review discipline, and keeping humans accountable while AI accelerates the mechanics.

Ron Veen is a JVM veteran with over 20 years of experience across enterprise systems, from mainframes to modern microservices. Veen is an Oracle Certified Java Programmer and certified Sun Business Component Developer. As a Lead Developer at Team Rockstars IT and a regular international speaker, Veen brings both hands-on experience and architectural perspective.

Let’s get started

Mike Scientific Repo: Tons (> 1200) of FREE Math, AI, Machine Learning Books, All reviews (almost 600), presentations and more learning resources: https://github.com/merlihson/scientific-resources

Science and AI with Mike: Telegram channel - fresh deep learning paper reviews and free books: https://t.me/science_and_ai_with_mike_english/

From Java to Kotlin at Scale – with AI in the Loop

by with

Most large engineering organizations are sitting on a paradox.

Their core systems – often millions of lines of Java – are stable, battle-tested, and deeply valuable. But they’re also increasingly out of step with how modern teams want to write software: safer by default, more expressive, and easier to reason about. Kotlin promises exactly that. The problem is not why to migrate, it’s how to do it without freezing product velocity or introducing subtle, system-wide risk.

This is where Ron Veen’s migration story becomes relevant. Instead of treating Java-to-Kotlin as a one-shot rewrite or a tooling exercise, he approaches it as a controlled, layer-by-layer transformation – reconstructed on an OpenMRS codebase to mirror real-world enterprise constraints. The twist: AI is not just assisting but actively accelerating the migration – while humans remain accountable for every semantic decision.

What emerges is not just a migration playbook, but a shift in how we think about large-scale code evolution. The bottleneck is no longer writing code – it’s governing correctness at speed. And in that world, success depends less on conversion tools and more on how teams handle semantics, frameworks, and oversight under AI-assisted velocity.

To this end, today’s issue focuses on three action points for senior engineers and engineering leaders:

• Kotlin migrations succeed when you treat them as a semantic change (types, nullability, and idioms), not a mechanical translation.

• Framework constraints (Spring/Hibernate/JPA) are where migrations usually get stuck – unless you plan for Kotlin’s “final by default” stance and use the right compiler plugins.

• AI can compress weeks of conversion work into days, but it shifts the hard work to governance: review strategy, testing cadence, and repeatable patterns.

1: Treat migration as a semantic refactor, not a syntax swap

One of the easiest mistakes in a Java-to-Kotlin migration is to treat it as a tooling problem. Run the IDE converter, clean up a few warnings, and move on. Veen’s work makes it clear why that approach fails: working Kotlin isn’t the goal – idiomatic Kotlin is. And the gap between the two is where most of the risk, and value, lies.

Veen’s core point is easy to miss if you’ve only used IDE “convert Java to Kotlin” tools: working Kotlin is not the goal – idiomatic Kotlin is. The difference shows up most sharply in nullability, because Kotlin forces you to say what can be null and what cannot.

That’s also why Kotlin’s Java interop rules matter for incremental migration. Kotlin can call existing Java “naturally,” but when Kotlin reads Java types, it must assume any reference might be null. Kotlin handles this by using platform types, which relax compile-time null checks and can still fail at runtime if your assumptions are wrong. For leaders, the practical implication is: your migration policy must include a nullability strategy (what becomes T, what becomes T?, and when you tighten types).

Veen’s entity conversion example shows why this is a worthwhile discipline:

Java Version

@Entity 
@Table(name = "location_tag") 
@Audited 
@AttributeOverride(name = "name", column = @Column(name = "name", nullable = false, length = 50)) 
public class LocationTag extends BaseChangeableOpenmrsMetadata { 
 
    private static final long serialVersionUID = 7654L; 
    private Integer locationTagId; 
    
    public LocationTag() { 
    } 
 
    public LocationTag(Integer locationTagId) { 
        this.locationTagId = locationTagId; 
    } 
 
    public LocationTag(String name, String description) { 
        setName(name); 
        setDescription(description); 
    } 
 
    @Id 
    @GeneratedValue(strategy = GenerationType.IDENTITY) 
    @Column(name = "location_tag_id", nullable = false) 
    public Integer getLocationTagId() { 
        return this.locationTagId; 
    } 
 
    public void setLocationTagId(Integer locationTagId) { 
        this.locationTagId = locationTagId; 
    } 
 
    @Override 
    public Integer getId() { 
        return getLocationTagId(); 
    } 
 
    @Override 
    public void setId(Integer id) { 
        setLocationTagId(id); 
    } 
 
    @Override 
    public String toString() { 
        return getName() != null ? getName() : ""; 
    } 
} 

Kotlin Version

@Audited 
@Entity 
@Table(name = "location_tag") 
@AttributeOverride(name = "name", column = Column(name = "name", nullable = false, length = 50)) 
class LocationTag() : BaseChangeableOpenmrsMetadata() { 
 
    @Id 
    @GeneratedValue(strategy = GenerationType.IDENTITY) 
    @Column(name = "location_tag_id", nullable = false) 
    var locationTagId: Int? = null 
 
    constructor(locationTagId: Int?) : this() { 
        this.locationTagId = locationTagId 
    } 
 
    constructor(name: String?, description: String?) : this() { 
        this.name = name 
        this.description = description 
    } 
 
    override fun toString(): String = name ?: "" 
 
    override fun getId(): Int? = locationTagId 
 
    override fun setId(id: Int?) { 
        locationTagId = id 
    } 
 
    companion object { 
        private const val serialVersionUID = 7654L 
    } 
} 

With this approach, boilerplate shrinks, but more importantly, intent becomes visible in the type system.

For senior reviewers, the takeaway is that “semantic migration” is reviewable. You can ask: Do the nullability markings reflect reality? Did we remove boilerplate without losing invariants? Are we leaning on Kotlin’s strengths (properties, expression bodies, Elvis operator) in ways that reduce bug surface?

2: Framework constraints are the hidden migration backlog

Most Java-to-Kotlin migration pain doesn’t come from language syntax. It comes from frameworks that depend on bytecode tricks: proxies, reflection, and runtime enhancement.

Two Kotlin compiler plugins exist precisely to reduce this friction:

• The all-open plugin addresses Kotlin’s default of making classes and members final. Kotlin’s docs explicitly call out that this default is inconvenient for frameworks such as Spring AOP, which require classes to be open so proxies can be created.

• The no-arg plugin generates a synthetic zero-argument constructor for annotated classes, which helps frameworks like JPA instantiate entities via reflection even when Kotlin doesn’t define a zero-arg constructor explicitly.

These are leadership-level concerns because they determine whether your migration scales. If teams manually sprinkle open and constructors across hundreds of files with inconsistent rules, you get a “looks migrated, runs broken” outcome.

Veen shows this in his service implementation example:

@Service("medicationDispenseService") 
@Transactional 
open class MedicationDispenseServiceImpl : BaseOpenmrsService(), MedicationDispenseService { 
 
    @Autowired 
    lateinit var dao: MedicationDispenseDAO 
 
    @Transactional(readOnly = true) 
    override fun getMedicationDispense(id: Int?): MedicationDispense? = 
        dao.getMedicationDispense(id) 
 
    override fun saveMedicationDispense(dispense: MedicationDispense): MedicationDispense { 
        require(dispense != null) { "MedicationDispense cannot be null" } 
        return dao.saveMedicationDispense(dispense) 
    } 
 
    @Transactional(readOnly = true) 
    override fun getMedicationDispenses( 
        patient: Patient, 
        drug: Drug?, 
        dateRangeParam: DateRangeParam?, 
        startIndex: Int?, 
        limit: Int? 
    ): List<MedicationDispense> { 
        require(patient != null) { "Patient cannot be null" } 
        return dao.getMedicationDispenses(patient, drug, dateRangeParam, startIndex, limit) 
    } 
} 

In the preceding code, the open keyword is a runtime requirement, not a stylistic choice, when frameworks proxy your classes for transactions.

A practical leadership move here is to standardize how these requirements are met: prefer compiler plugins (where feasible) over manual edits. Kotlin’s docs note that you can use a Spring-specific wrapper (kotlin-spring) on top of all-open, and that Spring project templates typically enable it by default. Likewise, Kotlin documents that kotlin-jpa is a wrapper on top of no-arg and automatically targets common JPA annotations.

3: AI accelerates conversion, but governance determines quality

Veen’s migration stance is not “AI replaces engineers.” It’s closer to: AI replaces repetitive conversion work, so humans can spend time on correctness and architecture.

Recent developments reinforce why that governance burden is growing. Anthropic positions Claude Opus 4.6 as improved for coding tasks that involve longer planning, more reliable operation in larger codebases, and stronger debugging and code review – with a very large context window intended for long, multi-step work. In practice, Anthropic’s telemetry-based research suggests users are already letting Claude Code run autonomously for longer on the most complex work: the 99.9th percentile “turn duration” in Claude Code nearly doubled from under 25 minutes to over 45 minutes over a three-month span, and experienced users enable auto-approve more often while interrupting when needed.

That pattern maps directly onto migration risk. The faster you transform files, the easier it is to:

• Propagate a wrong nullability decision across a layer.

• Preserve a Java anti-pattern in Kotlin clothing.

• Break a proxy/reflection expectation in subtle ways.

For senior engineers and architects, the emphasis shifts to controlling the blast radius. Veen’s own best practices (small batches, frequent commits, tests between batches, and explicit pattern guides) are not process overload – they are how you turn “AI speed” into “organizational safety.”

Even Kotlin itself nudges teams toward explicitness where Java might have relied on convention. For example, Kotlin treats Java’s checked exceptions as unchecked – the compiler won’t force catches – so interoperability requires deliberate documentation patterns (for example, @Throws for Java callers, as Veen shows).

A smaller but telling example from Veen’s validator conversion shows how Kotlin idioms can reduce branching and clarify intent – if reviewers insist on it:

@Component 
@Handler(supports = [EncounterRole::class]) 
class EncounterRoleValidator : RequireNameValidator() { 
 
    @Autowired 
    private lateinit var encounterService: EncounterService 
 
    override fun validate(obj: Any, errors: Errors) { 
        super.validate(obj, errors) 
        if (errors.hasErrors()) return 
 
        val encounterRole = obj as EncounterRole 
        val name = encounterRole.name?.trim() 
 
        if (!name.isNullOrBlank()) { 
            val duplicate = encounterService.getEncounterRoleByName(name) 
            if (duplicate != null && encounterRole.uuid != duplicate.uuid) { 
                errors.rejectValue( 
                    "name", 
                    "EncounterRole.duplicate.name", 
                    "Specified Encounter Role name already exists" 
                ) 
            } 
        } 
    } 
} 

Key Takeaways

• Migration success depends on semantic decisions, not just conversion volume. Lock down your nullability rules early, and review them as architecture, not style.

• Use Kotlin’s compiler plugins to meet framework requirements at scale. Kotlin is final by default; all-open (and wrappers like kotlin-spring) exist because frameworks such as Spring AOP need open classes.

• Plan for reflection and JPA. If your entities need zero-arg constructors for runtime instantiation, no-arg (and wrappers like kotlin-jpa) can generate them safely.

• AI makes migration fast, but it increases the need for governance. Long-context models are designed for sustained, multi-step coding work.

• Adopt “human-in-the-loop” guardrails that match how agents are used in practice – users increasingly auto-approve and let agents run longer, intervening when needed.

🧠 Expert Insight

🔍 In case you missed it…

🛠️ Tool of the Week

Exposed — Kotlin’s type-safe SQL framework, now at 1.0

Exposed is JetBrains’ open-source SQL framework for Kotlin, hitting 1.0 this week with R2DBC support for reactive database access and a guaranteed stable API, making it a production-ready database layer for teams adopting Kotlin.

📎 Tech Briefs

• Java 26 released — Java 26 ships ten JEPs including HTTP/3 for the HTTP Client API, lazy constants in second preview, and an AOT cache that now works with any garbage collector including ZGC.

• Kotlin 2.3.20 released — The release removes significant manual Maven setup by automating source root configuration and stdlib inclusion, and adds name-based destructuring declarations to the language.

• Java 26 support lands in IntelliJ IDEA — IntelliJ IDEA adds full Java 26 support this week, including inspections and quick-fixes for lazy constants, primitive patterns, and the extended ahead-of-time cache for ZGC.

• Kotlin Foundation joins Google Summer of Code 2026 — The programme is accepting applications for projects spanning Swift-to-Kotlin interop, tail call support in Kotlin/Wasm, and a Kotlin education landscape research report, with the deadline on March 31.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

AI can refactor your code in seconds, but it can introduce subtle breakages just as quickly. Learn how to combine AI-assisted reasoning with AST-based structural guarantees to refactor code safely.

Online: March 14 | 10:00 AM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for $48
✓ Includes 300+ page Mastering ast-grep ebook

Register now

✍️ From the editor’s desk,

Welcome to the 38th issue of Deep Engineering!

The rise of AI-generated code is putting pressure on a familiar engineering constraint: review does not scale as easily as output. Anthropic’s own recent research on Claude Code points in the same direction. It found that longer autonomous coding sessions are becoming more common, and that experienced users increasingly shift from step-by-step approval to letting the tool run and intervening when needed. But, if more code is being written with less direct review, how do engineers stop poor design from shipping in the first place?

Anthropic’s new Code Review system for Claude Code is one answer at the downstream end. It runs five specialized reviewers in parallel, scores findings on a confidence scale, and only posts high-confidence comments back to GitHub. That is a useful safeguard. But it also clarifies the limit of review itself. The more teams rely on layered automation to inspect pull requests, the more important the original design decisions become. Review can catch defects. It cannot compensate for weak boundaries, poor abstractions, or code that was hard to reason about from the start.

That is the context for this week’s Expert Insight from Steven F. Lott. Lott has been programming since the era when computers were large, expensive, and rare, and has worked with Python since the 1990s. He is the co-author of the newly published fifth edition of Python Object-Oriented Programming, updated for Python 3.13 with added material on areas such as type hints, testing, and professional software engineering practice. In this issue, his piece looks at object-oriented design as a practical engineering discipline: dependency injection, protocols, and duck typing as choices that shape coupling, change, and maintainability in production systems. We have also included Chapter 1, “Object-Oriented Design,” from the new edition for readers who want the full foundation.

Let’s get started.

PulseMCP Newsletter

Our friends at PulseMCP recently wrote up a great post on the rise of agentic engineering. It highlights both the very public, high profile case studies of top tier engineering organizations adopting the practice, and gets into the weeds of what’s working in the PulseMCP team’s personal coaching of engineering teams trying to make the transition from “AI pair programming” to “autonomous agents”.

Check out PulseMCP →

🧠Expert Insight

Part 2: Getting Started with Object-Oriented Programming in Python

by

Python provides three primary ways to structure immutable data: dataclass with frozen=True, NamedTuple, and TypedDict. Each carries different trade-offs that matter in production systems.

A dataclass with frozen=True provides immutability through runtime enforcement. When you attempt to modify a frozen dataclass instance, Python raises an exception. This makes frozen dataclasses useful when you need immutability guarantees but also need features like inheritance, custom methods, or post-initialization processing. The runtime check adds minimal overhead and catches mutation attempts immediately in development and testing.

NamedTuple provides immutability through tuple semantics. Because NamedTuple instances are actual tuples under the hood, they inherit tuple’s immutability at the language level. This makes NamedTuple marginally faster than frozen dataclass and ensures that mutation is structurally impossible rather than runtime-prevented. NamedTuple works well for simple data containers that do not require inheritance or complex initialization logic.

TypedDict does not provide immutability at all. TypedDict exists purely for type checking and documentation. At runtime, a TypedDict is just a regular dictionary with no special behavior or constraints. This makes TypedDict appropriate when you need to type-hint dictionary structures for API contracts or configuration, but you do not control the actual dictionary creation or mutation.

The choice between these three depends on what guarantees your system actually requires. If you need inheritance or custom methods, use frozen dataclass. If you need maximum performance for simple data structures, use NamedTuple. If you need to type-hint existing dictionary-based interfaces without changing runtime behavior, use TypedDict.

Hashability and when it matters

Immutable objects in Python can be hashable, which means they can be used as dictionary keys or stored in sets. This is not just a convenience feature. Hashability enables certain architectural patterns that would otherwise require significantly more complex code.

When you need to deduplicate objects, track which objects have been processed, or use objects as cache keys, hashability becomes a structural requirement rather than a nice-to-have feature. A frozen dataclass or NamedTuple containing only hashable fields is automatically hashable. A regular mutable dataclass is not hashable because its contents could change after being added to a set or used as a dictionary key, which would break the fundamental contract of hash-based collections.

SOLID principles in Python

The SOLID principles originated in statically-typed object-oriented languages like Java and C++, but they translate to Python with some adjustments for Python’s dynamic nature and duck typing.

Single Responsibility Principle states that a class should have one reason to change. In Python, this often means keeping data structures separate from business logic, keeping validation logic separate from transformation logic, and keeping I/O operations separate from computation. When a class starts accumulating methods that could logically belong to different concerns, it is time to split it.

Open/Closed Principle states that software entities should be open for extension but closed for modification. In Python, this is typically achieved through composition and protocols rather than deep inheritance hierarchies. Instead of modifying existing classes to add behavior, you compose new classes that delegate to existing ones or define protocol-based interfaces that new classes can implement without touching existing code.

Liskov Substitution Principle states that subtypes must be substitutable for their base types. In Python, this means that if your code expects a file-like object, any object that implements the file protocol (read, write, close) should work correctly. The principle prevents surprising behavior when substituting one implementation for another.

Interface Segregation Principle states that clients should not depend on interfaces they do not use. In Python, this translates to keeping protocols narrow and focused rather than creating large monolithic base classes with dozens of methods that most implementations leave empty or raise NotImplementedError.

Dependency Inversion Principle states that high-level modules should not depend on low-level modules; both should depend on abstractions. In Python, this is achieved through dependency injection and protocols. Instead of instantiating concrete dependencies inside a class, you pass dependencies in as constructor arguments, and instead of depending on concrete types, you depend on protocols that define the interface you actually need.

Dependency injection without frameworks

Dependency injection in Python does not require a framework. The pattern is simple: instead of creating dependencies inside a class, pass them in as constructor arguments.

python

# Without dependency injection
class OrderProcessor:
    def __init__(self):
        self.payment_gateway = StripeGateway()  # Hard dependency
        self.email_service = SendGridService()  # Hard dependency
    
    def process(self, order):
        self.payment_gateway.charge(order.amount)
        self.email_service.send_confirmation(order.email)

# With dependency injection
class OrderProcessor:
    def __init__(self, payment_gateway, email_service):
        self.payment_gateway = payment_gateway
        self.email_service = email_service
    
    def process(self, order):
        self.payment_gateway.charge(order.amount)
        self.email_service.send_confirmation(order.email)

The second version allows you to swap payment gateways or email services without modifying OrderProcessor. It allows you to test OrderProcessor with mock implementations that do not actually charge credit cards or send emails. It makes the dependencies explicit in the constructor signature rather than hidden inside the implementation.

Protocols and duck typing

Python’s protocols allow you to define interfaces without requiring inheritance. A protocol specifies what methods an object must have without requiring the object to inherit from a specific base class.

python

from typing import Protocol

class PaymentGateway(Protocol):
    def charge(self, amount: float) -> bool: ...

class StripeGateway:
    def charge(self, amount: float) -> bool:
        # Stripe implementation
        return True

class PayPalGateway:
    def charge(self, amount: float) -> bool:
        # PayPal implementation
        return True

Both StripeGateway and PayPalGateway satisfy the PaymentGateway protocol without inheriting from it. Type checkers like mypy will verify that any object passed where a PaymentGateway is expected actually implements the charge method with the correct signature.

This is duck typing with type safety. You get the flexibility of duck typing (if it walks like a duck and quacks like a duck, it is a duck) combined with static verification that the duck actually has the methods you are going to call on it.

Continue Reading

Chapter 1: Object-Oriented Design

The complete foundation chapter from Object-Oriented Python by Steven Lott, from Python Object-Oriented Programming (Fourth Edition) that teaches how to build robust and maintainable object-oriented Python applications and libraries.

Chapter 1 Object Oriented Design (from Python Object Oriented Programming Build Robust And Maintainable Object Oriented Python Applications And Libraries , Fourth Edition)

1.05MB ∙ PDF file

Download

Download

🔍In case you missed it…

🛠️ Tool of the Week

mypy — static type checker for Python

Highlights:

• Catches type errors before runtime: Analyzes Python code statically to detect type inconsistencies, function signature mismatches, and attribute errors before the code ever runs, preventing entire classes of bugs from reaching production.

• First-class support for OOP patterns: Understands inheritance hierarchies, protocols, generics, and abstract base classes, making it especially valuable for object-oriented codebases where polymorphism and interface contracts matter.

• Gradual typing for existing projects: Works alongside untyped code, allowing you to add type hints incrementally to legacy projects without requiring a full rewrite, and can be configured to enforce strict typing only where needed.

Learn more about mypy

📎 Tech Briefs

• Python 3.15.0a7 (March 10, 2026) — The seventh alpha for Python 3.15 gives developers an early look at changes that matter to Python design work, including a built-in frozendict, typed extra items for TypedDict, TypeForm, and further JIT performance gains.

• Python 3.12.13, 3.11.15, and 3.10.20 security releases (March 3, 2026) — Python shipped source-only security releases for 3.10–3.12 that fix header-injection issues, control-character handling bugs in HTTP and cookies, and bundled libexpat vulnerabilities.

• Copilot code review now runs on an agentic architecture (March 5, 2026) — GitHub moved Copilot code review to an agentic tool-calling architecture that pulls broader repository context so review comments are higher-signal, lower-noise, and more grounded in architectural intent.

• 60 million Copilot code reviews and counting (March 5, 2026) — GitHub says Copilot code review usage has grown 10x to more than one in five code reviews on GitHub, and says its current system is optimized around accuracy, signal, and speed rather than comment volume.

• Request Copilot code review from GitHub CLI (March 11, 2026) — GitHub now lets developers request Copilot review directly from the terminal with gh pr edit and gh pr create, pulling AI review more tightly into the existing pull-request workflow.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Back by demand (3rd run). Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member. Online: Sat Apr 11, 10:30 AM–Sun Apr 12, 4:00 PM ET. Limited seats.

Register now

✍️ From the editor’s desk,

Hello, It’s a privilege to take over as the new Editor-in-Chief of Deep Engineering, which already has a strong foundation and readership.

With support from colleagues at Packt, I also plan to expand coverage into system design, cloud native applications, platform engineering, and other areas of modern software development where engineering leaders have strong interest.

Thank you.

Welcome to the 37th issue of Deep Engineering!

This week’s Android Security Bulletin includes a familiar warning for anyone who builds systems close to the metal: “There are indications that CVE-2026-21385 may be under limited, targeted exploitation.” Whatever the specific exploit chain looks like, memory corruption remains a production-grade failure mode, not a theoretical one.

This week’s Expert Insight comes from Shreyans Doshi (ex Flow Traders, Morgan Stanley), a software engineer who has spent the last several years building trading infrastructure in C++ and Rust. His article, From NIC to P99: Engineering Low-Latency C++ Trading Systems in 2026, walks through the full stack of decisions required to hit a p95 wire-to-wire latency under 50 microseconds. It covers kernel bypass and memory pool design, NUMA affinity, hardware clock selection, and observability at nanosecond granularity.

This issue answers three questions that recur in performance-critical systems (trading and otherwise):

• Where the latency budget actually goes once you include the NIC, kernel boundary, CPU topology, caches, and clocks—not just business logic.

• What belongs on the critical path vs. off it, using a clean pipeline model (ingestion → checks → pricing → strategy → OMS).

• How to measure regressions early, using stage-level histograms and system-level tracing that exposes jitter sources you won’t see in application profiles.

Let’s get started.

OpenClaw Masterclass Workshop

Learn how to turn AI into a safe and cost-effective operational backbone with live case studies from experts who are running their business

Register

Before we dive in…

On the morning of August 1, 2012, Knight Capital Group opened for trading as one of the largest market makers in the US, handling roughly 10% of all equity trading volume on American exchanges. By 10:15 AM, 45 minutes after the market opened, the firm had lost $440 million and was effectively finished as an independent company.

The cause was a software deployment error. A new version of Knight’s order routing system had been pushed to seven of its eight production servers. The eighth server, missed during deployment, continued running a dormant piece of code that had not been used since 2003. When the market opened, that server began firing child orders into the market at full speed to fill just 212 customer orders, and kept going. The system sent more than 4 million orders before anyone intervened. Ninety-seven automated alert emails had been generated referencing the error before the market even opened. And nobody even acted on them. (SEC enforcement order, October 2013)

Knight Capital did not fail because its engineers were careless or inexperienced. It failed because the engineering discipline required to run software at trading-system speeds was not matched to the speed at which the system could cause damage. That discipline covers deterministic deployment, observable behavior, and a tested kill path. In a system capable of sending thousands of orders per second, the gap between “something is wrong” and “the damage is done” is measured in minutes, not days.

This is what Shreyans Doshi addresses in his article.

🧠Expert Insight

From NIC to P99: Engineering Low-Latency C++ Trading Systems in 2026

by Shreyans Doshi

Trading systems have been evolving for decades, moving from manual execution based on charts and indicators to automated strategies that fire within tens of nanoseconds using FPGA and ASIC hardware. Both approaches still coexist today, but most advanced and systematic trading has moved into the sub-100 microsecond range. In this article I want to focus on the hardware and software aspects you need to understand in order to build a trading system with p95 wire-to-wire latency under 50 microseconds.

Traditional technology stacks like Java, JavaScript, TypeScript, and C# offer a lot of flexibility and scalability for large-scale applications, but their inherently nondeterministic nature makes it very difficult to get below a p50 latency of 500 microseconds. While there are ways to optimize garbage collection and other Java components for latency, most standard Java libraries do not focus on low-level concepts like aligning data to cache lines or reducing cache line misses. C and C++ applications optimize for a specific platform at the cost of portability, but that trade-off is exactly what gives you access to manual memory management and the ability to avoid multithreading pitfalls like false sharing. When the goal is to reduce latency, you need lower-level languages like C, C++, or Rust on critical paths. I will not cover pure hardware implementations like FPGAs or ASICs here since they are not required for our 50-microsecond latency target.

Important definitions

Before diving into the architecture and hardware optimizations, it will be helpful to establish exactly how performance is measured in a low-latency environment.

Wire-to-wire latency

Wire-to-wire latency is measured from the moment a market data packet is received on the NIC from the exchange to the moment an order is sent back to the exchange on the same NIC. You compute it as order send timestamp minus market data receive timestamp. The most precise measurements require NIC cards with hardware timestamping enabled, so the NIC stamps every inbound and outbound packet with a hardware timestamp that you can later use to analyze the latency graph. Exanic cards provide APIs for retrieving these hardware timestamps.

Assumptions about the system

To hit the latency target of under 50 microseconds, you need to optimize both hardware and server setup alongside your software, and the following assumptions underpin everything in this article.

• Colocated host: Most stock exchanges in traditional finance provide colocation services where your servers sit physically adjacent to the exchange’s servers, removing extra network jitter from the equation.

• UDP market data access: Most stock exchanges distribute market data via UDP multicast to all subscribed parties, and this is the fastest way to consume and react to it.

• Kernel bypass NIC: The system runs on NIC cards enabled for kernel bypass and tuned for low-latency operation.

• Single venue: These systems are single-venue, meaning at any point they process market data to generate orders for one stock exchange only. For multi-venue setups, the assumption is one colocated host per venue with no shared critical-path processing.

Workload model

Most trading systems follow this model of data processing in abstract terms.

Every strategy engine takes market data from the exchange as its primary input and produces orders sent to the exchange as its primary output, moving through multiple stages as described below. Exchange responses are a secondary input to the engine but are not on the hot path for latency calculations.

Market data ingestion

The first layer is responsible for ingesting the market data (raw or normalized) from the network card. This layer routinely processes terabytes of data per day and can exceed a petabyte on busy days at venues like CME or Eurex. On an average day at CME you can expect 20 to 30 million trades and 100 to 150 million open interest events in a single segment. Processing a full L1 feed for one segment means receiving traffic in the range of a few terabytes to hundreds of terabytes per day, so both software and hardware need to be extremely optimized.

Filtering and checks

Filtering and checks come next, cleaning up incoming data and validating it before it reaches downstream stages. This layer handles packet drops, invalid market data, trading phase transitions like moving from an active session to a halted one, and anomalous price changes that might indicate a software bug. When something goes wrong at this stage, the right response is often to instruct strategies to stop rather than pass bad data forward.

Pricing engine

The pricing engine receives valid and normalized market data and can range from a simple no-op or currency conversion all the way to a full ML-powered model. Lightweight pricing fits comfortably on the critical path. Complex ML models that require large amounts of memory and processing do not, because they would blow the latency budget. In practice, ML-powered engines in low-latency applications split into two processes: time-consuming training and inference model preparation happen off the critical path, and a lightweight inference step runs on the critical path using small linearized models fed configuration parameters from the heavier ML output. This keeps the heavy lifting off the hot path while still giving the strategy access to model-driven signals. Colocation also constrains available compute, so you want to reserve it for work that directly impacts critical path performance.

Strategy evaluation

Strategy evaluation compares transformed or theoretical prices against real prices and fires an order when it identifies an opportunity. Strategies generally fall into three categories.

• Maker: Maker strategies keep quotes on both the buy and sell sides to provide liquidity and take advantage of the bid-ask spread. Exchanges often provide incentives to HFT firms that add liquidity this way.

• Taker: Taker strategies identify mispricing and fire IOC or FOK orders for immediate execution.

• Hedging: Hedging strategies reduce risk or book profit by executing the opposite side of an existing position.

Order aggregation and routing (OMS)

Order aggregation and routing (OMS) can be combined with strategy evaluation in simple setups, but should be separated as complexity grows. It serves three purposes: aggregating orders from multiple strategies to reduce the number of orders sent to the exchange, managing exchange responses and passing normalized results back to the strategy, and handling the routing logic that grows more complex as your strategy count increases.

Latency map: where the latency lies

Now that the high-level components are clear, it is worth mapping out where the latency actually sits so you can allocate your budget with precision. Beyond the business logic itself, your latency profile depends on CPU model and generation, NIC type and model, operating system, software architecture, and business logic complexity.

Beyond the business logic stages, several factors contribute latency that stays invisible if you only look at application code. These factors come from hardware, OS and kernel processing, network behavior, and I/O, and they play an outsized role in ultra-low-latency systems. Even if they occur infrequently, they introduce jitter and latency spikes that directly hurt your p99. Although many of these latency sources depend on a multitude of factors, you can work with approximate cost ranges that provide a useful benchmark.

Continue reading

🛠️Tool of the Week

bpftrace — open-source high-level tracing language for Linux systems, built on eBPF

Highlights:

• Instant latency visibility where it leaks: Attach probes to kernel/user-space events (kprobes/tracepoints/uprobes) to pinpoint tail-latency sources—scheduler delays, IRQ/softirq behavior, and “surprise” syscalls—without changing your C++ code.

• Built-in distribution tooling (histograms by default): First-class aggregations like hist() and quantize() make it trivial to generate p50/p95/p99-style views (not just averages), which is exactly what you need for jitter and p99 debugging.

• Ready-to-run scripts for common low-latency failure modes: The bundled tools/ library covers practical staples like syscall counts, run-queue latency, and retransmits—fast to deploy during an investigation, easy to keep as regression checks.

Learn more about bpftrace

📎 Tech Briefs

• PostgreSQL 18.3 / 17.9 / 16.13 / 15.17 / 14.22 released: PostgreSQL shipped an out-of-cycle patch set to all supported branches to fix regressions (including fixes touching standby behavior, substring() encoding edge cases related to a CVE fix, pg_trgm stability, and restoring json*_strip_nulls() immutability for index use).

• GitHub Actions: upload/download artifacts without forced ZIP: GitHub Actions added support to upload and download non-zipped artifacts (via actions/upload-artifact@v7 with archive: false and actions/download-artifact@v8), reducing friction for single-file downloads, in-browser viewing, and “double-zip” workflows.

• DPDK Governing Board minutes: quality-first 26.03 cycle + AI-assisted review gate): DPDK’s Feb 17 governing-board update highlights a code-hardening 26.03 cycle (roughly “half the commits” focused on fixes/backports) and describes introducing AI-assisted code reviews hosted by the UNH lab as an additional quality gate after standard CI passes.

• Node.js 25.8.0 (Current) released: Node.js 25.8.0 landed with notable changes including SQLite DatabaseSync additions, expanded diagnostics channel support, a new --permission-audit flag, and dependency updates (including an npm upgrade).

• Dependabot alert assignees GA: GitHub made Dependabot alert assignees generally available, enabling explicit ownership of dependency vulnerability alerts (UI + REST API + webhooks) and visibility across repo/org/enterprise alert views.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Leave a comment

Give AI agents the knowledge of your most experienced engineers. Unblocked builds context from your team’s code, PR history, conversations, documentation, planning tools, and runtime signals to surface what matters for the task at hand – while resolving conflicts and respecting permissions. Ship faster with AI outputs that reflect how your system actually works.

See how it works

✍️From the editor’s desk,

Dear reader,

It has been a privilege to serve you as we built Deep Engineering over the past year. This is my final issue, and next month the newsletter will continue under new editorial leadership. I’m glad to introduce as the new Editor-in-Chief of Deep Engineering.

Deep Engineering remains committed to serving the software community with timely, expert-led insights. Thank you for reading and for being part of this journey with me.

Welcome to the 36th issue of Deep Engineering!

Cloudflare’s Q4 2025 DDoS Threat Report described a record 31.4 Tbps attack and hyper-volumetric HTTP floods exceeding 200 million requests per second—a useful reminder that production systems run in a hostile environment. Today’s issue is grounded in my conversation with Archit Agarwal, Principal Member of Technical Staff at Oracle working on ultra-low-latency authorization services in Go, and the author of The Weekly Golang Journal. As Agarwal says, we must assume the system will fail and the system will be attacked, then work backward into controls that keep the blast radius small.

In this issue, we explore what makes a system design hold up in production once cost, failure, and security constraints become non-negotiable? Agarwal’s guidance is consistent across topics: delay microservices until “splitting signals” are visible, treat performance per dollar as a design requirement, cap autoscaling to prevent runaway spend during attacks, and keep dependency risk contained through abstraction, version discipline, and unified observability. Agarwal also offers battle-tested tips on cracking the system design interview.

You can watch the complete interview and read the transcript here, or read on for distilled insights.

Build an Internal Developer Platform from scratch (2-day live workshop)

Most IDPs fail when teams build tools, not products. This 2-day live, hands-on workshop shows how to design and build a production-ready IDP with platform-as-a-product patterns, adoption, governance, and reliability. Sat, Apr 25, 2026 — 11:00 AM ET.

Reserve your seat →

System Design That Holds Up in Production with Archit Agarwal

A system design diagram is only useful if it reflects the constraints the system will live under. Archit Agarwal’s answer to most design questions is the same: make the trade-offs explicit, and choose them deliberately.

When microservices are worth it

It is now generally established best practice to begin with a monolith when requirements are still changing and the domain is still unfamiliar. Agarwal recommends keeping module boundaries clean so that separation is possible later, but he delays distribution until there is clear evidence it will help.

He looks for “splitting signals”:

• Deployment friction shows up: Releases get larger, slower, and harder to roll back.

• The blast radius grows: A defect in one area increasingly breaks unrelated user paths.

• Independent scaling becomes necessary: One workload needs more capacity than others during predictable peaks.

• Technology needs diverge: A subsystem truly benefits from different runtimes, storage, or processing tools.

Microservices add team autonomy, but they also add coordination overhead and network latency. According to Agarwal, splitting makes sense when the monolith’s pain is observable, repeatable, and clearly outweighs the operational tax of moving to services.

Performance per dollar as a design requirement

Cost is part of the architecture. Agarwal evaluates choices as “performance per dollar,” so teams weigh latency gains against the full cost of running and operating the system.

A cost-aware loop can:

• Make spend visible through dashboards, so teams connect architecture to real numbers.

• Put cost into design reviews alongside latency, availability, and security trade-offs.

• Build for horizontal scaling, then scale only when demand and reliability targets prove the need.

• Manage data by lifecycle. Keep frequently accessed data in hot tiers and move cold data to cheaper storage.

Design for a hostile environment

It is also important to design around two expectations: systems fail, and systems get attacked. Planning for both keeps failure domains small and recovery predictable.

Recent threat telemetry supports the posture. Cloudflare’s Q4 2025 DDoS report describes a record 31.4 Tbps attack and a subsequent campaign with hyper-volumetric HTTP floods exceeding 200 million requests per second. The same report says DDoS attacks surged 121% in 2025, averaging 5,376 mitigations per hour on Cloudflare’s network.

Agarwal recommends a practical resilience toolkit that ties these risks to concrete controls:

• Add layered defenses at the edge before the application spends CPU.

• Rate-limit meaningfully by user, token, IP, and geography, and tune limits using real traffic.

• Cap autoscaling so an attack cannot trigger runaway capacity and spend.

• Separate failure domains with multi–availability zone deployments, and reserve multi-region for the paths that must stay up.

• Keep observability complete, because incident response depends on fast visibility.

Supply chain exposure belongs in the same threat model. Sonatype reported 394,877 new malicious open-source packages in Q4 2025 alone, a 476% jump compared to the prior three quarters combined, with 99.8% of the malware it saw in that quarter originating from npm. ReversingLabs’ 2026 software supply chain report also points to npm as the primary hotspot, reporting 10,819 malicious packages detected there in 2025 (nearly 90% of its detections across OSS platforms) and describing a first-of-its-kind self-replicating registry-native worm dubbed Shai-hulud that compromised close to 1,000 npm packages across two campaigns.

Design implications follow naturally: pin and review dependency updates, validate new versions before promotion, avoid automatic upgrades on production-critical paths, and enforce least privilege and deny-by-default authorization so that a compromised component has limited reach.

Depend on services without letting dependencies own you

Modern systems rely on external services and SaaS APIs. Agarwal recommends keeping those dependencies from becoming single points of failure.

His approach is pragmatic:

• Put an abstraction layer in front of major external services, so the application talks to an interface rather than a vendor-specific client library.

• Prefer portable standards where they fit, such as containers and unified telemetry.

• Replicate only what is mission-critical across regions or providers, and keep observability in one place.

Ultra-low latency without sacrificing the whole codebase

Agarwal recommends treating latency like a budget: every hop and allocation spends from it. Ultra-low-latency work, in his view, is mostly subtraction.

Replicable strategies:

• Put compute closer to users when geography is on the critical path.

• Remove network hops from hot paths, and keep the performance-critical code lean.

• Reuse connections and choose efficient service-to-service protocols when calls are unavoidable.

• Match hardware and runtime to the goal, because software choices cannot fully offset weak infrastructure.

He also draws a boundary that protects maintainability: only a small part of most systems needs extreme optimization. Keep the rest readable and easy to change.

How to communicate system design under pressure

Agarwal’s interview advice doubles as a template for real design discussions: keep the design aligned with requirements, and keep your reasoning easy to follow.

His recommended structure for cracking the system design interview:

• Align on functional requirements first.

• Capture nonfunctional requirements next, because they drive architecture.

• Present a high-level design, then zoom in one component at a time.

• State trade-offs as you choose, including what you gain and what you give up.

When constraints change midstream, he recommends restating the change, identifying the affected parts, and updating only what needs updating. That approach shows control, adaptability, and a clear “commit history” of decisions.

🧠Expert Insight

🛠️Tool of the Week

Envoy Proxy: a high-performance L7 proxy that standardizes service-to-service and edge traffic management (routing, resilience, security, and telemetry) across distributed systems.

Highlights:

• Advanced traffic routing & load balancing: dynamic routing rules, weighted traffic splitting, and multiple load-balancing policies.

• Resilience controls built in: timeouts, retries, circuit breaking, and outlier detection to contain failures.

• Security + observability hooks: mTLS/TLS termination, external authorization integration, and first-class metrics/logging/tracing support.

Learn more about Envoy Proxy

📎Tech Briefs

• Cloudflare outage (BYOIP route withdrawal): Explains how a change in BYOIP prefix management unintentionally triggered BGP route withdrawals for some customers, causing timeouts and connectivity failures, and documents the mitigations and follow-up controls.

• Post-quantum encryption across Cloudflare One (SASE): Cloudflare has rolled out standards-based post-quantum hybrid ML-KEM across Zero Trust and WAN on-ramps/off-ramps (including TLS/MASQUE/IPsec paths) without requiring specialized hardware changes.

• Private incident + maintenance notifications (SSO & Slack) — Fastly:
  Fastly has introduced private, security-sensitive incident and maintenance notifications with integrations like SSO and Slack to improve operational transparency for customers without exposing details publicly.

• apt.postgresql.org operational updates: The PostgreSQL project’s apt repository announces automatic apt-retrievable changelogs, colocated build logs, and repository changes as Ubuntu 26.04 work begins and Ubuntu 25.04 reaches EOL.

• PostgreSQL Anonymizer 3.0: Dalibo’s PostgreSQL Anonymizer 3.0 adds parallel static masking via background workers and JSON import/export for masking rules, alongside security fixes and breaking-change notes.

That is all from me today. Thank you for reading this issue of Deep Engineering.

We will be back next week with more expert-led content.

Signing off one last time,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

(Don’t forget to stay awesome!)

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Leave a comment

Move MCP from demos to production. In this 5-hour online workshop you’ll cover OAuth 2.1 auth pitfalls, OpenTelemetry observability, scaling persistent connections, and managing context growth/cost. Led by Saxo Bank Principal Dev Peder Holdgaard Pedersen. When: Mar 29, 10:30am–3:30pm ET / 7:30am–12:30pm PT.

Get tickets on Eventbrite

✍️From the editor’s desk,

Welcome to the 35th issue of Deep Engineering!

This is the second installment in Deep Engineering’s ongoing series adapted from chapter 2 of The IT Strategy Playbook by Scott Millett (Packt; publishing May 2026). Millett is a long-time IT leader and architect who has served in CIO/CTO roles and led large-scale transformation work across multiple sectors (as you saw in Part 1’s “100 Day Plan” boardroom story).

Today’s adaptation focuses on a practical problem: how technical realities become executive decisions. Millett lays out a repeatable approach for communicating strategy—clarifying the single objective, framing the narrative with SCQA (Situation–Complication–Question–Answer), and supporting the recommendation using the Pyramid Principle—so boards and executives can evaluate trade-offs and commit to a direction.

This is well aligned with what US technology leaders are navigating this week: Gartner’s Feb 17 Q&A piece argues that fragmented AI regulation is accelerating spend on AI governance platforms as organizations try to manage compliance and operational risk. At the same time, a multi-industry coalition urged the Commerce Department to preserve the core of the NIST AI Risk Management Framework, a signal that many enterprises are already using it as a shared language for oversight and internal controls. And CISA’s Feb 13 announcement of CIRCIA rulemaking town halls reinforces that incident reporting expectations—and the need to communicate risk and response clearly—remain active, moving targets.

Let’s get started.

Safely Refactor Large Production Codebases with AI

Safely refactor large production codebases with AI. Learn a workflow that pairs Claude Code with ast-grep’s AST-based rewrites for structural guarantees—migrations, guardrails, and best-practice enforcement. Ticket includes Mastering ast-grep + prompt templates. Mar 14, 10–11:30am ET / 7–8:30am PT.

Get tickets

🧠Expert Insight:

The IT Strategy Playbook (Part 2): Closing the Story Gap by Scott Millett

“The most powerful person in the world is the storyteller.” — Steve Jobs

In the early 2000s, Nokia dominated mobile phones. Its strategy, built on exceptional hardware engineering and global supply-chain execution, worked flawlessly until 2007. The company didn’t fall because its engineers lacked insight. It fell because the organization couldn’t convert critical technical knowledge into strategic intent. The signals were present; the translation mechanisms were not.

Nokia’s software teams had long recognized that Symbian’s architecture was reaching its limits. It was fragmented, costly to evolve, and increasingly misaligned with the emerging touch-driven experience. Alternative platforms such as Maemo and later MeeGo were being explored, and the technical risks were well understood within engineering. But these concerns were filtered through a hardware-first culture that interpreted platform limitations as minor technical defects, not as threats to the business model. The communication path from “this architecture will constrain what the business can do” to “this requires a strategic shift” simply wasn’t there.

At the same time, senior leadership continued to optimize short-term device metrics. They still viewed the competitive landscape as a handset battle rather than an ecosystem battle, and the implications of Apple’s and Google’s platform strategies never translated into a unified architectural or strategic response. Nokia’s organizational structure created a communication gradient: technical truths lost meaning as they travelled upward, and strategic signals failed to reorient investment, priorities, or direction.

Within five years, the world’s leading mobile manufacturer was forced to exit the handset business, not because it lacked capability, but because it lacked the ability to translate insight into coherent action. This chapter focuses on the story gap: how to communicate strategic needs clearly, how to ensure signals travel across the organization intact, and how to avoid the silent failures that undermine strategy long before the market ever does.

In this chapter, you’ll learn how to:

• Communicate strategic intent with clarity, ensuring technical insight translates into decisions rather than confusion

• Structure a coherent strategic narrative using proven frameworks to move from context to conclusion

• Develop presentation material that supports decision-making, not just information sharing

• Deliver your message effectively, guiding senior stakeholders from understanding to commitment

You will use these skills to close the story gap: translating strategy, architectural intent, and plans into a narrative that enables decisions. Not more slides, but a clear line from context to conclusion, so leaders understand the trade-offs, commit to a direction, and act.

How to Be An Effective Communicator

Delivering a technology strategy is not simply an act of presenting information; it is an act of leadership and influence. The best strategy in the world will fail if the communicator lacks clarity of purpose or the ability to authentically connect with the audience.

Clarify your objective

Before you open a slide deck or write a single email, you must answer one question with absolute, non-negotiable clarity: What is the one thing I need the audience to believe or do when I stop speaking or reading? This is the foundation of strategic communication. Without a single, clearly defined objective, your message will lack focus, and your audience will leave confused about the desired next step. Your objective should never be passive, such as “to inform the Board” or “to update the team.” It must be an action verb tied to a measurable outcome. Consider the examples in table 2-1 where we take a passive goal and turn it into an active objective.

When you define your objective down to the ‘one thing,’ you achieve two crucial results:

1. Elimination of Noise: Every slide, chart, and word you use must directly serve that single objective. If a piece of data doesn’t help you achieve the objective, it belongs in the appendix, or nowhere at all.

2. Forcing the Answer: By knowing the desired action, you structure your entire presentation to lead logically and inevitably to that conclusion. Your presentation becomes a persuasive roadmap, not a rambling document review.

Master the three modes of Persuasion

As an IT leader, you must consciously deploy all three modes to secure buy-in. Relying solely on data (Logos) is rarely enough; you must also establish your credibility (Ethos) and connect your strategy to the organization’s goals and fears (Pathos). The following subsections detail how to weave each of these essential appeals into your strategic narrative to maximize influence and drive action.

Continue reading...

🔍In case you missed it…

🛠️Tool of the Week

Mermaid — Open-source “diagrams as code” for flowcharts, sequence diagrams, and architecture visuals (rendered from plain text)

Mermaid (mermaid-js/mermaid) is a text-based diagramming tool that turns Markdown-like syntax into diagrams you can embed in docs, RFCs, and strategy decks—useful when you want visuals that stay versionable and easy to review.

More about Mermaid

📎Tech Briefs

• CIRCIA town halls / reporting requirements (US critical infrastructure CIRCIA rulemaking: DHS/CISA scheduled sector-specific virtual town halls (plus two general sessions) to gather additional stakeholder input on refining the scope and burden of the proposed cyber incident reporting rule.

• GitHub Agentic Workflows (technical preview): GitHub introduced Markdown-authored “agentic” automations that compile into standard Actions workflows, aiming to make repo automation more natural-language and security-constrained by default.

• Attack-surface and app-sec bundling: Cloudflare and Mastercard announced a partnership to combine attack surface monitoring/risk scoring with Cloudflare’s application security controls for faster remediation across internet-facing assets.

• CISA KEV update: CISA added four vulnerabilities to its Known Exploited Vulnerabilities Catalog (evidence of active exploitation), reinforcing KEV-driven prioritization for vulnerability management.

• VictoriaMetrics LTS release: VictoriaMetrics shipped LTS release v1.122.15 (released 2026-02-13) with bugfixes intended for production observability deployments.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Code review is stuck. It's slow (manual), noisy (tools), or misses things. Qodo 2.0 is different. With highest precision & recall, it finds critical issues other review tools miss, while limiting noise. Learning from your codebase & PR history, Qodo gives fixes developers actually use.

Try Qodo for Free

✍️From the editor’s desk,

Welcome to the 34th issue of Deep Engineering!

On February 10, 2026, the OpenJDK Quality Group warned that JDK 27 will remove ThreadPoolExecutor.finalize(), a source-incompatible change that can surface as compile errors in code that calls or overrides it. It is a useful reminder that JVM modernization happens under load while production systems keep running. Today’s issue looks at what “practical modernization” means after the Kotlin hype cycle, grounded in my conversation with José Dimas Luján Castillo and Ron Veen, co-authors of Kotlin for Java Developers.

Luján Castillo brings a mobile-first view shaped by 15 years building Android, iOS, and Flutter applications and leading teams globally, including work across 500+ mobile apps, plus experience as a teacher and author. Veen brings an enterprise JVM lens from 20+ years in the Java ecosystem, spanning mainframes to microservices, with certifications including Oracle Certified Java Programmer and Sun Business Component Developer, and hands-on work around Jakarta EE migrations and modern concurrency.

You can watch the complete interview and read the transcript here, or read on for distilled insights.

Today’s issue also includes the complete Chapter 15: Coroutines from their book in addition to our regular Tool of the Week pick and Tech Briefs.

Let’s get started.

Sponsored:

Webinar | How to Build Faster with AI Agents: Learn how full‑stack developers boost productivity by 50% with AI agents that automate layout, styling, and component generation through RAG and LLM pipelines. See how orchestration and spec‑driven workflows keep you in control of quality and consistency. Save your seat!

Kotlin and Java after the Hype Cycle with José Dimas Luján Castillo and Ron Veen

A language switch succeeds or fails in the space between code and people. Teams can adopt Kotlin fast and still ship Kotlin that behaves like Java. The most useful guidance is keeping modernization practical while systems keep running.

Kotlin’s first gains: safety and continuity

Luján Castillo says Kotlin’s benefits were easy to see on Android because “Java was too verbose.” He names one feature that changed everyday work immediately: “null safety. It’s automatic.” He also describes the adoption detail that matters most in mature codebases: Kotlin’s “interoperability with Java.” “Maybe I can do the next steps in my applications with Kotlin, but I don’t need to fight with the legacy code even if it’s in another language,” he says, and he reminds teams that “the main problems are still there even if you change the language.”

Veen’s enterprise view focuses on maintenance and defects. “There will be less code—and less code is good (because it means) less code to maintain,” he says, and he expects Kotlin to help because it “tends to be less error-prone,” with null handling as a steady source of bugs.

The migration trap: translating Java line by line

Both warn that Kotlin adoption can stall at Java-flavored Kotlin. Veen calls it a natural reflex: teams start Kotlin while still thinking in the patterns they already know, because there is still a project to ship. “It’s not a drop-in replacement,” he says. “You really have to be willing to learn.”

Luján Castillo describes the common first step: “99% of developers start by just translating the code.” Translation helps people learn syntax, but it does not deliver the major improvements teams expect. “If you are translating each line, you will see you don’t have less code—you have the same code in a different way,” he says. He looks for a shift where developers start asking whether Kotlin makes a line unnecessary.

Veen gives a concrete way to test that shift. He rewrote an implementation and reduced it “from 250” lines “down to under 100,” not because shorter is always better, but because it forced him to ask how to “leverage Kotlin’s native way of doing things.” Luján Castillo describes “immutability first” as a sign that developers have started to think in Kotlin’s defaults rather than Java’s habits.

Concurrency decisions: coroutines, virtual threads, and structure

Luján Castillo frames concurrency as choosing a model you can work with under pressure. For coroutines, he highlights the structure: “You have cancellation, you have propagation, you have control of the whole cycle.” For virtual threads, he emphasizes familiarity. They are “very traditional,” and he sees them as attractive when a system already has “a very complex structure with layers.” He avoids declaring a winner: “It’s not a model about who is better.”

Veen names the shared goal behind both approaches: async work that can “appear sequentially in the code,” so developers can reason about flow and avoid “callback hell.” OpenJDK’s current structured concurrency work follows that reasoning-first intent. JEP 525 describes subtasks whose lifetimes are “confined to the syntactic block” of their parent task so they can be managed “as a unit,” and it calls structured concurrency “a great match for virtual threads.”

Modernization without regret: boundaries first, then change

Luján Castillo argues that “microservices, in my experience, is not the goal, OK? It’s the consequence for that.” Before a team breaks apart a monolith, he asks, “Are we prepared for paying the real cost to use microservices—not just the money?” and “are we prepared to use this architecture or not?” He also rejects the idea that monoliths are obsolete: “you can use monoliths without problems for huge systems.”

Veen explains where the cost shows up. Microservices require “a lot more overhead” in monitoring, and debugging grows harder when incidents span “dozens or hundreds of services.” His default is a “modular monolith,” where modules have clear boundaries and must use a “predefined API” to interact. He then watches for signals that justify extraction, such as a module that forces frequent redeployments or has distinct scaling needs.

Both speakers treat adoption as a leadership problem. Veen recommends bringing skeptics into the work early and using mixed teams and code reviews to build confidence. Luján Castillo recommends “small limits” and warns leaders away from shallow metrics: “Don’t try to read productivity… as: ‘If you have more lines of code or less lines of code.’”

JetBrains lists Kotlin 2.3.10 as released on February 5, 2026, which supports the kind of incremental upgrades both speakers describe

🧠Expert Insight:

1. The complete Deep Engineering interview with José Dimas Luján Castillo and Ron Veen

2. The complete Chapter 15: Coroutines from Kotlin for Java Developers by José Dimas Luján Castillo and Ron Veen (Packt, 2025)

🛠️Tool of the Week

OpenTelemetry Java Agent — Open-source, zero-code auto-instrumentation for JVM services (traces, metrics, and more)

The OpenTelemetry Java Agent (from open-telemetry/opentelemetry-java-instrumentation) is a Java agent JAR you attach at startup to capture telemetry from popular Java libraries and frameworks without changing application code.

Highlights:

• Zero-code startup integration: Attach the agent with the JVM -javaagent flag to “gather telemetry data from a Java application without code changes.”

• Collector-first by default, configurable when needed: The agent’s default path uses OTLP to send telemetry to an OpenTelemetry Collector endpoint (and you can configure it via system properties or environment variables).

• Open source, production-oriented ecosystem, recently updated

More about OpenTelemetry Java Agent

📎Tech Briefs

• Kotlin 2.3.10 Released: Kotlin 2.3.10 is a new Kotlin/JVM release, giving teams a concrete upgrade point for Kotlin codebases that need to stay current with the Java ecosystem.

• OpenJDK JDK 26 Release-Candidate Builds (Build 34): DK 26’s RC builds are now available, which is useful for teams that want to validate JVM and runtime behavior changes (including concurrency-related changes) against real workloads before GA.

• Spring Boot 4.0 Migration Guide: Spring Boot’s 4.0 migration guide captures the practical baseline shifts (Java 17+, Spring Framework 7.x, Jakarta EE 11, Kotlin version requirements) that enterprise JVM teams need to account for in upgrades.

• Quality Outreach Heads-up (JDK 27): Removal of ThreadPoolExecutor.finalize(): This heads-up flags a source-incompatible change where removing ThreadPoolExecutor.finalize() can trigger compile errors in code that directly calls or overrides it.

• Optimizing Java Class Metadata in Project Valhalla: A Project Valhalla update digs into how the JVM is evolving class metadata to support value classes and flattening/inlining, which matters for performance-sensitive JVM designs over the long term.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Code review is stuck. It's slow (manual), noisy (tools), or misses things. Qodo 2.0 is different. With highest precision & recall, it finds critical issues other review tools miss, while limiting noise. Learning from your codebase & PR history, Qodo gives fixes developers actually use.

Try Qodo for Free

Qodo CEO Itamar Friedman in a conversation with Deep Engineering said that SWE-bench mainly measures code generation rather than maintainability or security—and that code can pass tests while still introducing serious issues. Read the full piece:

✍️From the editor’s desk,

Welcome to the 33rd issue of Deep Engineering!

On Jan 26, 2026, Oren Yomtov (Koi Security) wrote about finding six “PackageGate” bypasses across JavaScript package managers—bypasses that can undermine the two defenses widely repeated after Shai-Hulud: disable lifecycle scripts and commit lockfiles with integrity checks.

The lockfile is now part of the boundary you rely on, and the boundary can fail at the edges. This is the focus for today’s issue. We also bring you Ivo Balbaert’s (lecturer in web programming and databases at CVO Antwerpen and Packt author), fifth article in our ongoing Building with Mojo series. Today’s Mojo deep dive covers how Pixi structures environments and uses pixi.lock, and how modules and packages get imported and shipped as artifacts.

Let’s get started.

Your Lockfile Is Now Part of Your Threat Model

Late last year, the Shai-Hulud 2.0 incident underscored why lockfiles have become security-critical. This aggressive npm supply chain attack compromised over 700 packages and exposed credentials from some 25,000 repositories. Attackers hijacked maintainer accounts of popular projects (e.g. Postman, Zapier) and pushed malicious updates. In each infected package, a rogue preinstall script ran early, before tests or security scans, to surreptitiously install tools and exfiltrate secrets. Developer machines, CI pipelines, and cloud workloads that pulled these packages had credentials stolen and uploaded to attacker-controlled repositories. This worm-like malware spread rapidly through trusted package workflows – a masterclass in how our reliance on open-source packages can be turned against us. Traditional network defenses offered little protection when the attack piggybacked on legitimate package updates.

The immediate response across the JavaScript ecosystem was to double down on two measures: disable implicit code execution and pin every dependency. Security guides and corporate policies began advising developers to run package managers with “ignore scripts” flags (to block auto-running install scripts) and to check in lockfiles for all projects. The logic behind this was that If no untrusted postinstall code runs, and if every dependency version is locked, attackers can’t sneak in. In theory, “if malicious code can’t run on install, and your dependency tree is pinned, you’re covered”.

Holes in the Safety Net

However, the reality proved far more complicated. In January this year, Oren Yomtov, security researcher with Koi Security revealed “PackageGate,” a set of six zero-day vulnerabilities spanning multiple package managers (npm, pnpm, Bun, and others) that blew holes through these defenses. One flaw in pnpm (CVE-2025-69263) showed how a lockfile could be rendered toothless: pnpm was recording certain git or HTTP-based dependencies without any integrity hash in its lockfile, so a remote server could insidiously serve different code on each install – completely undermining the “locked” state. Another pnpm bug (CVE-2025-69264) let malicious git-hosted packages execute code during install despite script blocking features. Attackers found ways to bypass lockfile integrity and re-enable install scripts, enabling remote code execution even in projects that followed best practices.

The Shai-Hulud attackers (and copycats like “PhantomRaven”) were already experimenting with such techniques, such as sneaking malicious directives via nested git dependencies. It became clear that the lockfile itself is part of the threat model: any weakness in how it’s generated or enforced can be targeted. According to Javed Hasan, co-founder and CEO of Lineaje, this underscores “how brittle many existing trust assumptions remain” in our software pipelines. Even the largest package registry was not immune to complacency – npm’s maintainers initially declined to patch one of the reported exploits from Yomtov, deeming it “works as expected”. This response was scary: the very safety net (lockfiles and script blocking) installed after the worst supply-chain attack in npm’s history was proven to have holes, and not all providers rushed to fix them.

Treating lockfiles as security artifacts means ensuring they contain strong integrity guarantees and that your build process honors them. Modern lockfiles (whether pixi.lock, package-lock.json, or Cargo.lock) don’t just pin versions; they often embed a cryptographic hash for each package download. During installation, the package manager is supposed to verify the content against those hashes and fail if there’s a mismatch. This is how a lockfile acts as a gatekeeper: if an attacker compromises a registry or MITMs a connection, they can’t slip altered code past the hash check. However, the pnpm incident showed that if those hashes are omitted or not verified in all cases, the lockfile “provides no protection” at all. Similarly, if your tool allows alternate install flows (like git URLs or tarball links) without equal scrutiny, attackers will find that side door. Part of making lockfiles a security boundary is keeping your tooling up-to-date – in this case, upgrading pnpm promptly to get the patch that reinstates hash enforcement.

Hardening the Supply Chain

The wave of supply-chain compromises has prompted ecosystem-wide changes. One major vector in Shai-Hulud’s spread was stolen npm credentials – which allowed attackers to publish malicious versions of packages they didn’t own. In response, registry providers are tightening authentication and publishing workflows. For example, as of November 2025 GitHub (which operates npm) disabled classic long-lived npm tokens and required a move to fine-grained access tokens with 2FA enforcement and expiration. By mid-November, all legacy npm tokens were revoked and replaced by short-lived 2-hour session tokens for publishing. This means even if an attacker pilfers a token from a CI log or a maintainer’s machine, its window of use is dramatically narrowed. GitHub’s guidance to maintainers is to adopt “trusted publishing” – tying package publishing to your CI’s identity via OpenID Connect – and to mandate 2FA for any package modifications. This approach binds the act of releasing a new package to a verified workflow (for instance, a signed-in GitHub Actions job), removing the human weakness of stolen passwords or tokens from the equation. It’s a direct policy response to attacks: make the act of injecting a bad update much harder.

Other ecosystem measures complement the lockfile’s role in security. Services like GitHub’s Dependabot and Microsoft Defender for Cloud now alert on suspicious package behavior or known-compromised dependencies, encouraging teams to “freeze” or replace those dependencies quickly. Two defenses became “standard advice” after the Shai-Hulud worm attacks:

1. disable lifecycle scripts that run automatically during installation

2. Commit lockfiles with integrity checks (such as package-lock.json or pnpm-lock.yaml) to version control.

Lockfiles record the exact dependency versions and integrity hashes, and on later installs the package manager checks downloaded packages against those hashes and fails if they don’t match—so a compromised package update should be blocked by the integrity mismatch. However, Yomtov argues that the two defenses remain worth following but do not fully hold if package-manager bypasses exist.

In Mojo’s Pixi workflow, as Ivo Balbaert demonstrates in today’s feature article, developers can use pixi install --frozen (or set PIXI_FROZEN=true) to ensure CI installs only what’s in the lockfile and fails if the lock isn’t up to date. This mirrors analogous flags like npm ci or pip install --require-hashes in other ecosystems. The overarching theme is consistency: the lockfile and the published packages must match, and any deviation is a potential intrusion. By enforcing this at multiple levels—developer machines, CI pipelines, and registry policies—we establish the lockfile as a firm security checkpoint rather than a passive snapshot.

If the code is the new perimeter in cloud-native apps, then the lockfile is quickly becoming the new gatekeeper in our build systems. It’s not a silver bullet; as we’ve seen, it can be bypassed if we get complacent. But when used in tandem with hardened tooling and policies, the lockfile is a powerful line of defense.

Best Practices

• Pin and verify every dependency: Commit and maintain your lockfiles (e.g. pixi.lock, package-lock.json). Use package managers’ integrity features so that each dependency’s content is verified against a hash. If your ecosystem supports it, require hashes for any external or git-based dependencies as well.

• Enforce frozen installs in CI: Configure CI/CD pipelines to install dependencies in locked mode (e.g. Pixi’s --frozen flag or npm ci). This ensures no new package versions or surprises slip in during automated builds. Any lockfile change should be deliberate and code-reviewed.

• Avoid unofficial sources and dynamic resolution: Prefer canonical registry packages over direct URLs or git references. If you must use a git or tarball dependency, pin it to a specific commit/tag and treat it with extra scrutiny. Unpinned artifacts can bypass lockfile guarantees.

• Harden publisher credentials: Enable 2FA on package registries and migrate to fine-grained, short-lived tokens or OIDC-based trusted publishing. Eliminate long-lived secrets from your workflows; use ephemeral credentials tied to CI jobs so that stolen credentials quickly become useless.

• Monitor and react quickly: Stay aware of security advisories for your package manager (e.g. pnpm CVEs) and apply updates that patch lockfile or installer vulnerabilities. Monitor dependency changes in your supply chain (via tools or manual review) and have an incident response plan (e.g. rapid credential rotation and dependency isolation) for suspected supply-chain compromises. Each lockfile update is a security event – treat it with the same caution you would a code change in your core application.

In Mojo’s Pixi workflow, as Ivo Balbaert shows in today’s feature on modules, packages, and Pixi, pixi.lock and pixi install --frozen offer a concrete example of how lockfiles act as a security boundary and under what conditions. Balbaert’s article also walks through Mojo’s module/package structure and import syntax, including re-exporting via __init__.mojo, and shows how to import packages from source or as compiled .mojopkg artifacts, plus how to pull in external Mojo community packages with Pixi.

🧠Expert Insight:

🛠️Tool of the Week

Sigstore Cosign — Open-source signing and verification for OCI container images (and other artifacts)

Cosign is a CLI and set of workflows for signing and verifying container images (and related supply-chain artifacts) using Sigstore’s model (including keyless signing via OIDC). It’s designed for teams that want cryptographic “who produced this?” and “has it changed?” checks that can be automated in CI/CD and enforced at deploy time.

Highlights:

• Keyless signing with workload identity: Cosign can sign artifacts using OIDC-issued identities (no long-lived private key to manage), so signatures are tied to who/what ran the build (e.g., a CI workflow).

• OCI-native signatures and attestations: It stores signatures (and can store SBOM/provenance attestations) alongside OCI artifacts in registries, so verification works wherever the image travels.

• Policy-friendly verification primitives: Cosign supports verifying by identity claims (issuer/subject), signatures, transparency log inclusion, and certificate chains—building blocks you can wire into admission controllers or deployment gates.

Learn more about Cosign

📎Tech Briefs

• GitHub eyes restrictions on pull requests to rein in AI-based code deluge on maintainers: GitHub is weighing tighter PR controls and filtering as maintainers report a surge of low-quality, AI-generated submissions overwhelming open-source review capacity.

• Notepad++ updater was compromised for 6 months in supply-chain attack: Attackers hijacked Notepad++ update infrastructure to selectively deliver trojanized installers to targeted users, illustrating how “trusted update paths” become the distribution channel.

• Xcode 26.3 adds support for Claude, Codex, and other agentic tools via MCP: Apple is integrating agentic coding tools directly into Xcode, including support for Model Context Protocol (MCP), pushing more automation into the IDE workflow.

• CVE-2026-23890: pnpm path traversal in bin linking :NVD describes a pnpm flaw where malicious packages can create executable shims/symlinks outside node_modules/.bin, turning installs into a filesystem write primitive.

• ‘PackageGate’ flaws open JavaScript ecosystem to supply chain attacks: Coverage of the PackageGate class of issues where package-manager behavior can bypass script-blocking / lockfile-based assumptions and enable execution through attacker-controlled dependencies.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

On February 3 at 11:00 AM PT, we’ll introduce Cyberhaven’s data lineage powered and unified DSPM and DLP platform. You’ll see how one AI-native solution can finally keep up with the way data really moves. Be among the first to see how a unified DSPM and DLP platform can change how your organization protects its most important data.

Register now

✍️From the editor’s desk,

Welcome to the 32nd issue of Deep Engineering!

Engineering teams today are being asked to operationalize systems whose behavior cannot be fully inferred from source code. Once a model sits inside a product workflow, the work shifts to measurement, monitoring, and change control: what the system does today, what it did yesterday, and what it will do after the next model update.

AWS’s Generative AI Lens (updated Nov. 19, 2025) puts operational requirements—traceability, monitoring, automation of lifecycle management—on the same footing as security and reliability for AI systems. The accompanying architecture note describes expanded material on responsible AI, data architecture, and agentic workflows, reflecting what teams are now deploying in practice.

If an organization cannot evaluate systems rigorously, it cannot claim to operate them reliably. OpenAI, in a Nov. 19, 2025 post, centers evaluation as an operational discipline for business deployments: setting measurable targets, tracking performance shifts, and making decisions about release readiness. Anthropic’s model reporting, updated Dec. 4, 2025, treats capabilities and safety evaluations as documentation that must be maintained alongside deployment safeguards. Dependencies can and will change on a schedule outside any single team’s control; regression testing and rollback planning must become part of normal operations.

Today’s feature sits in this environment. We have collaborated again with Richard D. Avila who has over two decades of experience building complex software and AI systems and has led architecture for everything from large-scale simulations and autonomous platforms to data analytics systems. Avila, along with Imran Ahmad has also co-authored Architecting AI Software Systems. The “AI architect” Avila describes in today’s article is a role shaped by post AI production realities and a function we predict will become the cornerstone of successful software engineering teams.

🧠Expert Insight:

The Rise of the AI Architect by Richard D Avila

In Richard D Avila’s previous article, he pointed out that building an AI prototype in the lab is one thing, but turning it into a dependable product is quite another. This is where AI Architecture and the role of the AI architect becomes relevant. Much like a software architect ensures a system’s structure can support its goals, an AI architect is emerging as the key figure to bridge cutting-edge AI with solid engineering. Google, for example, recently appointed Koray Kavukcuoglu (formerly of DeepMind) as its first Chief AI Architect to accelerate the integration of advanced models “into our products, with the goal of more seamless integration, faster iteration, and greater efficiency”. Without sound architecture, AI innovations won’t translate into sustainable, reliable solutions. But who is an AI Architect and what do they do?

AI systems and technology are very quickly impacting so many areas of modern life. Whether it be a seemingly simple task, such as detecting a cat in an image, or the autonomous operation of a vehicle. There is also a shift toward greater acceptance of AI technologies into society. AI systems are primarily built into complex software. What makes AI-enabled systems unique is that they have, at their heart, the notion of decision making. This extended functionality puts new demands on the processes and guides for building these sorts of systems.

Architecting fundamentals can help navigate these complexities and the successful building and operation of these sorts of systems. Historically, an architect would aid in the building of a system and then would be able to sort of “walk away” or execute a new project. In this age of AI, this is simply not the case. The challenges of observability, integrated development, testing, the voice of the user, and their centrality to business outcomes place new demands on an architect.

Challenges that impact AI enabled systems

• Complex integrations: For example, is the system going to be a hybrid cloud or on premise deployment? What dependencies exist on external software?

• High performance bar: the system is expected to be deployed to impact key business functions or activities that the organization shall depend upon.

• Depending on the technology, using the AI/ML can incur significant costs.

• If not well architected or instrumented, the system can be a challenge to troubleshoot.

What can architecture do to help?

• By modeling and conducting robust conceptual designs, the unique perspective and aspects of the AI technology are identified.

• The use of tactics and patterns partitions the system for better performance and meeting non-functional requirements.

• Robust architecting helps keep a holistic view of the end to end system.

The Rise of AI Architects

As modern technology progresses, the practice of architecting also evolves. We are now witnessing the rise of the AI architect. The modern AI architect will be closer to operations and must help with managing the complexity of systems. These include:

• The architect is the principal approver for the completion of acceptance gates for going from an AI prototype to a production deployment. He ensures the concept has merit, evaluates the results of the prototype, guides design of the production model, evaluates the performance of prototypes, and finally accepts the production deployment.

• They integrate and synthesize solutions to balance the data science, data engineering, software development, operations, and business teams.

• The architect is the principal owner of the non-functional requirements and interfaces across the AI system.

• They must be involved in the observability, where they need to understand what decisions the software is making.

From Idea to Production Life cycle

As AI engineering is maturing, a notional process is emerging that takes an AI idea to production. The initial stage is discovery and evaluation. It looks to determine if the data, algorithms, and compute resources exist to make a system that will have impact. The next stage is prototyping; here one now adds complexity to the software, both by adding more production domain elements, to test how well the new system would work. It is at the end of the prototype phase where a key decision is made to determine if it is warranted to take the prototype to a production environment.

With the passage out of the prototyping phase, a controlled execution of the model is done, where it is observed and monitored to ensure it is working as expected and delivering value. With the monitoring phase complete, a controlled rollout of the system is done to production, with another layer of monitoring to ensure the systems it is now impacting still also function as expected. Finally, with these gates completed, a full deployment can be done and now the system goes into a monitor and evaluation phase. The evaluation is to see if the model needs to be adapted or retrained.

One key item is as the system is now in operation – a new set of metrics for system observability need to be captured. Below are some basic ones; this should not be considered exhaustive:

1. Accuracy

2. Data throughput

3. Area under the Curve

4. Time for processing

5. Cost per transaction

How often these metrics should be reviewed really depends on how time dependent AI performance is to the enterprise. That said, these observability metrics should be reviewed frequently by the operations teams, and on a weekly basis or sooner by the other stakeholders, to see how AI model performance is impacting their areas of responsibilities.

Specific Areas an AI architect can Influence

As has been mentioned, the AI architect needs to have a solid understanding of, and be able to communicate in the language of, AI/ML and data science topics. This includes the perspective in understanding the role of non-functional requirements as applied to AI/ML systems. How to extend their knowledge for architecture modeling should incorporate systems where there is to be a sort of non-human decision making. They will also need to fortify and understand how users and stakeholders of the system are to be impacted, aided, and potential friction points. They will need to be close to guiding and evaluating the rapid prototyping in support of software development. The architect needs to consider interfaces and how their management will impact the decision making of the system. The use of AI/ML technologies inherently add a layer of complexity. The architect acts at his own peril with the notion that a simple AI application can readily be made in a less rigorous manner. Adding a layer of AI into almost any application raises the complexity significantly.

The AI architect must by necessity straddle AI/ML and software engineering. This is to ensure the architect can navigate the alphabet soup of techniques, algorithmic aspects, and technologies that exist. It is very easy for an AI architect to be overcome by the various other roles needed to deploy modern AI systems. In many enterprises, there are typically several functional teams: Data Team, Development Team, ML Ops, Legal/Compliance. Below are some of the major facets that the architect influences and needs to be involved with:

Data Team

The architect works with the data team to ensure that the product question can indeed be answered by the data that exists or is to be used. He is pivotal in levying requirements for the speed, format, and consistency checks that must be met.

Development Team

The architect works to ensure top-level designs, interfaces, and non-functional requirements are being met by the designs being created. He is more in a reviewer mode. He also aids in clarifying engineering requirements and providing guidance to address design questions and challenges.

MLOps Team

The architect, in this role, defines how well the production system is required to perform, and how the system is to be instrumented and monitored for correct execution. He will also define the canaries and fail-safe mechanisms to ensure issues with the AI components do not compromise the full production environment. The mechanisms for model and system updating are also laid out by the architect.

Legal and Compliance Team

In this capacity, the architect provides the technical insight, design guidance, and evidence that the AI system is meeting the legal and compliance needs of the enterprise. The architect also oversees re-design activities and troubleshooting to ensure legal and compliance aspects are indeed met.

Incident Run Book

There needs to exist a run book that the architect is a principal in its development. A run book should look to use a layered manner to address an error in the AI model or its interfaces. Clear mechanisms and architecture need to exist to be able to disengage the model from a production system, and put the production system into a nominal configuration. The run book should also provide for clear expected outputs of the different stages of the system – that is, it should be documented what “correct” looks like and why at each stage to enable quicker troubleshooting. The error handling within the system should also be readily indexed in the run book so that system, data, and model tracing can occur. Finally, the run book should also include clear traceability to software repositories, configuration information, and points of contact.

Conclusion

These are exciting times for being an AI architect. The field is in its infancy and there are many excellent and challenging domains where one can make a significant impact. In this new era one needs to accept that there will be constant learning and adaptation. As the techniques and concepts of AI/ML continue to increase, the combinations and applications simply keep growing. That said, having a solid foundational knowledge of AI/ML concepts shall mitigate short-term knowledge gaps. Are you ready to become an AI architect?

🔍In case you missed it…

🛠️Tool of the Week

OpenTelemetry Collector — Vendor-neutral telemetry pipeline for traces, metrics, logs, and profiles

The OpenTelemetry Collector is an open-source service that sits between your applications and your observability backends, receiving telemetry, processing it (filtering, sampling, enriching), and exporting it in a controlled, consistent way. It’s designed to reduce the sprawl of per-vendor agents while keeping the data path configurable and auditable.

Highlights:

• Pluggable pipelines you can reason about: Collector configs are explicitly built from receivers, processors, exporters, and connectors, enabling transparent ingestion → transformation → delivery flows instead of opaque agents.

• Unified multi-signal collection: One codebase can handle traces, metrics, and logs (and supports profiles via components), allowing teams to consolidate telemetry paths and apply consistent policy controls. (open-telemetry.

• Production-grade release cadence and governance: The Collector shipped v1.49.0/v0.143.0 on 05 Jan 2026, with published breaking changes and end-user changelogs—useful when you operate it as shared infrastructure.

Learn more about OpenTelemetry Collector

📎Tech Briefs

• Scaling PostgreSQL to power 800 million ChatGPT users: Details how OpenAI scaled PostgreSQL for ChatGPT/API workloads—using a single primary with large-scale read replication and hard-earned operational lessons from overload-driven SEVs.

• Route leak incident on January 22, 2026: Cloudflare’s post-incident report explains how an automation/policy misconfiguration leaked IPv6 routes for ~25 minutes (triggering congestion/loss), and lists concrete guardrails they’re adding in automation and validation.

• What’s new with Google Cloud— January 24, 2026: Google Cloud’s weekly roundup compiles recent product updates, announcements, and engineering resources in one place (a useful “diff feed” for platform teams tracking change).

• Power agentic workflows in your terminal with GitHub Copilot CLI: GitHub introduces Copilot CLI, positioning the terminal as a first-class surface for agentic developer workflows rather than a thin wrapper around the IDE.

• Headlamp in 2025: Project Highlights: The Headlamp maintainers recap what shipped across 2025 and what they’re prioritizing next—useful signal for platform orgs standardizing cluster UX and operational visibility.

That’s all for today. Thank you for reading this issue of Deep Engineering. We have some really exciting things planned for Deep Engineering members in 2026 and we can’t wait to tell you all about it very soon.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

We are back with cohort two of our live, hands-on workshop with Patrice Roy (author of C++ Memory Management, ISO C++ WG21 member) focused on practical ownership models, RAII, smart pointers, exception safety, allocators, and debugging memory issues in production C++ code. Use code DEEPENG50 for 50% off Standard and Bulk tickets (not valid on Premium Pass).

Save 50% with DEEPENG50

✍️From the editor’s desk,

Happy New Year — and welcome to the first Deep Engineering issue of 2026!

Most senior engineers can solve hard problems. The real test is whether the team can repeat that success—six months later, with new constraints, when “future you” has lost the context that made the design feel obvious at the time.

This issue is Part 1 of a two-part series grounded in our interview with Sam Morley, Research Software Engineer and Mathematician at the University of Oxford and author of The C++ Programmer’s Mindset.

In today’s installment we cover four guiding principles from Morley:

• Be explicit about your process

• Decompose relentlessly

• Treat abstraction as a multi-cost decision

• Use the STL as your baseline

We’ve also included the complete Chapter 1: Thinking Computationally from Morley’s book. Our tool of the week today is Tracy, a practical complement to this mindset. In Part 2, we’ll extend the same framing into maintainability under “power tools,” hardware-aware performance, and safer concurrency. You can also watch the complete interview and read the transcript here.

Adopting the C++ Programmer’s Mindset (Part 1) with Sam Morley

Adopting the “C++ programmer’s mindset” means combining sound engineering habits with deep knowledge of the language’s strengths. In this first part, we learn from Sam Morley that senior engineers should actively reflect on how they solve problems, relentlessly break down big challenges, choose abstractions with care, and take full advantage of the rich standard library. These practices lead to code that is not just correct, but also maintainable and efficient by construction. As Morley puts it, it’s about “how you marry [your process] with the C++ language and the broader system to make a better, faster solution.”

Embrace Computational Thinking and Process Awareness

Senior C++ engineers often solve complex problems intuitively, but Sam Morley argues that making this process explicit is key to growth. Morley emphasizes “thinking about your process” – after solving a problem, step back and ask how you approached it.

By breaking down challenges, identifying abstractions, and noting familiar patterns or novel obstacles, engineers can refine their problem-solving toolkit.

This reflection helps avoid repeated mistakes. As Senior iOS Engineer, Jeremy Fitzpatrick also writes, retrospectively analyzing how errors happen – and how you fixed them – is vital to prevent falling into the same pitfalls again. Morley adds that consciously “documenting” your approach (even just mentally) builds a knowledge base you can pass on to junior developers. This mindset shift – from just writing code to analyzing how you write it – turns problem-solving into a teachable, improvable skill.

The C++ programmer’s mindset ties this process to the capabilities of the language and system. Morley notes that experienced devs already use “computational thinking,” but connecting those abstract problem-solving steps with C++’s specific features and the underlying machine can lead to more efficient solutions. In practice, this means considering how C++ abstractions (like classes, templates, or the STL) and system resources (like CPU caches or IO throughput) inform each step of your solution. Adopting this broader perspective – thinking not just about what to code but why and how in context – is what elevates a senior engineer’s effectiveness.

Break Problems Down

A core principle Morley highlights is decomposition: any non-trivial software problem must be broken into smaller parts.

“The notion that one can solve a problem without breaking it down... is kind of folly,” he says.

Even if it happens unconsciously, every big challenge comprises sub-problems. Experienced engineers should do this deliberately and strategically. Some ways of slicing a problem are more efficient than others, so it pays to practice systematic decomposition.

Morley illustrates this with a real-world project processing video data through an AI service. The task – extract frames from 100,000+ videos, send frames to an Azure ML API, then store the results – was obviously too complex to tackle monolithically.

Morley split it into four stages: reading and decoding video files, dispatching frames to the cloud API (with careful rate limiting), collecting the results, and writing outputs to disk. Each stage was further subdivided: for example, the output-writing stage gained a buffered queue to prevent slow disk writes from bottlenecking the faster network responses. This iterative divide-and-conquer approach revealed hidden bottlenecks (disk I/O in this case) and allowed targeted fixes (introducing back-pressure when the buffer grew).

The take-away: decompose early and often. Start with large components, then refine each into smaller, more tractable units until reaching operations you know how to implement (or can leverage from libraries).

This not only makes problems solvable; it also isolates performance issues and enables concurrency where possible. By “bringing down the level of the larger problem to small, atomic things,” you can apply known solutions or standard libraries at each step. Smart problem breakdown is foundational to the C++ mindset – and indeed any engineering mindset – for complex systems.

Choose the Right Abstraction for the Job

C++ offers many ways to abstract – from simple functions to classes, templates, and beyond – but Morley cautions that no abstraction comes free.

“People claim zero-cost abstractions... They’re really not; every abstraction has a cost,” he says.

This cost may not be at runtime, but could be compile-time overhead or added code complexity. For instance, template instantiations can incur heavy compile-time costs even when they impose little or no runtime penalty. As one accepted definition puts it, zero-cost abstractions in C++ mean “no runtime cost, only compile time cost (the code will be slower to compile)”. In practice, template-heavy code can slow builds and strain developers’ ability to reason about the code. Maintaining a balance is essential.

A useful way to make the trade-offs explicit is to treat “abstraction cost” as multi-dimensional:

• Runtime cost: additional instructions, indirection (e.g., virtual dispatch), cache effects.

• Compile-time cost: template instantiation, header inclusion blow-up, longer incremental builds.

• Cognitive cost: readability, debuggability, and how hard the code is for “future you” (and teammates) to understand.

• Change cost: how easy it is to adapt the abstraction when requirements shift (interfaces that are too narrow or too general can both be expensive to fix).

Choosing the appropriate abstraction level

Morley’s rule of thumb is to aim for the simplest abstraction that solves the problem without painting yourself into a corner. In practice, that translates into a tiered decision model:

• Start with a straightforward function if the behavior is local, stateless, and unlikely to require customization: This keeps the call site obvious, the code easy to inspect, and the build impact minimal.

• Move to a small function template when you need type flexibility at low runtime cost: Morley will template a utility function early if it might be reused with different types, noting that “the cost of doing this is basically nil” in terms of added complexity—provided the template stays small and the diagnostics remain tractable.

• Reach for a class only when there is a clear need that a function (templated or not) does not satisfy. Morley’s three “good reasons” are:

  • State management: you need to encapsulate and protect internal invariants (e.g., std::vector managing allocation and lifetime correctly).

  • Interchangeable interface: you want a stable outward contract while swapping underlying mechanisms (e.g., reading from disk vs. network behind a consistent read()/write() API).

  • Customization point: you want controlled extensibility—often via class templates and specialization patterns that are harder to express cleanly with function templates alone.

Classes—especially those using inheritance/virtual functions—bring additional considerations:

• Potential runtime overhead (e.g., vtable dispatch) that may be fine off the hot path.

• Increased cognitive load (more surface area, invariants, lifecycle, ownership).

• More complicated testing and integration if responsibilities are not tightly scoped.

Beware over-abstraction

It is also important to beware of over-abstraction. It’s easy, especially for seasoned C++ engineers, to create elaborate class hierarchies or template meta-programs that technically solve the problem but introduce needless complexity. Every additional layer of indirection or generality can obscure the logic.

Morley’s countermeasure is disciplined minimalism:

• Design the minimal abstraction that solves today’s problem.

• Treat “clever” template tricks and extra abstraction layers as a last resort, not a default.

• When adding complexity, ask: what concrete benefit does this buy—performance, flexibility, or clarity?

• If the answer is not specific, measurable, and defensible: remove the abstraction.

Keeping abstractions lean reduces build times and makes life easier for “future you” (and anyone maintaining the code). Morley notes that over-specifying an interface can be just as costly as under-designing one: if an interface is too narrow, you may end up reworking it when the real generality becomes clear.

Leverage the Standard Library and Proven Tools

One practical takeaway that Morley stresses is to use the C++ Standard Library (STL) whenever possible. Reinventing data structures or algorithms that are already well-implemented is usually wasted effort and risk.

“The STL is always there and you can always use it... these are very good, high-performance facilities that make your life easier,” Morley notes.

Using the standard <algorithm> or <vector> isn’t just about saving coding time — it sets a baseline of reliability and portability. Relying on the STL means you spend less time debugging low-level issues or re-optimizing basic operations, and more time on the unique logic of your application.

Mastery of the standard library often distinguishes top-tier C++ developers. Knowing the ins and outs of containers and algorithms allows you to recognize common problems (searching, sorting, transforming collections, etc.) and apply a ready-made solution. This can dramatically speed up development.

Morley points out that using STL algorithms to get an initial solution can reveal where the real performance bottlenecks are — frequently it’s not the standard component itself. In many cases, the out-of-the-box std::sort or std::unordered_map will be sufficient. If not, you have a working baseline to optimize further.

If you do need more specialized tools, there are established libraries like Boost or Abseil. These offer:

• Extended containers (e.g., Boost multi-index, Abseil flat_hash_map)

• Drop-in replacements with different trade-offs (e.g., small-size optimized vectors)

Google engineers Jeff Dean and Sanjay Ghemawat suggest techniques such as using an absl::InlinedVector (which stores small sequences in-place) for performance-critical code, as this yields gains “without adding non-local complexity” — essentially a free win if your data fits in the small buffer. Knowing about such tools is part of the seasoned C++ mindset: you raise the floor of what problems you can solve effortlessly. Why hand-roll a ring buffer or thread pool when battle-tested implementations exist? The wisdom is to trust well-known libraries for standard needs — and reserve custom implementations for truly novel problems, after asking “why am I re-inventing this wheel?” (Often, you shouldn’t be.)

However, being a good C++ citizen also means keeping an eye on the costs. The STL’s flexibility might not always be optimal for your use case (e.g., std::map might be too slow for very large data if cache misses dominate, where a flat array plus binary search could outperform). Morley’s approach is:

• Start with the idiomatic solution using STL or Boost

• Measure performance

• Iterate based on evidence

• Swap in specialized containers/algorithms if needed

But even then, prefer using a well-regarded library implementation over crafting your own. Writing your own containers or memory management is notoriously tricky — a fact Morley underscores. Issues like exception safety, pointer invalidation, and performance tuning of allocations are devilishly complex. “If you are reimagining containers, you should be asking why rather than how,” Morley says. In modern C++ development, leaning on the collective wisdom embedded in the standard library (and its extensions) is a mark of pragmatism and professionalism.

Up Next

Next month, In Part 2, we will delve into additional facets of this mindset: keeping C++ codebases clean and team-friendly, balancing advanced “power tools” like metaprogramming with readability, thinking in terms of hardware (CPU architecture, memory hierarchy) for performance, and strategies for safe concurrency and memory management.

🧠Expert Insight

1. The complete Chapter 1 from Sam Morley’s book, The C++ Programmer’s Mindset:

2. Our complete interview with Sam Morley in article and video format:

🛠️Tool of the Week

Tracy Profiler — Real-time timeline profiler for C++ systems

Tracy is an open-source, low-overhead profiler that captures CPU/GPU activity, allocations, and synchronization behavior into a unified timeline view, so you can identify bottlenecks and contention based on traces rather than inference.

Highlights:

• CPU profiling with low-friction integration: Direct support for C and C++ (and additional language integrations), designed for instrumented “zones” and practical iteration.

• GPU profiling across major APIs: GPU support spans OpenGL, Vulkan, Direct3D 11/12, Metal, OpenCL, and CUDA, enabling end-to-end frame and workload analysis.

• Memory and concurrency visibility: Tracks memory allocations, locks, and context switches, making contention and allocation churn observable rather than inferred.

Learn more about Tracy Profiler

📎Tech Briefs

• GitHub Copilot CLI adds “plan mode” and richer agent controls: In a January 21, 2026 GitHub Changelog post, GitHub says Copilot CLI now supports plan mode (toggle with Shift+Tab) plus features like configurable reasoning effort for GPT models and a new /review command for reviewing changes directly in the terminal.

• CodeQL 2.23.9 ships, with a Kotlin deprecation clock starting: GitHub’s January 20, 2026 changelog notes CodeQL 2.23.9 is released (no user-facing CLI/query changes), and states Kotlin 1.6 and 1.7 support is deprecated and slated for removal in CodeQL 2.24.1 (planned February 2026).

• Google Cloud Storage introduces “dry run” for batch operations: In Cloud Storage release notes dated January 16, 2026, Google says you can now use dry run mode to simulate Storage batch operations jobs “without modifying or deleting data,” to validate configuration before running the real job.

• AWS announces EC2 X8i GA with large-memory specs and published perf deltas: On January 15, 2026, AWS developer advocate Channy Yun announced general availability of EC2 X8i instances, claiming up to 6 TB memory, 1.5× more memory capacity, 3.4× more memory bandwidth, and up to 43% higher performance versus prior-generation X2i instances (with workload-specific figures also listed).

• Mandiant releases Net-NTLMv1 rainbow tables to accelerate deprecation: In a January 16, 2026 Google Cloud blog post, Nic Losby (Mandiant) says Mandiant is publicly releasing Net-NTLMv1 rainbow tables, asserting the dataset enables defenders/researchers to recover keys in under 12 hours using consumer hardware costing less than $600 USD.

That’s all for today. Thank you for reading this issue of Deep Engineering. We have some really exciting things planned for Deep Engineering members in 2026 and we can’t wait to tell you all about it very soon.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

If you are making decisions about AI systems in 2025, you cannot afford a fuzzy view of agents.

Join us on Dec 13, 2025 · 9am–2pm ET / 6am–11am PT for a 5-hour deep dive with the creators of AG2/AutoGen and leading researchers from MIT and Cambridge on real multi-agent architectures, autonomous research agents, and AI-native companies. Deep Engineering readers get 50% off with code FINAL50.

Register now

✍️From the editor’s desk,

This is our final Deep Engineering issue of the year as we break for the holidays 🎄. Thank you for reading, sharing, and thinking with us in 2025 — I hope you get time to rest, recharge, and come back to your work with a clearer head and a mysteriously missing backlog in the new year.☃️

In TestGrid’s 2025 benchmark, based on 7.3 million automated test runs across 55,800 organizations, the average pass rate is 75% — better than last year’s 59%, but still roughly one in four runs failing. For teams betting on CI, CD, and AI-assisted development, that gap is where production risk hides.

How should teams rethink test-driven development and developer testing in the context of legacy systems, AI-generated code, and increasingly automated pipelines?

To explore that, I sat down with Kevlin Henney — independent consultant, speaker, writer, and trainer. Henney works with companies on code, design, practices, and people; contributes to the Modern Software Engineering YouTube channel; co-authored A Pattern Language for Distributed Computing and “On Patterns and Pattern Languages” in the Pattern-Oriented Software Architecture series; and edited 97 Things Every Programmer Should Know and co-edited 97 Things Every Java Programmer Should Know. His work sits at the intersection of everyday development practice and long-term design thinking.

In this issue, we focus on:

• TDD vs “just testing”: what TDD actually is, and why conflating it with ad hoc developer testing hides deeper gaps.

• Design pressure from tests: using tests as executable specifications that expose coupling, cohesion, and legacy-code seams worth reshaping.

• AI in the loop: why AI-generated code and tests increase the need for testing and review skills rather than replacing them.

You can watch the full conversation with Kevlin Henny below, read the complete Q&A here, or scroll down for the distilled insights.

Sponsored:

The First AI Employee for Your Mobile App: Fload monitors your app 24/7, catches issues before they cost you money, finds new growth opportunities, and predicts what’s coming next, all automatically. Connect your app in 60 seconds.

Try Fload Free

Rethinking Test-Driven Development for the AI Era with Kevlin Henney

Test-driven development sits in an awkward place in many teams: widely cited, unevenly practiced, and often misunderstood. For some developers, TDD is a niche technique only for greenfield code; for others, it’s reduced to “writing some unit tests” after the fact. Meanwhile, real-world pressures — legacy systems, language ecosystems, CI pipelines, even AI-generated code — make it challenging to apply TDD consistently under tight deadlines. In my conversation with Kevlin Henney, we talked about:

• Why TDD adoption stalls even for experienced developers

• The misconceptions that blur the line between “developer testing” and true TDD

• How tests can shape software design without losing sight of the bigger architectural picture.

• Best practices for introducing tests into large legacy codebases

• How language and ecosystem culture influence testing practices

• What distinguishes good, specification-like tests from brittle checklists.

• Henney’s perspective on AI-assisted development — warning about the risks of offloading testing to code generators — and explained why, in an era of increasingly automated code, testing and review skills matter more than ever.

What follows is a distilled summary of a part of our extensive conversation, which I highly recommend reading or listening to in full here.

Why TDD Is Hard to Adopt

For many experienced engineers, TDD requires unlearning comfortable habits. It forces a fundamentally different workflow – writing tests first and working in small increments – which “is always going to be difficult” when you’re used to another rhythm. Many developers also lack a strong testing practice to begin with, so TDD’s strict discipline feels like a shock.

Henney notes that the classic “red, green, refactor” mantra by itself often isn’t convincing; without context, developers ask, “Why write a test for something I haven’t built yet?” The answer is that the failing test defines the next behavior you intend to add. You’re writing a concrete example of a new requirement – essentially a to-do list item for the code. TDD limits you to one small change at a time, preventing the urge to over-build and ensuring you constantly get feedback (tests passing or failing) on your design. But making this mental shift is hard without practice.

Common Misconceptions

TDD often suffers from misunderstandings. Some of the myths Henney debunks include:

• “TDD is only for new code.” In reality, you can introduce TDD practices into legacy systems gradually – it’s not limited to greenfield projects.

• “TDD just means writing tests.” No – it refers to a specific test-first development cycle. Writing some tests after coding is simply testing, not TDD.

• “TDD isn’t about testing, it’s about design.” Not exactly. TDD is an intensive testing technique and a design aid. The tests are the vehicle, driving better design, but you are absolutely still writing tests (and lots of them) as you go.

Tackling Legacy Code

When dealing with a large, untested codebase, aim for incremental improvements instead of perfect coverage. Henney’s advice:

• Start with a strategic slice: Identify a part of the code you need to modify for upcoming work, and focus your efforts there. Break dependencies to isolate that component (using safe refactoring tools and the compiler, if available), then add some tests around it and refactor in small steps. You can’t fix everything at once, but you can gradually create pockets of well-tested code.

• Prioritize trouble spots: Use version history and bug reports to find the areas that change often or cause frequent issues. Target those “hot spots” for testing and cleanup first, where better tests will have the most payoff. Stable sections of the system can wait until they actually need attention.

What Do Good Tests Look Like?

Quality tests are the backbone of TDD. Henney offers a couple of guidelines for writing tests that stay valuable over time:

• Test one concept per test: Each unit test should exercise a single behavior or scenario, not a grab-bag of things. This usually means writing tests at the level of use (what the code should do), not one test per function. For example, instead of separate tests for deposit and withdraw methods, write tests that express the outcomes (e.g. depositing increases the balance, withdrawing decreases it). A good test name reads like a specification, so if it fails, you immediately know what requirement broke.

• Use tests to get design feedback: If a piece of code is hard to test, treat that as a sign that the design might need refactoring. TDD shines when you listen to those signals. For instance, if you’re setting up excessive mocks or needing to call private functions, consider splitting responsibilities or adding interfaces to decouple the design. The goal is to make both the code and the tests simpler. As Henney puts it, often “excessive use of mocking indicates a design that could be simplified.”

The Role of AI and the Future of TDD

Henney argues AI doesn’t replace the need for good testing – if anything, it heightens it. “If you do not already have an automated testing habit, now is a good time to start,” he says. AI tools can generate a lot of code or tests in seconds, but this often creates an illusion of speed – you get output quickly, then spend days fixing it. Henney advises teams to be skeptical and thorough: use AI as an assistant for ideas, but review everything it produces. Treat AI suggestions as if they came from a junior developer – useful as a starting point, but to be thoroughly reviewed, tested, and refined by experienced developers.

In the coming years, as more coding tasks become automated, the differentiator for great engineering teams will be their testing and review skills. TDD remains as relevant as ever. Its emphasis on clarity, rapid feedback, and thinking through requirements can keep software quality high. The specific tools may evolve, but the core discipline of writing tests to drive design and guard against regressions will continue to pay dividends.

🧠Expert Insight

🛠️Tool of the Week

Hypothesis: Property-based testing for Python)

Hypothesis is a mature, open source property-based testing library for Python. Instead of hand-picking a few example inputs, you describe the space of valid inputs and Hypothesis automatically generates many concrete cases—including edge cases you probably would not have thought of.

Where it helps teams

• Hard-to-enumerate behaviors: Algorithms, parsers, serialization code, numeric logic, scheduling logic, and anything with a large or subtle input space.

• Guarding against AI-generated regressions: If AI suggests an implementation, property-based tests help check that it satisfies an invariant across many inputs instead of just the obvious happy paths.

• Design feedback: When a function is hard to express as “for all inputs in this domain, the following relationship holds,” that friction often reveals API or modeling issues worth revisiting.

Learn more about Hypothesis

📎Tech Briefs

• TestGrid | Continuous Testing Benchmark Report 2025: Trends and Findings: TestGrid’s latest benchmark blog post analyzes 7.3 million automated tests across 55,800 organizations and shows how continuous testing tied directly into CI pipelines affects reliability, runtime, coverage, and AI-assisted failure analysis in production environments.

• CMocka 2.0 Released: Enhancing Unit Testing in C: Andreas Schneider announced CMocka 2.0, a major upgrade of the open-source C unit testing framework that modernizes it around C99 types, adds richer assertion and mocking APIs, supports TAP 14 output, and improves CI/CD integration while keeping API stability for existing test suites.

• VectorCAST + AI: Instantly Generate Unit Tests from Your Requirements (Early Access): Vector introduces an early-access program for an LLM-powered test-case generation engine in VectorCAST that turns detailed natural-language requirements into executable unit tests for C/C++, aiming to accelerate requirements-level coverage while keeping engineers in the review loop.

• John Pourdanis | More Scripts, Same Problems: Why Test Automation isn’t Automagic: In this practitioner essay, John Pourdanis argues that simply adding more automated checks does not guarantee quality and emphasizes risk mapping, shared ownership, exploratory work, and feedback loops over raw script volume for effective test automation strategies.

• eInfochips | Accelerating Continuous Integration with Buildkite, GitHub, and S3 Bucket: eInfochips has outlined a production-grade CI setup using Buildkite, GitHub, and AWS S3, walking through org setup, agents, queues, pipeline-as-code YAML, and test automation best practices focused on fast feedback and reliable quality gates.

That’s all for today — and for this year. Thank you for reading Deep Engineering, sharing it with colleagues, and pushing us with thoughtful feedback. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back in the new year with more expert-led, practice-focused issues.

Merry Christmas🎄and Happy Holidays🦌 — and don’t forget to stay awesome🌟!
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

🎶A small musical send-off for the year: Vince Guaraldi Trio – “Christmas Time Is Here”

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Have a friend or teammate who keeps saying they want to learn Python?

On Thu, Dec 12, 9:00–10:30am ET, Python Illustrated authors Maaike and Imke van Putten are running a free, virtual, live beginner intro to Python in an online editor—no installs, no setup, just Hello World, simple logic, a small mini-project, and Q&A with a book giveaway.

Spread the word—and a bit of holiday cheer—from Deep Engineering.

Register for free →

✍️From the editor’s desk,

In his keynote at AWS re:Invent 2025, AWS CEO Matt Garman said that AI agents will ultimately have a bigger impact than the internet or cloud computing themselves. At the same event, AWS introduced an Agentic AI Specialization for partners, with CrowdStrike highlighted as an inaugural launch partner securing agentic workloads on AWS. Enterprises are now being asked to run AI—and agentic AI—in production, at scale, on real architectures.

What does it take to architect AI systems that meet real constraints on cost, reliability, security, sustainability, and operations?

To explore that, we’ve collaborated again with Imran Ahmad, data scientist, educator, and author focused on algorithms, AI, and cloud computing. He leads machine learning projects for the Canadian government, teaches at Carleton University, and is an authorized instructor for AWS and Google Cloud. With Packt, he has authored several books including Architecting AI Software Systems (2025, with co-author Richard D Avila).

Here’s what’s in today’s issue:

• Feature: “Architecting AI Systems for the Real World” - Key Insights from my conversation with Imran Ahmad

• The complete “Chapter 1: Fundamentals of AI System Architecture” from Architecting AI Software Systems by Richard D Avila and Imran Ahmad.

• The full Deep Engineering interview with Imran Ahmad, covering agentic AI, RAG vs. agentic RAG, UX, cross-functional teams, and how the architect’s role is changing.

• And In today’s Tech Briefs, we look at MCP’s November 2025 spec and the security research landing alongside it, Fara-7B as Microsoft’s on-device agentic model for direct UI control, vLLM-Omni as a new framework for efficient omni-modal model serving in production, Anthropic’s internal study on how engineers actually use Claude and Claude Code in daily work, and more.

Let us begin.

Architecting AI Systems for the Real World: Key Insights from Imran Ahmad

AI is everywhere, but turning a machine learning prototype into a reliable, cost-effective product is hard work. Simply dropping a model into an existing system rarely works—success requires sound architecture and planning. Imran Ahmad has seen this firsthand as a data science leader and AI educator and shares practical advice on designing scalable, sustainable AI systems that work in the real world.

Why AI Solutions Need Strong Architecture

An AI product isn’t just a good model—it’s the whole system around it. To take a prototype from the lab to production, you must design for real-world conditions from the start. Ahmad suggests rating your architecture against 6 key pillars:

• Security: Protect data and access.

• Reliability: Works consistently.

• Performance: Meets speed/latency needs efficiently.

• Cost: Runs affordably at scale.

• Sustainability: Avoids wasted energy/resources.

• Operational Excellence: Maintainable operations.

The stronger your design is in these areas, the better its chances of thriving in production. Ahmad also stresses nailing software fundamentals (like modular design and solid data pipelines) before adding AI. For example, you can use cloud elasticity (scaling resources on demand) to achieve high performance without paying for idle servers.

From Concept to Design – Iterate and Evaluate

Designing an AI solution means choosing the right components (data pipelines, models, interfaces) for the job. Be prepared to iterate on this design. The first architecture you sketch might not be optimal. As Ahmad puts it, “Your first attempt at the design may not be the best one. You have to rethink; you have to evolve.” A good way to vet a design is to build a pilot (a scaled-down version) and see if it meets your speed, reliability, and cost goals. If not, adjust early before full deployment.

Another step is checking the project’s viability up front. Ahmad suggests asking: Cost – can we afford it at scale? Speed – will it make things faster? Accuracy – are the results better than the current method? You don’t need improvements in all three, but be clear about which benefit your AI delivers and ensure it outweighs any drawbacks.

Architecting for Scalability and Flexibility

AI systems often need heavy compute power, but avoid running expensive resources 24/7 if you don’t have to. One smart strategy is to decouple data and compute: keep data in durable storage, and spin up compute instances only when needed (for training jobs or batch processing), then shut them down. This yields high performance on demand while keeping costs and energy use in check.

Also, design differently for model training vs. inference. Training jobs can run in batch mode (e.g. overnight on cloud GPUs) and then be turned off. Serving live predictions might require always-on servers for low latency, but not every feature needs instant results. If a task can be done hourly or daily, use batch processing instead of real-time APIs and save resources.

Choosing cloud vs. on-premises deployment is another factor. It depends on data sensitivity, regulations, and legacy systems. If you move to cloud, make sure to redesign to use cloud features like auto-scaling—simply moving an on-prem design into cloud without changes can end up costing more. Aim for an architecture that scales and adapts efficiently as needs grow.

Avoid Pitfalls – Lock-In and Rigor

Beware of tying your solution to one provider’s tools. Vendor lock-in is risky if that provider fails or becomes too expensive. Use abstractions and standard frameworks so you can swap components easily. For example, use a library that supports multiple ML backends instead of coding directly to one vendor’s API, and containerize your application to keep it portable across clouds.

Another pitfall is weak engineering practices. Many AI projects lack tests, monitoring, or documentation. Treat an AI system like any critical software: write tests for data pipelines and model outputs, monitor performance, and plan for maintainability and security. Have cross-functional stakeholders agree on what a successful outcome looks like (covering both model metrics and system reliability/UX). This ensures the final system is robust from all angles.

User Experience Matters

No matter how powerful the AI, users will reject it if it’s hard to use or understand. Always design the AI solution as a simple service for the end user, hiding the complexity. For example, Google Maps runs complex algorithms for routing and traffic, but to the user it’s just “enter destination, get directions.” Likewise, make sure your AI’s outputs are presented in a clear, helpful way. Good UX builds trust and encourages adoption of the system.

Looking Ahead

The AI landscape continues to evolve. One major trend is multi-agent systems—multiple AI agents collaborating autonomously. This approach brings new capabilities and new challenges, and better methods for integrating enterprise knowledge into AI are emerging as well. For architects, such changes make continuous learning essential. The architect’s role is now a constant involvement—guiding the system through iterations. And while AI technology moves fast, core principles remain the anchor of success: keep systems secure, reliable, efficient, cost-effective, maintainable, and user-friendly.

🧠Expert Insight

1. The complete “Chapter 1: Fundamentals of AI System Architecture” from the book, Architecting AI Software Systems by Richard D Avila and Imran Ahmad

2. The complete Deep Engineering interview with Imran Ahmad

🛠️Tool of the Week

KServe: Kubernetes-Native Model Serving for Scalable, Vendor-Agnostic AI Inference

KServe is an open-source, Kubernetes-native platform for deploying and operating predictive and generative AI models in production using a standard, cloud-agnostic inference API.

Highlights

• Open-source and actively maintained: Apache-2.0 licensed CNCF project with ongoing development and an active community.

• Proven in production: Adopted by organizations running real-world inference workloads across multiple model types and frameworks.

• Built for scalable inference: Provides an InferenceService abstraction, autoscaling (including scale-to-zero), GPU support, and support for common ML/LLM frameworks.

• Cloud-agnostic architecture: Runs on any Kubernetes cluster (on-prem or cloud), helping teams avoid hard coupling to a single vendor’s serving stack.

Learn more about KServe

📎Tech Briefs

• MCP’s November 2025 Spec: Long-Running AI Agents Meet Very Real Security Issues: Deep Engineering’s MCP analysis pairs the new 2025-11-25 Model Context Protocol spec—focused on long-running tasks and enterprise security—with fresh research on real-world MCP deployments that have already suffered exposed servers, critical vulnerabilities, and a 35-day cross-tenant data leak.

• Fara-7B: On-device agentic model for computer use: Microsoft Research introduces Fara-7B, a 7B-parameter “computer use agent” that drives the UI directly (mouse, keyboard, scrolling) instead of just answering in text. It runs locally on Copilot+ PCs and via open weights (MIT license) on Foundry and Hugging Face, targeting real-world web tasks like form filling, booking, and information lookup.

• vLLM-Omni: Omni-modal model serving for text, image, audio, and video: The vLLM team announces vLLM-Omni, an extension of vLLM for high-throughput omni-modality serving. It adds support for text, images, audio, and video, and goes beyond standard autoregressive LLMs to handle diffusion transformers and other parallel generation architectures.

• How AI is transforming work at Anthropic: Anthropic publishes an internal study on how its own engineers use Claude and Claude Code: 132 engineers surveyed, 53 interviews, and usage data analysis. Engineers report using Claude in ~60% of their work with self-reported ~50% productivity gains and ~27% of Claude-assisted work representing tasks that simply wouldn’t have happened otherwise.

• CrowdStrike Operationalizes and Secures Agentic AI Workloads on AWS: CrowdStrike becomes an inaugural AWS Agentic AI Specialization partner and describes how it is “operationalizing and securing agentic AI workloads on AWS.” The release outlines an “agentic SOC” (security operations center) where human analysts orchestrate fleets of security agents built on the CrowdStrike platform.

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Join us on Dec 13, 2025 · 9am–2pm ET / 6am–11am PT to discover how AG2, autonomous research agents, and real-world multi-agent systems are reshaping science, engineering, and AI-native companies. Get 40% off with code DEEPENG40 and secure your spot at the frontier of agentic AI. Why attend?

Unlock 40% off – Register with DEEPENG40

✍️From the editor’s desk,

According to McKinsey’s November 2025 State of AI survey, 23% of enterprises say they’re already scaling at least one agentic AI system and another 39% are experimenting — yet in any given function fewer than 10% have agents at scale. At the same time, Salesforce’s SCUBA (Salesforce Computer-Use Benchmark for Agents) — a 900-plus-task benchmark built from real CRM workflows — finds that leading open-source computer-use agents currently solve under 5% of realistic enterprise tasks end-to-end, while even the best closed-source systems only reach around 39% success, climbing to roughly 50% only when heavily augmented with human demonstrations.

If frontier models are already capable of solving 30–50% of complex workflows end-to-end in controlled benchmarks, why are so few enterprises able to get agents beyond narrow, local successes in production? One hypothesis, which we delve into in today’s issue is that the bottleneck is less about raw model capability and more about how we structure control and memory around these systems.

We have collaborated again with of (AI Architect | ex: AWS, Lululemon, Nike | Author of Clean Architecture with Python) to take us through moving from explicit, command-driven memory (!remember) to implicit, LLM-managed memory, with the host application reduced to infrastructure hooks and hard security guardrails. It’s a concrete design pattern for delegated control — and a useful lens on what we want our next generation of agents to be allowed to decide on their own.

Sponsored:

Quantum Effects 2026: Where Quantum Tech Meets Industry

Leading annual exhibition and conference for application-oriented quantum technology, uniting research, start-ups and business in Stuttgart, 6–7 Oct 2026.

Learn more

🧠Expert Insight

The Irony of Explicit Memory Controls by Sam Keen

In my previous post, The Memory Illusion, I demonstrated that LLM memory doesn’t require vector databases or sophisticated architectures. It’s fundamentally just text management. We built a proof-of-concept in ~150 lines of Python that stored memories in a simple markdown file. It worked. But it had an amusing limitation: The user had to remember to tell the AI to remember.

The system required explicit commands. Want the LLM to store your name? Type !remember “My name is Alex”. Want it to know your project preferences? Another !remember command. The irony was sharp: we’d outsourced memory to technology, only to burden ourselves with managing that memory manually.

This wasn’t an oversight. It was a conscious design decision. Our application code controlled every memory operation through explicit if/else logic. The host app was the memory manager, and the LLM was simply our text processor.

But what if we handed that authority to the LLM itself?

This isn’t an incremental improvement or “v2” of the same approach. It’s a fundamentally different philosophy: trusting the LLM to autonomously manage its own memory. The technical implications are profound. We move from programming specific behaviors to setting high-level intentions. We shift from writing parsing logic to defining trust boundaries.

This approach enables the system to handle its own errors, organize information without explicit rules, and maintain its own memory hygiene. All without writing a single if/else statement.

The Explicit Approach: When Code Defines Every Decision

In my original POC, every memory operation required explicit user commands. Here’s what a typical session looked like:

[You]: Hello, I’m working on a React app
[Claude]: Hi! What kind of React app are you building?
[You]: !remember I am building a React e-commerce application
[Claude]: [Memory saved]

Here we see that required use of !remember. Adding to the user’s cognitive load alongside their actual work.

Behind the scenes, our code intercepted every message, parsed for commands, managed file operations, and reconstructed the prompt with memories for each interaction. We were the brain; the LLM was just processing text within our constraints.

This gave us complete control. We defined in code the exact format of the memory file. Want memories timestamped? We coded it. Want them categorized? More code. Every behavior was explicit, predictable, testable.

This is how we’ve built software applications since the inception of the craft. We write the logic, we define the control flow, we handle the edge cases. It’s comfortable, familiar territory. But when working with LLMs, this traditional approach means we’re not fully leveraging what makes them truly powerful: their ability to understand context and make intelligent decisions autonomously.

The LLM’s contextual intelligence sits idle while our code makes every decision. This intelligence was trained on billions of examples of how humans organize and retrieve information.

Most importantly, users had to remember the commands, creating friction in your app’s usability. Edge cases multiplied. The code grew ever larger as we handled more scenarios, more commands, more special cases. We were swimming upstream against the fundamental capabilities of modern LLMs.

The Implicit Approach: LLM as Autonomous Manager

The paradigm shift is what matters. We’re implementing a harness that grants the LLM autonomous authority. While this example uses the Claude Agent SDK, the pattern can be implemented with other SDKs or custom code. The key is delegation of decision-making, not specific tooling.

Let’s look at the skeleton of our memory tool implementation:
Full code found in the companion app:

# memory_tool.py 
class LocalFilesystemMemoryTool(BetaAbstractMemoryTool):
    “”“
    The LLM calls these methods autonomously based on context.
    We provide the infrastructure; Claude makes the decisions.
    “”“

    # The hook methods (memory tools) Claude is made aware of:

    @override
    def view(self, command):
        “”“Claude calls this to read memories or list files”“”
        # Validate path, read file/directory, return contents

    @override
    def create(self, command):
        “”“Claude calls this to create new memory files”“”
        # Validate path, write file, log operation

    @override
    def str_replace(self, command):
        “”“Claude calls this to update existing memories”“”
        # Find text, replace it, handle errors

    @override
    def insert(self, command):
        “”“Claude calls this to add lines to memories”“”
        # Insert at specific line number

    @override
    def delete(self, command):
        “”“Claude calls this to remove memories”“”
        # Delete files or directories

    @override
    def rename(self, command):
        “”“Claude calls this to reorganize memories”“”
        # Move or rename files

The SDK provides these hook methods as the interface contract. We implement the file operations; Claude decides when to invoke them. No command parsing required from us.

The system prompt grants authority. Instead of telling Claude what to remember, we grant it authority:

This shift is profound. We’re not programming behaviors anymore. We’re setting intentions and trusting Claude to execute them. The orchestration that once consumed lines of code now happens autonomously, guided by our system prompt.

This hands-off approach of providing just the infrastructure hooks while delegating all decision-making to the LLM may seem minimal. That’s precisely the point.

If you’ve spent years writing deterministic code, this delegation feels uncomfortable. You’re trusting the LLM to make architectural decisions you once controlled. That discomfort is valid. Let’s add some clarity to the real world implications of this approach.

Under the Hood: Autonomous Decisions in Action

Let’s trace what actually happens during a conversation. These examples come from real sessions the companion app’s trace and render diagram features.

Creating New Memory

[You]: “I’m starting a new project. I’d like to build a web app for solo entrepreneurs
       to track their time and projects”

~Claude internally~: Creates new memory at path `project_solo_entrepreneur_app.txt`

[Claude]: “That’s an exciting project! A time and project tracker specifically for solo entrepreneurs ...”

You didn’t ask Claude to remember anything. Claude autonomously decided this information was worth preserving. It even made an architectural decision. It created a new project file rather than appending to a general profile. These are judgment calls the LLM now makes without any explicit rules from us beyond guidance in the system prompt.

Self-Correction

What happens when the LLM makes a mistake? In traditional systems, errors bubble up to users. Here’s how autonomous authority changes that dynamic:

[You]: “Let’s add these MVP features: client profiles, task tracking, time logging”

~Claude internally~: Tries to update `solo_entrepreneur.txt`,
gets FileNotFoundError, checks memory folder, finds correct
filename `project_solo_entrepreneur_app.txt`, successfully updates

[Claude]: “Got it! I’ve noted those MVP features for your project.”

Claude made a mistake with the filename, detected the error, diagnosed the issue, and self-corrected. All autonomously. No error messages to the user, no manual intervention required.

Our code deliberately re-raises exceptions to Claude rather than hiding them. This design turns errors into recovery opportunities. Claude receives the FileNotFoundError, understands what went wrong, and adapts its strategy. This resilience emerges naturally from giving the LLM authority rather than rigid rules.

Context-Aware Retrieval

The LLM decides when to check memories. No rules tell it to look for context on greetings, it just understands that’s useful:

[You]: “hello”

~Claude internally~: Checks memory folder, finds and reads
`user_preferences.txt` containing “Vacationing at the beach
as of 2025-11-03”

[Claude]: “Hi! How are things going at the beach today?”

Claude found relevant information and used it naturally in conversation.

These autonomous behaviors all emerge without us programming them explicitly. They include deciding on storage structure, recovering from errors, and retrieving contextual information.

Code vs. Prompts: Where Control Lives

The shift to implicit memory doesn’t mean abandoning all control. It means being strategic about where that control resides. By examining what stays in code versus what moves to prompts, we can understand the architecture of trust in AI systems.

Hard Boundaries in Code

The memory_tool.py file shows what explicit choices we’re still making in code:

Security Boundaries: The _validate_path() method explicitly ensures all operations stay within the /memories directory. This is hard-coded protection against path traversal attacks. It’s a security boundary we enforce in code, not through prompts.

Logging & Tracing: Every operation is explicitly logged for debugging and audit trails. We know every tool call Claude made to the memory system and what was read or written.

These coded constraints are examples of the guardrails we build to create a safe sandbox within which Claude operates autonomously. They’re guarantees enforced by our infrastructure, not suggestions in a prompt. In a production system you will expand on these guardrails until you have an acceptable risk level for your line of business.

The Power of Hooks: Beyond File Persistence

The SDK’s hook-based architecture (BetaAbstractMemoryTool) enables remarkable flexibility. While our implementation uses a filesystem backend, you could implement these same hooks to:

• Store memories in a PostgreSQL database for multi-user applications

• Use Redis for high-performance, distributed memory systems

• Implement vector embeddings for semantic memory retrieval

• Create hybrid systems that combine multiple storage backends

The LLM doesn’t care about the backend. It just calls the hooks and trusts the implementation to handle the details. This separation of concerns enables powerful architectural flexibility while maintaining the same autonomous decision-making paradigm.

Guidance Through Prompts

Between the hard constraints of code and the full autonomy of the LLM lies prompt-based guidance. These are strong suggestions that shape behavior without guarantees:

Keep your memory folder organized. Update existing files rather
than creating duplicates. Include metadata like dates when relevant.

When the memory folder exceeds 20 files, consolidate related
memories into broader topic files. Archive outdated information
with clear labels.

This achieves implicit memory maintenance without explicit commands such as !compact. The LLM monitors its own memory health and performs housekeeping autonomously.

For more structure in the memory files, we could specify a frontmatter format for all new memories. This creates a sort of pseudo-schema to be leveraged by the LLM and our instrumentation code:

All memory files should start with:
---
created: YYYY-MM-DD
relevance: current|archive
---

This gives Claude a framework without removing its autonomy. It can still make contextual decisions within these guidelines.

Making the Implicit Explicit: Explanatory Mode

For demonstration purposes in the companion chat app, I added explanatory versions of the prompts. These make the implicit decision-making visible for learning:

# Directive in the system prompt

IMPORTANT: You are a teaching tool to help the user understand how you
process and utilize memories. After each natural conversational response,
add a concise summary of the memory actions you took and why.

This produces illuminating results:

[You]: “ok, I think I’ll set a goal to finish the outline today, by noon”

[Claude]: “Excellent goal! Finishing the outline by noon today (2025-11-10) gives ...”

---
Memory Actions Taken:
- Updated article status to reflect current goal: completing the outline by noon today (2025-11-10)
- This helps track progress and provides context if we discuss the article later today or in future sessions

This explanatory mode becomes a powerful tool for understanding the LLM’s approach to memory and tuning your system prompts. It transforms the black box into a glass box, revealing the autonomous reasoning. Use it to validate that your prompt guidance is achieving the intended behaviors, then switch back to normal operation once satisfied.

The beauty of this division is that higher risk critical decisions stay in code where they’re guaranteed, while contextual decisions that benefit from intelligence and flexibility live in prompts. As models improve, the prompt-based behaviors get smarter automatically, while our security boundaries remain firm.

Conclusion: Learning from Scale

There’s a principle in AI research that keeps proving itself true. Systems built on general methods and scaled computation consistently outperform those with hand-crafted rules. Rich Sutton calls this “The Bitter Lesson”. Bitter because it means our clever, specialized solutions inevitably lose to simpler approaches that leverage raw intelligence.

The shift from explicit to implicit memory perfectly illustrates this principle. And it reveals what becomes possible when we stop fighting it.

The Power of Delegated Intelligence

We’ve seen the behaviors that emerge: autonomous memory creation, contextual reorganization, self-healing from errors. These weren’t programmed. They emerged from granting the LLM authority within safe boundaries.

This pattern extends beyond memory. It extends to complex decision-making in your application. Routing requests, organizing data, managing workflows. All can potentially be delegated to intelligence rather than encoded in logic. The infrastructure you build becomes a framework for capabilities you haven’t even imagined yet.

Hybrid model: Codify workflows that need precise control as tools, then let the LLM autonomously decide when and how to use them. This gives you explicit control over critical operations while still leveraging the LLM’s decision-making for orchestration.

As models improve, your system automatically gets better. When the next generation releases, you update one parameter. Your existing infrastructure suddenly makes smarter decisions. No refactoring. No new edge cases. The same hooks you implement today become more capable tomorrow.

Trust but Verify: Your Implementation Philosophy

This shift changes how we architect AI systems. Instead of writing decision trees, we adopt a “trust but verify” philosophy:

• Trust: Grant the AI authority through system prompts

• Verify: Monitor the behaviors that emerge

• Guide: Adjust prompts based on observed patterns

• Iterate: Refine boundaries as models improve

We’re reallocating the effort once spent on explicit control logic to validation and evaluation. Same total engineering effort, fundamentally better product. The interesting work moves from implementing specific behaviors to designing systems that exhibit emergent intelligence while maintaining appropriate guardrails.

The Path Forward

But here’s my challenge to you: run an experiment. The gap between what models can do and what we think they can do is often surprising. Many teams discover their explicit controls were solving problems the LLM could handle autonomously (and often better).

You can test this today. The companion app provides a complete learning laboratory:

• Multiple prompting strategies for testing delegation

• Full session tracing of every memory decision

• Visual sequence diagrams exported from conversations

• Real-time observation of memory folder organization

Clone it. Run it. Watch what emerges. Add your own prompts. Test your assumptions. Even if the model isn’t ready for your use case today, you’ll have the framework when the next version drops.

The original memory post showed that LLM memory is just text management. This exploration reveals a deeper pattern: we’re moving from programming behaviors to orchestrating capabilities. The question isn’t whether to trust AI with decisions. It’s understanding which decisions, with what boundaries, and how to monitor the results.

The tools are ready. The models are capable. The only thing standing between you and implicit memory is running that first experiment.

Want to explore further?

• Original approach: simple_llm_memory_poc

• Implicit approach: implicit-memory-system-poc

• Claude Agent SDK: Documentation

• Memory Tool: Official docs

• Anthropic on Agents: Building Effective Agents

🛠️Tool of the Week

Mem0 - Universal memory layer for AI agents

Mem0 is an open-source “memory layer” that sits between your agents and their storage, turning unstructured conversation logs into structured, updatable memories that can be shared across tools, sessions, and even multiple agents.

Highlights:

• Structured, editable memories instead of raw chat logs – Stores facts as atomic memory objects (with metadata like user, tags, timestamps) and supports explicit ADD / UPDATE / DELETE / NOOP operations..

• Backend-agnostic long-term storage – Works with multiple storage types (vector DBs, relational DBs, and other backends) behind a single API, so you can evolve from “just a vector store” to richer hybrid memory without changing your agent code.

• Designed for real agent ecosystems – Supports per-user / per-agent memories, shared memories across agents, and integrations into orchestration frameworks like AG2.

Learn more about Mem0

📎Tech Briefs

• Making Sense of Memory in AI Agents: Explains how AI agents handle memory—what it is, the main types and design patterns, how it’s implemented and managed across context and external storage, and why deciding what to remember, update, and forget is so challenging.

• AI inches toward a more human kind of memory: Describes new research on making AI memory more human-like, focusing on Google’s “Nested Learning” and its prototype model Hope, which spread learning across fast and slow layers so models can update over time, improve long-context recall, and reduce forgetting—though challenges around stability, security, and consistency remain.

• Breaking the Memory Trilemma: How to Build AI Agents That Actually Remember: Introduces the “memory trilemma” of accuracy, cost, and latency in AI agent memory, showing that naive long-context works best only for early interactions and then becomes too slow and expensive, and proposes a hybrid block-based extraction approach.

• AI has a memory problem, just like you do: This podcast episode uses a Trackmania speedrunning AI as a jumping-off point to unpack what “learning” vs “context” really mean in modern AI—covering LLM training vs RAG, short-term vs long-term memory, personalization, security and the dream (and risk) of deeply contextual AI assistants.

• SCUBA: Salesforce Computer Use Benchmark: SCUBA is a realistic benchmark from Salesforce that tests AI “computer-use” agents on 300 real-world Salesforce CRM. Experiments show that current agents, especially open-source and zero-shot ones, struggle badly on these enterprise workflows (often <5% success), while browser-based agents with strong closed models and demonstrations can reach around 40–50% task success, highlighting both how hard real business software is and how much demos can boost agent performance.

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Join us on Dec 13, 2025 · 9am–2pm ET / 6am–11am PT to discover how AG2, autonomous research agents, and real-world multi-agent systems are reshaping science, engineering, and AI-native companies. Get 40% off with code DEEPENG40 and secure your spot at the frontier of agentic AI.

Unlock 40% off – Register with DEEPENG40

✍️From the editor’s desk,

“You don’t need to train models; you need to design systems that can survive their uncertainty.” — Amar Akshat, SVP of Architecture at Paysafe Group

Last Friday, Bank of America told investors it will channel $4 billion of its $13 billion tech budget into AI, with its “Erica” assistant already logging 3 billion client interactions—work the bank says would otherwise require about 11,000 employees. And nearly 47% of US banking decision-makers now say their institutions have fully rolled out generative AI—up from just 10% in 2023

Against that backdrop, today’s issue narrows in on two core questions engineers and architects working in fintech and critical systems are grappling with:

1. How do you put AI in the critical path of payments and wallet flows without giving up determinism, auditability, and regulatory control?

2. What architectural patterns—cells, prompts-as-code, confidence routing, and vendor-neutral orchestration—can scale in production, instead of becoming yet another fragile AI sidecar?

To help answer these, we’ve collaborated with Amar Akshat, SVP of Architecture at Paysafe Group and author of the forthcoming book Decode the Compiler (Packt, 2026). At Paysafe, Amar leads AI-native modernization across payments, wallets, and compliance platforms. Earlier in his career, he helped shape the architectural foundations of Apple Pay and contributed to wallet and tokenization frameworks at Apple. Before that, at Morgan Stanley, he worked deep in build and integration systems and compiler workflows. His work sits exactly at the intersection our readers care about most: compiler-level thinking, distributed systems, and architecture as a living, observable system.

In today’s issue:

• Feature Article – “Agentic Architecture: Designing AI Systems with Guardrails for Trust”: A focused synthesis of Amar’s approach to agentic architecture, including:

  • Cell-based boundaries

  • Treating prompts as code in CI/CD

  • Confidence-based routing and “trust deltas”

  • Prompt manifests as “Dockerfiles for AI”

• Our complete interview with Amar Akshat: You can watch the complete conversation below or read the refined Q&A article here.

Today’s issue is about making AI structural to your architecture while staying inside the constraints that regulators, customers, and your own risk teams will require.

Let’s get started.

AI-Powered Development with Cursor Online Workshop: Ship Production Apps Fast

Nov 29, 2025 · 8:30am–12:30pm ET / 5:30am–9:30am PT · Online. Learn battle-tested Cursor workflows to ship production-grade full-stack apps faster. Get 40% off with code CURSOR40 and unlock 2 free eBooks via this follow-up form after purchase.

Use CURSOR40 – Save 40%

Architecting AI-Native Systems: Agentic Platforms, Guardrails, and Practical Patterns with Amar Akshat

According to Cloudera’s The Evolution of AI: The State of Enterprise AI and Data Architecture report, 96% of enterprises have at least some AI integrated into core processes (and 54% report “significant” integration). In domains like finance and payments, however, turning AI’s newfound capabilities into resilient, auditable, production-grade systems remains a formidable challenge. Architects must reconcile the non-deterministic nature of large models with the deterministic guarantees required for correctness, security, and compliance.

AI in the Architectural Loop: Agentic Systems and Human Judgment

Akshat’s current focus is what he calls:

“agentic architecture” – “designing systems that can reason about themselves.”

At Paysafe, this materializes in internal AI assistants (MCPX and ArchX) embedded into the development lifecycle. These agents codify architectural know-how:

“We have trained internal AI agents to analyze architectural decision records (ADRs) and suggest reusable design patterns, effectively learning from the scars of every project before it,” Akshat says.

Human judgment, however, remains non-negotiable.

“Human judgment still owns context, risk appetite, regulatory nuance, and product trade-offs… The AI can propose, but humans prioritize.”

He offers a rule of thumb:

“If a task mainly benefits from pattern recognition, that’s where putting AI in the loop makes sense. If it instead requires legal, financial, or compliance certainty, we keep it deterministic,” Akshat notes.

Patterns for AI-Infused Architecture: Data, Guardrails, and “Cell” Boundaries

Designing a reliable AI-heavy system starts with clear architectural boundaries and contracts. Akshat outlines three baseline elements his teams define up front:

• Data Postulate: What is the current state of data that the AI can access?

• Guardrail Postulate: What is allowed or not allowed for the AI to do? “Do you want to completely ground the system? Do you want purely deterministic responses or use an existing LLM?,” he says.

• System Prompt Package: The system’s initial prompts and configuration – how the AI is instructed and what output format is expected. “For example, do you want more deterministic performance or do you want more accuracy? More transparency or more speed? These are the kinds of trade-offs that we encode into that system prompt package,” Akshat explains. His team creates a “prompt manifest” for each AI service – “like a Dockerfile for AI,” as Akshat describes it.

With these elements in place, Akshat’s architecture uses a “cell-based” structure to isolate AI components.

“All of our AI deployments are [like] cells – the tiniest unit of life – with their own regulatory nuances within themselves,” he says. “The critical path of analysis never leaves the cell boundary, and it only leaves the cell for audit and storage purposes. That keeps us safe – first of all – fast, and then deterministic.”

Guardrails and Reliability: Building Trust in Non-Deterministic Systems

Even with good design patterns, AI systems can fail in unpredictable ways. Akshat’s approach is to engineer for reliability upfront, with multiple layers of guardrails and fallbacks:

• Confidence-Based Routing: “We route by confidence, really,” Akshat says. If the model’s confidence is below a threshold, the system escalates to a deterministic path or human approval.

• Guardrails and Schema Validation: “Prompt validation and schema enforcement… we use Pydantic plus semantics and sanity checks,” he notes. “A human only steps in for logic failures, not syntax,” Akshat says.

• Continuous Testing for Drift: “Basically, you have to look at AI prompts as code. That’s it,” Akshat says.

Auditability and Traceability

“We log every inference as an ‘architectural replay,’ which includes things like model ID, prompt version, and data snapshot,” says Akshat. Auditability is “incredibly important” in payments because “this is real data you’re dealing with… sometimes real transactions.”

Operationalizing AI Systems: Cost, Vendor Neutrality, and Skills

• Cost Control – “Cache, Batch, Distill”: Akshat’s mantra for cost-efficiency is: “cache, batch, distill.” Caching stores and reuses AI results wherever possible. Batching aggregates requests to process many at once. Distilling means using smaller, cheaper models for simpler tasks.

• Multi-Provider Strategy – Avoiding Lock-In: “We try to stick to the protocols the market is standardizing on – for example, the chat completions APIs and MCP,” Akshat says. His team built MCPX, an internal orchestrator that routes requests to different AI backends – OpenAI, Azure’s hosted models, Anthropic, or their own on-prem LLMs – all through the same interface.

• Skills and Process for the AI Era: In Akshat’s words, “You don’t need to train models; you need to design systems that can survive their uncertainty.” His organization introduced “AI Behavior Reviews” in their design reviews and has identified an “AI SRE” role – someone who can discern if an incident was caused by model drift versus a code bug.

“First principles still apply,” Akshat reminds us. CAP theorem trade-offs, Twelve-Factor app practices, least privilege security – these remain the north star. AI can help augment the architect – even act as a junior designer or reviewer – but it doesn’t replace the need for sound architectural thinking.

🧠Expert Insight

In the full Q&A that follows Akshat walks through MCPX and ArchX, cell-based boundaries, prompts-as-code, confidence routing, and concrete payments and wallet scenarios, showing exactly how his teams operationalize these principles under real regulatory and reliability constraints.

🛠️Tool of the Week

Guardrails: Open-source guardrails for production LLM applications

Guardrails is an open source Python framework for building reliable LLM applications by running configurable input/output guards around models and enforcing structured, validated outputs.

Highlights:

• Production-focused architecture: Ships both a Python library and a standalone Guardrails Server, with first-class docs on deploying the client/server model to production environments (including AWS), so teams can centralize safety policies without redesigning their apps.

• Typed, structured outputs: Uses JSON/Pydantic-style schemas plus a library of reusable “validators” in Guardrails Hub to turn free-form LLM responses into strongly-typed, semantically validated outputs, with automatic correction and redaction when checks fail.

• Actively maintained and ecosystem-friendly: Apache-2.0 licensed, with multiple releases in 2025 (latest v0.6.7) and integrations for LangChain, LlamaIndex, and the OpenAI SDK, making it straightforward to drop into existing LLM stacks.

Learn more about Guardrails

📎Tech Briefs

• Innovation Session | Introducing Microsoft Agent 365: the Control Plane for AI Agents: Focuses on how to onboard, train, and run autonomous AI agents securely at scale, with a strong emphasis on Agent 365 and Microsoft’s emerging “agent estate” model. It walks you through patterns for making agents adaptive to changing business needs while keeping them inside guardrails for data access, regulatory compliance, and threat protection, effectively positioning security/governance as part of the agent development lifecycle.

• Introducing Google Antigravity, a New Era in AI-Assisted Software Development: Google has launched Antigravity, an AI-first IDE where multiple agents can directly operate the editor, terminal, and an embedded browser to write, test, and verify code autonomously. It’s built around Gemini 3 Pro but also supports other models like Claude Sonnet 4.5 and GPT-OSS, and introduces “Artifacts” (task lists, plans, screenshots, browser recordings) as first-class objects to audit and review what agents did.

• GitHub Annotation Toolkit: open-source Figma library for accessible design handoff: GitHub has open-sourced its Annotation Toolkit, a Figma library of structured annotations that lets designers embed intent and accessibility behavior directly into their design files, instead of scattering it across comments and Slack threads. Built by GitHub’s accessibility design team after they found that nearly half of their audit issues could have been prevented with better upfront documentation, the toolkit gives teams reusable “stamps” to describe interactivity, responsive behavior, and WCAG-aligned requirements, and ships with checklists, tutorials, and a GitHub repo for contributions.

• Salesforce eVerse: Simulation environments for enterprise agents:
  Salesforce AI Research has launched eVerse, a simulation environment for training voice and text agents using synthetic data generation, stress-testing, and reinforcement learning before they ever see real customers. The goal is to fix “jagged intelligence” in enterprise agents—brilliant on benchmark tasks but brittle with messy, real-world inputs like noisy calls, frustrated tone, or off-script requests—by running them through lifelike, high-variance scenarios at scale.

• Forbes Tech Council – “How MCP Gateways Can Mitigate Hidden MCP Security Risks”: Suman Sharma, Head of PAM Engineering at Ping Identity argues that MCP gateways (sitting between agents and servers) are becoming essential to defend against malicious or misconfigured MCP servers, focusing on policy enforcement, monitoring, and isolation to contain prompt injection and data-exfiltration risks.

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Build your team’s quantum readiness in just three sessions. Join Robert Loredo and IBM guest experts Travis Scholten and John Buselli to learn how to evaluate quantum bets, design quantum-ready architecture, and secure your org against future quantum threats. Use code QUANTUM30 for 30% off.

Get you pass

✍️From the editor’s desk,

Anthropic just announced it plans to spend $50 billion on custom AI data centers in Texas and New York, aiming to bring them online from 2026 onwards. That sits on top of a Q3 report from TD Cowen showing US hyperscalers leased 7.4 GW of data center capacity in a single quarter—more than in all of 2024, with another ~10.2 GW in the pipeline. But, that much steel and silicon only pays off if the software running on it is fast, predictable, and safe to operate at scale.

So, what does it look like in practice to push more work into the compiler so our systems run faster and fail less often in production? Our feature, Compile-Time Code: Writing Faster, Safer Software Before It Runs, looks at how C++26, Rust, Swift, Zig, and Mojo are each moving computation, code generation, and safety checks into the compile step—using metaprogramming, static analysis, and hardware-aware specialization to trade a bit more build time for significantly better runtime behavior.

We then turn to Ivo Balbaert’s fourth article in our ongoing Building with Mojo series, which walks you through Mojo’s compile-time metaprogramming toolkit—aliases, @parameter loops and conditionals, constrained functions, parametric closures, and parameterized structs—and shows how to use these features.

Balbaert is a lecturer in web programming and databases at CVO Antwerpen and a long-time Packt author of introductions to new languages—including Dart, Julia, Rust, and Red.

Rounding out today’s theme, our Tool of the Week, Pyrefly, shows the same “do more before it runs” philosophy applied to large Python codebases, and this issue’s Tech Briefs track the surrounding ecosystem shifts—from VS Code’s new Agent HQ, LLVM 21.1.5, and NVIDIA’s GTC DC keynote on AI factories and extreme co-design, to Kubernetes 1.35’s release cycle and a recent Pragmatic Engineer conversation with Chris Lattner on Swift, Mojo, and high-performance AI engineering.

Let’s get started!

C++ Memory Management Masterclass (Cohort 2)

Stop chasing C++ memory leaks. In this 2-day live masterclass, Patrice Roy (ISO C++ committee member and author of C++ Memory Management) walks you through RAII, smart pointers, and modern techniques to write fast, safe, leak-free C++ code.

Reserve your seat

Compile-Time Code: Writing Faster, Safer Software Before It Runs with Ivo Balbaert

In the last year, mainstream and emerging languages have pushed more work into the compiler, letting us “write code that writes code” before programs ever run. C++26 has frozen its feature set with static reflection, so C++ code can inspect types at compile time and generate helpers such as enum-to-string mappers or bindings to other languages.

Swift’s macro system lets engineers attach tags to existing code that generate new code at compile time by operating on the abstract syntax tree, and the generated code is fully integrated into the compiler so it is treated exactly like manually written Swift and benefits from the same optimizations. Mojo, a new language for AI systems, goes further by allowing many functions and loops to run at compile time via aliases and @parameter decorators, in a style similar to Zig’s comptime. Across these ecosystems, the goal is the same: shift work into the compiler so the resulting binaries are faster, safer, and more predictable.

Performance Wins: Code Generation Ahead of Time

The most visible payoff is raw speed. In Mojo, any expression bound to an alias is evaluated at compile time, so the final binary just reads a constant instead of recomputing a value. For example, the following Mojo snippet from Balbaert’s article shows how an alias precomputes a sum at compile time, while the same function can still be called at runtime:

alias SUM = sum(10, 20, 2)

fn sum(lb: Int, ub: Int, step: Int) -> Int:
    var total = 0
    for i in range(lb, ub, step):
        total += i
    return total

fn main():
    print(SUM)           # => 70
    print(sum(10, 20, 2))  # => 70

In his recent “impossible optimization,” Evan Ovadia who is part of the Mojo Compiler Team shows a regex engine that parses a pattern and unrolls it entirely during compilation, generating a specialized matcher for each regex node. The compiler emits roughly twenty-eight times more inlined code, but the resulting email-matching regex runs about ten times faster than a conventional implementation and within roughly three percent of a hand-tuned C version . Metaprogramming effectively turns a generic algorithm into bespoke machine code.

Rust and Swift are converging on similar patterns. Rust and Swift are converging on similar patterns. Recent overviews describe Rust’s macro system as “structural” metaprogramming that operates on the abstract syntax tree rather than on raw text, integrating closely with the compiler’s hygiene and type-checking so that macros transform code safely at compile time. Swift macros, meanwhile, are described by Duolingo’s iOS architecture team as tags attached to declarations that operate on the Swift AST and generate additional code during compilation, with the expanded code treated exactly like handwritten Swift and benefiting from the same type checking and optimizations. In all three languages, the compiler effectively becomes a code generator that specializes logic ahead of time instead of relying on runtime indirection.

The trade-off is build cost. In the regex case study, the Mojo compiler had to generate around five times more code and spent noticeably longer compiling to achieve the speed-up. Rust developers report similar pain: the 2025 Rust compiler performance survey highlights procedural macros, deep generic code, and large dependency graphs as key reasons builds feel slow, and the compiler team has prioritized changes to reduce this overhead. Practically, aggressive metaprogramming needs to be reserved for performance-critical hotspots where the runtime gains justify extra compile time.

Safety by Static Analysis: Fewer Bugs at Runtime

Compile-time mechanisms are also being used to strengthen correctness. Rust’s borrow checker uses static analysis to enforce ownership and borrowing rules before the program runs. As Sunil Kumar, Principle Solution Architect at ailoitte explains, each value has a single owner and, when the owner goes out of scope, Rust frees the memory automatically, eliminating classes of bugs such as dangling pointers and double frees. The compiler also enforces that code can have many readers or exactly one writer, but never both at the same time, which prevents data races at compile time. These checks are purely static, so once the code passes, it runs at native speed with no runtime overhead.

Mojo adopts a similar philosophy, combining high-level syntax with Rust-like safety and systems-level control, and leaning on compile-time analysis to drive memory management rather than relying on a garbage collector. Tooling follows the same pattern. Meta’s Pyrefly, a new open-source Python type checker written in Rust, is designed to provide fast, incremental analysis so very large Python codebases can catch errors before runtime while still giving near-instant feedback in the editor. Together, these trends show static analysis moving into the compilation and tooling pipeline, improving safety without sacrificing performance.

One Language, Many Targets

Modern systems increasingly span CPUs, GPUs, and accelerators, and compile-time metaprogramming is becoming a key technique for targeting all of them efficiently. C++26’s static reflection is explicitly described as a building block for generating bindings to other languages and synthesizing code that adapts to platform-specific details at build time. Zig’s comptime, as Aleksey Kladov shows, supports partial evaluation and specialisation: marking a parameter as comptime forces the compiler to evaluate that dimension up front, leaving only the truly dynamic parts for runtime and enabling patterns like specialised serializers without separate code-generation steps. Mojo, built on MLIR and LLVM, is designed so that the same high-level kernel can be compiled into different low-level representations depending on the target CPU or accelerator, with loop unrolling and layout choices made at compile time rather than in hand-written device code. In each case, compile-time facilities let one codebase stretch across architectures while still producing optimized binaries.

Metaprogramming as a Design Tool

Compile-time metaprogramming is moving from niche trick to first-class design tool. Teams no longer have to choose strictly between productivity and performance; with the right language support, they can write generic, high-level code and let the compiler specialize it.

The emerging best practice is to treat compile-time facilities as a scarce, high-leverage resource: use them to pre-compute expensive decisions, encode invariants, and adapt to hardware, while keeping a close eye on build-time cost and complexity. For teams evaluating Mojo alongside Rust, Swift, C++, and Zig: structured macros, strong static analysis, and disciplined compile-time execution can deliver faster, safer systems without degenerating into unmaintainable metaprogramming hacks.

🧠Expert Insight

If you want to move from the architectural view back down into concrete syntax, the following article by walks you through Mojo’s compile-time model step by step—from aliases and comptime execution to parametric functions, decorators like @parameter for and @parameter if, constraints, and closures. It’s a focused tour of the language features behind the trends discussed above, illustrated with small, targeted examples.

🛠️Tool of the Week

Pyrefly: Meta’s high-performance Python type checker

Pyrefly is an open source, Rust-based type checker and language server for Python. It powers lightning-fast IDE feedback and static typing over very large codebases and is intended to replace their previous internal checker in production.

Highlights:

• Scale + speed: Pyrefly can type-check over 1.85 million lines of code per second on Meta’s infra, and the entire Instagram codebase in 13.4 seconds vs 100+ seconds with the previous checker.

• “Invisible” safety: Designed for instant IDE feedback, so teams can push more correctness checks earlier without slowing down developers.

• Gradual adoption: It supports large, partially-typed Python codebases and can infer many types automatically, making it realistic for real-world migration rather than greenfield only.

Learn more about Pyrefly

📎Tech Briefs

• VS Code 1.106—Agent HQ and MCP Security Controls for AI-Heavy Workflows: The October 2025 Visual Studio Code release doubles down on AI-assisted development with a new “Agent HQ” experience that centralizes Copilot and other agents into a single dashboard, plus improved Model Context Protocol (MCP) authentication and governance. Teams can define custom agents, control which MCP servers are allowed in a workspace, and require pre-/post-approval for tools, which matters if you’re letting agents run commands or touch internal systems.

• LLVM 21.1.5—Toolchain Stability for Compilers, Runtimes, and Browsers:

  This is a point update but relevant for anyone shipping code on top of LLVM (including Clang, lld, libc++, MLIR, and a long tail of language runtimes and tools). The release bundles targeted fixes across x86 codegen, sanitizers, linkers, and build infrastructure, and is already being packaged by major distros like Arch Linux.

• NVIDIA GTC Washington, D.C. Keynote with CEO Jensen Huang: NVIDIA’s GTC DC keynote frames AI and accelerated computing as America’s “next Apollo moment,” arguing that GPUs, AI factories, and extreme co-design across chips, systems, software, and data centers are the new essential infrastructure for science, industry, and national competitiveness. Jensen Huang walks through how this model underpins everything from 6G networks and quantum error correction to humanoid robots, autonomous vehicles, and U.S. re-industrialization via NVIDIA’s expanding hardware, software, and partner ecosystem.

• Kubernetes v1.35 Release Cycle—Alpha Builds Landing Ahead of December GA: Kubernetes v1.35 is now deep into its release cycle, with alpha.3 tagged in late October and code freeze scheduled for early November ahead of a planned GA on 17 December 2025. The release team’s schedule highlights ongoing work on enhancements, production-readiness reviews, and test freezes, while the alpha builds are already available via the main kubernetes/kubernetes GitHub releases page.

• From Swift to Mojo and High-Performance AI Engineering (podcast): Gergely Orosz sits down with Chris Lattner (LLVM, Swift, Mojo) to unpack how language and compiler design can lower the barrier to AI development while still hitting systems-level performance. The episode covers why Lattner thinks readability matters more than “AI-native” languages in an LLM world, how Mojo layers Python-like ergonomics on top of features like compressed floating-point formats and compile-time metaprogramming.

  The Pragmatic Engineer

  From Swift to Mojo and high-performance AI Engineering with Chris Lattner

  Stream the latest episode…

  Listen now

  5 months ago · 73 likes · Gergely Orosz

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Just two days left! In 3 high-energy hours, Dr. Bruce Douglass and Brian Moberley fast-track your shift to SysML 2—live demos, toolchain patterns, Q&A, and a certificate. Online this Saturday. I hope to see you there.

Last few seats remaining — book now

✍️From the editor’s desk,

Octoverse 2025 just set a new baseline for how fast teams can move: developers merged ~43.2 million pull requests per month this year (up ~23% year over year), closed a record 5.5 million issues in July, and, for the first time, TypeScript became the most-used language on GitHub—clear signals of tighter feedback loops, heavier automation, and a shift toward typed, AI-assisted workflows.

This issue asks—and answers—four core questions:

1. How do you convert that velocity into governed, repeatable delivery?

2. What are the least-regret guardrails for CI/CD and software supply chain security on GitHub right now?

3. How do you use Copilot and emerging agents responsibly—accelerating output while maintaining review rigor and code quality?

4. Which collaboration patterns—inner source, docs-first habits, rule sets—have the most impact in busy repos?

To find the answers, we sat down with Ayodeji (“Ayo”) Ayodele—Senior Customer Success Architect at GitHub, DevOps evangelist, and Agile coach with 18+ years across Financial Services, Tech, FMCG, Manufacturing, and the Public Sector. Ayo partners with CIOs and engineering leaders across Asia, Oceania, and Africa to translate platform capabilities into outcomes: developer velocity, stronger security posture, and healthier cross-team collaboration. He’s also the author of the GitHub Foundations Certification Guide (Packt, 2025).

In this issue:

• Our complete interview with Ayo on moving from “using Git” to leading with GitHub. You can watch the complete interview below, read the complete Q&A, or listen to/download the audio only version here.

• The complete Chapter 15: “Security Practices and User Management” from Ayodele’s book as an immediately applicable checklist (2FA, roles/teams, scanning, audit logs, incident playbooks).

• Feature Article: Leading with GitHub: Scalable Collaboration, CI/CD, and AI with Ayodeji Ayodele distilling insights from our conversation.

• Tool of the Week: Trivy

• A breakdown of Octoverse 2025: TypeScript takes #1 as AI reshapes the stack

Sponsored:

Securing the Autonomous Enterprise—From Observability to Resilience: Current security stops at passive observation. Rubrik Agent Operations is the enterprise platform that unifies observability, governance, and recoverability for AI. Join us on November 12th to discover how Rubrik is leveraging its leadership in cyber resilience to protect your autonomous future. Save my spot

Leading with GitHub: Scalable Collaboration, CI/CD, and AI with Ayodeji Ayodele

GitHub now sits at the center of modern software delivery. In a conversation with Deep Engineering, Ayodeji Ayodele – a Senior Customer Success Architect at GitHub with nearly two decades in DevOps – shared how to move from simply using Git to truly leading with GitHub. He outlines ways to scale collaboration, automation, and security practices without slowing down delivery.

Open Ways of Working, Inside the Firewall

According to Black Duck’s 2025 Open Source Security and Risk Analysis Report, 97% of applications contain open-source components (averaging 70% of the codebase).

Ayodele emphasizes using GitHub’s collaboration features to break down silos, both in open source and within enterprises.

“Open source is the backbone of modern software,” he notes.

With an estimated 50–60% of all software now built on open-source components, he encourages engineers to contribute to public projects to sharpen their skills. Equally, Ayodele advocates bringing open practices inside companies through inner source.

“Bringing open-source practices inside companies—what we call inner source—breaks down silos and accelerates innovation,” he says.

“Transparency, forking and pulling workflows, and opening discussions all drive better code and teamwork,”

Ayodele explains, adding that a blame-free, well-communicated rollout and strong executive buy-in can overcome cultural resistance. At Microsoft, the DevOps Dojo’s InnerSource program created a single source of truth and drove “IP reuse,” cutting redundancy and enabling collaboration across 36 countries. At IBM, the enterprise InnerSource program—led by IBM Research—explicitly targets open collaboration and reuse at scale across teams.

To make inner source work, teams need clear contribution guidelines, leadership support, and a documentation-first culture. Because any change can face resistance, Ayodele suggests over-communicating plans and having leadership visibly back the inner source initiative. GitHub’s features (Issues, Discussions, project boards, etc.) enable asynchronous teamwork across time zones, but only if context is well documented so colleagues can work without a meeting. By embracing open collaboration patterns internally, organizations unlock their developers’ collective potential.

CI/CD: Shipping Fast Without Pipeline Overload

Ayodele defines CI/CD as:

“a practice and a set of standards that helps you automate all of the very tedious, boring, repetitive tasks that you would ordinarily have to do by hand.”

Done right, pipelines eliminate human error, enforce consistency, and allow teams to “build with confidence” while shipping updates far more frequently. In Ayodele’s view, CI/CD is integral to modern software development – but it must be implemented wisely.

He cautions against letting “pipeline bloat” slow engineers down. Instead, teams should keep pipelines lean and purposeful: break large processes into smaller parallel steps, cache dependencies to avoid needless rebuilds, and introduce automated quality gates at key points.

For example, a GitHub Actions workflow can block a pull request from merging if tests fail or a code scan finds critical vulnerabilities. Automating early and often – while cutting superfluous steps – lets organizations accelerate delivery. With reusable workflows and built-in checks, the release process becomes repeatable and reliable. The payoff is not just speed but confidence that every change is vetted before production. This design prevents one long serial pipeline from blocking everyone’s work.

Security by Design: Protecting the Software Supply Chain

Even as teams move faster and adopt open collaboration practices, Ayodele underscores that security is everyone’s job. Modern software relies on myriad open-source dependencies and automated workflows, so guarding the software supply chain is critical. Fortunately, GitHub provides multiple layers of defense that teams can enable.

“Use the built-in tools you have on GitHub to protect your code at every stage,” he urges.

For example, secret scanning can catch credentials before they leak, while branch protection rules and required reviews prevent risky merges. Enabling Dependabot alerts keeps dependencies up to date, and turning on code scanning flags flaws in the codebase. Ayodele also advises verifying builds: using GitHub Actions to generate SLSA provenance attestations ensures artifacts haven’t been tampered with. He recommends enforcing two-factor authentication for all contributors and mandating signed commits, so every change is traceable to a verified identity. By weaving security into every step, teams can move quickly without compromising on safety.

AI as the New Pair Programmer

Perhaps the most game-changing trend in the GitHub ecosystem is the integration of AI assistance directly into the developer workflow. Ayodele, with 20 years in the industry, says

“I’ve never seen anything like this before”.

Tools like GitHub Copilot have evolved from autocomplete-style helpers to capable “co-developers” that can generate code, explain it, write tests, and even autonomously create pull requests.

The impact on productivity is tangible: GitHub’s Octoverse 2025 report shows that the preview of Copilot’s new coding agent in early 2025 coincided with a significant jump in output – “developers closed 1.4 million more issues [in March] than the prior month”, reaching an all-time high of 5.5 million issues closed in a single month by mid-2025.

And it’s not just quantity of work; quality is getting a boost as well. AI-powered features are catching and fixing common bugs and security issues at scale. For example, Copilot’s “Autofix” feature is now automatically resolving thousands of OWASP Top 10 security flaws in code each month, contributing to that drop in critical alerts. These advances support Ayodele’s observation that teams using AI are seeing higher test coverage and fewer vulnerabilities.

That said, Ayodele and industry surveys alike urge a balanced, responsible adoption of AI. Developers are rapidly embracing these tools. According to StackOverflow’s 2025 Developer Survey, 46% of developers don’t trust the accuracy of AI outputs, and only a tiny fraction (around 3%) “highly trust” the answers from AI. The vast majority prefer to double-check with a human colleague when unsure, and about 75% say they would turn to a co-worker if they don’t fully trust an AI’s answer. Ayodele acknowledges this skepticism:

“It’s difficult to trust what you don’t know… you may need to understand what’s going on under the hood.”

He advocates using AI as a learning tool and accelerant, not a crutch or replacement for human insight. For junior engineers especially, Copilot can help explain code and suggest solutions, but it’s vital that they still practice critical thinking and code review.

Leading into the Future

The role of a software engineer is evolving. Mastering GitHub in 2025 means mastering a blend of technical tools and human skills. On the technical side, that’s proficiency in collaborative workflows, automation pipelines, security practices, and AI-augmented coding. On the human side, it’s creativity, problem-solving, communication, and adaptability. Ayodele says that creativity and empathy remain uniquely human strengths:

AI can help with complex tasks, but AI can’t replace creativity. AI cannot replace empathy. It cannot replace problem-solving. These are innate skills for humans. …So I don’t think AI will replace human beings, you know. So you want to focus on creativity, on communication, on design thinking, and, … adaptability as well.

Engineers should focus on the skills that never go out of style: learning how to learn, and learning how to work with others (including our AI tools). In the next 5–10 years, successful developers and tech leaders will be those who continuously adopt new technologies like Copilot while upholding high standards and mentoring others. As Ayodele puts it,

Armed with a learning mindset and the platform knowledge to take advantage of GitHub’s capabilities, today’s developers can confidently navigate a world where software delivery is faster, more open, and more intelligent than ever.

🧠Expert Insight

1. The complete “Chapter 15: Security Practices and User Management” from the book, GitHub Foundations Certification Guide (Packt, 2025), by Ayodeji Ayodele

2. Deep Engineering’s complete conversation with Ayodeji Ayodele

🛠️Tool of the Week

Trivy (Aqua Security)

A fast, all-in-one open-source scanner that finds vulnerabilities, misconfigurations (IaC), exposed secrets, and generates SBOMs across containers, OS packages, repos, and Kubernetes. Apache-2.0 licensed.

Highlights:

• Coverage in one tool reduces pipeline sprawl (CVE scan, IaC misconfig, secrets, SBOM).

• Low friction to adopt org-wide via reusable workflows in GitHub Actions.

• Proven at scale in a commercial artifact platform, signaling maturity for enterprise use.

Learn more about Trivy

Sponsored:

82% of data breaches happen in the cloud: The reality is you can’t stop every single attack so survival depends on how fast you can recover. Join Rubrik for the Cloud Resilience Summit on December 10th to build true cyber resilience by shifting to an “assume breach” strategy, gain practical, real-world cloud insights, ensure rapid business recovery and minimal financial impact with a cloud restoration strategy. Save my spot

📎Tech Briefs

• Octoverse 2025 lands with fresh benchmarks for engineering leaders: GitHub’s annual report highlights growth (e.g., developer sign-ups, pull requests merged, issues closed) and trends like typed languages and AI—use it to calibrate team throughput and tooling choices. Here is a concise breakdown.

• Node.js 24 enters LTS (“Krypton”): The project promoted v24.x to Long-Term Support on Oct 28, 2025 (v24.11.0), making this the recommended production line through April 2028—plan upgrades and CI matrices accordingly.

• You can now assign code-scanning alerts to Copilot for automated fixes (public preview): From the alert or via security campaigns, Copilot creates a remediation plan and opens a draft PR—tightening MTTR and turning findings into actionable changes.

• Docker Compose patched a high-severity path traversal (CVE-2025-62725): The flaw could allow arbitrary writes on the host when resolving malicious OCI Compose artifacts; upgrade to Docker Compose v2.40.2 or later.

• React 19.2 ships with new primitives and SSR improvements: The Oct 1 release introduces features like <Activity />, useEffectEvent, and cacheSignal, plus partial pre-rendering enhancements—worth evaluating for performance and DX wins.

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

The HubSpot Developer Platform gives you the tools to build, extend, and scale with confidence. Create AI-ready apps, integrations, and workflows faster with a unified platform designed to grow alongside your business.

Start Building Today

✍️From the editor’s desk,

According to Forrester’s 2026 Technology & Security Predictions, one-quarter of CIOs will be asked to bail out business-led AI failures in 2026 and two-thirds will need to justify spend explicitly in business-value terms. BCG’s The Widening AI Value Gap report, on the other hand, finds that only 5% of companies—the “future-built”—are reliably capturing AI value, with agents already delivering ~17% of 2025 AI impact.

So what gaps are the remaining 95% failing to fill? And how can senior technologists bridge these gaps to align IT strategy with business outcomes?

To answer these questions, today we are kicking off the much anticipated IT Strategy Playbook series in collaboration with Scott Millett. He starts the series with a strong claim: “IT strategy is business strategy.” Millett is an IT leader, strategist, and author with 25+ years as CIO, CTO, and architect across large-scale transformations. Articles in the series are adapted from his forthcoming book, The IT Strategy Playbook (Packt, May 2026) and Part 1 introduces his Define → Shape → Deliver loop for closing the gaps between strategy, architecture, and execution. Deep Engineering readers are going to get an exclusive preview of the book’s content in development over the next few months and we invite you to leave constructive feedback as you follow along in the comments.

In today’s issue:

• Editorial Feature: From Outputs to Outcomes: Closing the IT Strategy–Business Gap with Scott Millett

• The IT Strategy Playbook (Part 1): Define, Shape, Deliver by Scott Millett

• Tech Briefs including Forrester’s 2026 predictions, BCG’s report, the 32nd edition of the IBM CEO study, and more

• Tool of the Week: Hugging Face Text Generation Inference (TGI)

Let’s get started.

Quantum Readiness, Now: Executive Masterclass (Nov 29–30)

Quantum Computing for Leaders—2-day executive masterclass, Nov 29–30 (US ET). Build quantum-ready roadmaps, pilot hybrid workflows, and plan quantum-safe security. Live online with Robert Loredo. Early bird 30% off with code QUANTUM30 (ends today).

Reserve your spot →

From Outputs to Outcomes: Closing the IT Strategy–Business Gap with Scott Millett

If you’re a senior technology leader, you’ve likely heard the refrain: “be more strategic” and “align IT with the business.” Yet moving from technological activity to business outcomes is easier said than done. In part 1 of the series adapted from Scott Millett’s upcoming book, The IT Strategy Playbook, a simple boardroom question – “What problem are we trying to solve?” – exposes how even a polished IT plan can fall flat when it’s all outputs and no outcomes.

Four Gaps Between Tech Effort and Business Impact

In his first article in the series, Millett identifies four recurring “gaps” that hinder IT leaders from delivering strategic value. These are not technical failures, but disconnects in understanding and execution:

• The Understanding Gap: When IT initiatives aren’t clearly tied to business goals. This gap is apparent if IT cannot articulate why a tech project matters for competitive advantage. It’s no surprise that when the IBM Institute for Business Value and Oxford Economics surveyed 2,000 CEOs from 33 countries and 24 industries for the IBM CEO study they found that only 25% of AI initiatives have delivered their expected ROI.

• The Translation Gap: The disconnect between high-level strategy and the enabling architecture. It’s one thing to declare a cloud or AI strategy, and another to design coherent systems and data architectures for it. Rushed investments often lead to fragmented systems. In the same study, it was found that half of CEOs admit tech adoptions have left them with disconnected, “piecemeal” technology.

• The Focus Gap: Failing to prioritize for impact. When everything is a priority, nothing delivers value. Too often the IT project portfolio becomes a sprawl of low-value initiatives lacking clear prioritization. This gap leads to IT organizations that are busy but not effective in moving the business needle. In an HBR article, Antonio Nieto-Rodriguez, CEO of Projects & Company, writes: “Many organizations are aware that they are suffering from project overload, slowing execution and obscuring strategic priorities. Yet even with this knowledge they struggle to kill projects to release resources and energy to focus on their most important priorities.”

• The Communication Gap: Inability to communicate strategy in clear, business terms. This amplifies all the other gaps – if executives don’t understand the why and what of IT’s plans, buy-in evaporates. According to Forrester’s 2026 Technology & Security Predictions, CIOs must “speak fluent ‘business value’” to win executive buy-in. IT plans must be explained in terms of tangible business outcomes rather than technical outputs.

These gaps are showing up quite starkly when it comes to enterprise AI adoption. In fact here is what Forrester has predicted regarding AI spend in the coming year:

“With fewer than one-third of decision-makers able to tie the value of AI to their organization’s financial growth, CEOs will lean more on their CFOs to approve AI investments based on their ROI in 2026. As financial rigor slows production deployments and wipes out proofs of concept, enterprises will defer a quarter of their planned AI spend into 2027.”

Additionally , according to BCG’s The Widening AI Value Gap report:

“Future built companies (which make up only 5% of the sample) prioritize AI opportunities through a clear, sequenced roadmap rather than switching on all enterprise applications at once (like the remaining 95%).”

Define → Shape → Deliver: A Playbook for Strategic IT

How can IT leaders close these gaps? Millett proposes a simple but powerful mental model called the DSD Loop – Define, Shape, Deliver. Its philosophy: intent first, design second, implementation last. By defining the business objectives clearly, shaping the architecture accordingly, and then delivering focused initiatives, CTOs/CIOs can ensure every technical choice ties back to a business outcome.

This approach turns the typical “technology-first” mindset on its head. Too often, organizations chase shiny objects – implementing technology for its own sake without a defined problem to solve. The DSD playbook instead begins with the outcome in mind. Each phase informs the next: Define the why and what, Shape the how (technology, teams, architecture), and then Deliver the when through a concrete roadmap.

Interestingly, delivery comes last in this loop. Starting with coding or tool selection before nailing the “why” guarantees a disconnect. The DSD framework provides discipline to link every project (“output”) to a strategic outcome.

Keeping Technology Tied to Business Value

Boards and CEOs now demand clear returns on technology – CIOs are expected to be innovation enablers driving growth, not just IT operators. Technology leadership must translate into tangible business results.

Yet rising scrutiny comes with challenges. Forrester has predicted that in 2026:

“One-quarter of CIOs will be asked to bail out business-led AI failures in their organization.”

When business units bypass IT, CIOs often end up fixing the mess. Avoiding that scenario means jointly defining initiatives from the start and enforcing good governance. As Frederic Giron, VP and senior research director at Forrester said:

In 2026 “Success will be defined not by the sophistication of their (CIOs’) AI models, but by how seamlessly those models are integrated into end-to-end processes to drive both efficiency and growth. The CIOs who thrive will be those who can balance these complex priorities, acting as strategic partners who connect technology investment directly to business outcomes.”

At the moment, According to BCG’s report, only about 5% of companies are “future-built” to fully capture AI’s benefits, while the rest lag behind. Those top performers succeed by tightly aligning tech initiatives to business strategy and continuously refining that alignment.

Bridging the gap from IT output to business outcome isn’t a trivial endeavor – but it’s a core part of the CTO/CIO role. By addressing the understanding, translation, focus, and communication gaps, technology leaders can reorient their teams from doing things right to doing the right things. The DSD loop offers a pragmatic way to ensure that the main thing (business value) indeed stays the main thing. In the end, an IT strategy only matters if it delivers tangible business results.

From here, it’s about turning alignment into action. Scott Millett’s first article kicks off the IT Strategy Playbook series with the Define → Shape → Deliver loop and the concrete artifacts and checkpoints to carry strategy into delivery.

🧠Expert Insight

🛠️Tool of the Week

Hugging Face Text Generation Inference (TGI)

TGI is an open-source, production-grade LLM serving stack (Rust + Python + gRPC) that powers Hugging Face’s own Inference API, Inference Endpoints, and HuggingChat. If you want a battle-tested way to serve open-source LLMs with streaming, batching, tracing, and metrics—without building your own server—it’s a strong default.

Key capabilities:

• Throughput & latency features: continuous batching, tensor parallelism, SSE streaming, FlashAttention, and PagedAttention.

• Ops hooks: OpenTelemetry tracing and Prometheus metrics out of the box.

• Structured outputs & tool use: “Guidance”/JSON-style constrained generation for function calling.

• Multi-backend support: NVIDIA, AMD ROCm, Intel Gaudi, AWS Trainium/Inferentia, and even TensorRT-LLM.

Learn more about Hugging Face TGI

📎Tech Briefs

• Forrester’s 2026 Technology & Security Predictions: Forrester flags five pressure points for tech leaders: a quarter of CIOs will be asked to bail out business-led AI failures; enterprises will push 25% of planned AI spending into 2027; “neoclouds” will seize $20B in genAI revenue, chipping away at hyperscaler dominance; the time to fill developer roles will double amid shifting talent dynamics; and quantum security will command >5% of the overall IT security budget—together signaling bigger budgets, tighter scrutiny, and a premium on translating strategy into measurable business outcomes.

• IBM Institute for Business Value’s “2025 CEO Study: 5 mindshifts to supercharge business growth”: Based on a Q1-2025 survey of 2,000 CEOs across 33 geographies and 24 industries, IBM outlines five leadership “mindshifts” — Make courage your core; Embrace AI-fueled creative destruction; Cultivate a vibrant data environment; Ignore the FOMO, lean into ROI; Borrow the talent you can’t buy — as a roadmap to move from productivity to performance with agentic AI. The study reports that 61% of CEOs say their organizations are actively adopting AI agents and preparing to implement them at scale, and leading CEOs expect AI to have significant to transformational impact on objectives from decision-making and customer experience to productivity and growth.

• BCG’s “The Widening AI Value Gap: Build for the Future 2025” report: BCG’s October-2025 study (n=1,250) finds only 5% of companies—“future-built” firms—are achieving AI value at scale, while 60% report minimal value despite substantial spend; these leaders are widening the gap, delivering 1.7× revenue growth and 1.6× higher EBIT margins than laggards and expecting faster gains as agentic AI rises from 17% of total AI value in 2025 to 29% by 2028; BCG attributes lagging performance to diffuse efforts and weak top-management commitment, urging end-to-end workflow reinvention over scattered pilots and disconnected initiatives.

• CIO.com | “Why CIOs need to master the art of adaptation” by Andrada Fiscutean: With gen-AI reshaping work, the article recommends CIOs should couple adaptability with structured decision criteria (security, ROI, scalability, compliance), favor modular systems, and pilot-then-scale approaches over “fail fast” fads, and build cultures that support experimentation and learning; the piece also cites Foundry’s 2025 State of the CIO showing the role’s shift toward innovation and transformation.

• 🎥IBM Think 2025 Keynote “Usher in the future of work with AI agents built for the enterprise”: Ritika Gunnar, General Manager, Data & AI, Software, IBM, frames AI agents as the next enterprise step—autonomous, proactive, and adaptive “systems of intelligence”—and cites early adoption (agents in >85% of environments this year; >50% expanding into essential production next year) alongside real gains (HR handling 94% of requests, sales reclaiming a quarter of the week, procurement cycle times down ~70%) and real challenges (silos, weak integration, agent sprawl, enterprise “-ilities”).

That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Leave a comment