Architecting microservices for seamless agentic AI integration

If you’re looking to incorporate Agentic AI into your software, the first step in your journey should be to examine the foundation on which it will be built: your architecture.

In this article, I’ll break down how to build a solid foundation for adopting Agentic AI by leveraging microservices and event-driven design, and why this is so important. I’ll explain foundational concepts of Agentic AI, the shortcomings of traditional software architecture when adopting it, provide event-driven architecture examples, and how to prepare your architecture for adoption.

Foundational concepts of AI: LLMs & Agentic AI

Before diving into frameworks, let’s cover the fundamentals. If you’re already familiar with both of these concepts, feel free to skip ahead.

What are LLMs?

The rapid progress of AI systems over the past few years has been driven by advancements in hardware and software, and more specifically, the key component known as the Large Language Model (LLM).

With modern hardware and software, we can now train LLMs on massive datasets—a process known as model training. The resulting neural networks compress and reorganize information into patterns and connections, loosely inspired by the structure of the human brain.

These neural networks enable models to interpret language and generate new content—such as text, images, audio, or video—based on learned patterns and context.

All of this progress has led to the most familiar use case: prompt-based LLMs. Users interact through a browser-based interface by entering text instructions that specify the desired outcome—such as summarizing an article, drafting an email, or generating an image or video.

Typical usage of LLMs using a web browser. A user provides a prompt that describes the desired outcome, and the LLM generates a corresponding output.

What is Agentic AI?

Agentic AI takes things a step further than making a single human-driven request and receiving a single output. Instead, Agentic AI is a type of AI system designed to act autonomously to take actions to achieve specific goals with limited supervision. An example might be an AI agent designed to detect and respond to cybersecurity threats, or to diagnose and troubleshoot system anomalies without waiting for human triage. 

Agentic AI typically uses an LLM as the core enabler of agentic behavior, using it to identify context, turn goals into subtasks, make decisions, and generate human-like responses. 

The basics of adding agentic AI to software architecture

Integrating agentic AI into software architecture means enabling LLM-driven agents to perform tasks and actions in response to system events—using the same pattern-recognition and generative capabilities that power text, image, and video outputs.

Your typical software architecture. The question is how to integrate A.I., i.e., LLM, into it.

As shown above, the text prompt is replaced by events, actions, and live data from the software architecture. The output may involve creating or updating records, calling functions, generating reports, or triggering processes—each tailored to the system's specific context.

Integrating agentic AI into your software architecture can enable autonomous decision-making, improve operational efficiency, enhance scalability, and support real-time responses to events.

However, there’s often a significant barrier in the path of this adoption, and that’s your legacy software architecture.

Why traditional architecture is a bad fit for agentic AI adoption

Traditional software architecture typically includes a front-end application and a monolithic backend—comprising a backend API and a huge database. This works for a predictable, restricted front-end experience with user-triggered actions, but not for an AI that mimics reasoning and thinking to solve problems and carry out actions. 

To support agentic AI, the architecture must give the LLM access to the right data, sufficient performance, and proper guardrails—enabling it to perform tasks safely and effectively. Below is an example of how traditional software architecture fails to do this.

In Figure 1, the backend API exposes specific data and functionality from the monolith via externally accessible HTTP endpoints. However, much of the monolith remains inaccessible—buried functionality and data that the LLM can’t reach. This limits the agent’s context for reasoning and restricts its ability to execute meaningful actions or processes.

In Figure 2, performance-sensitive components—such as a monolithic database handling live transactions—are exposed to the AI agent. An agent that can mimic thinking and reasoning to solve a problem, but how considerate will the agent be when accessing sensitive parts of your software architecture, like the database?

The good news is that Agentic AI isn't as crude as simply placing an LLM in the middle of your software architecture. However, two things are needed for us to incorporate agentic AI:

• Architecture that can deliver clear instructions, short-term and long-term context, and a straightforward interface for the AI agent to interact with it, and;

• An LLM that has been designed to function as an AI agent.

How to turn a normal LLM into an AI agent

Here’s a simplified example of how an LLM might operate as an AI agent. While implementations vary by platform, this model assumes a microservices-based architecture, which we'll cover in more detail later.

In this setup, a microservice receives input from the broader software system—acting as the trigger for the AI agent. To improve context for reasoning, the agent draws on data from short-term and long-term memory microservices. This information is used to generate a structured set of instructions—effectively mimicking a prompt that the LLM can understand and process.

During reasoning, the LLM may continue referencing memory services to refine its understanding and break the goal into a list of tasks.

Once the LLM is ready to act, it interacts with the Outgoing Interface microservice. This microservice enables the agent to retrieve additional data or perform tasks within the software architecture. In both cases, the microservice provides guardrails to prevent undesirable interrogation of the broader system.

Supporting an AI agent like this with microservices is a natural fit due to the performance requirements during inference and reasoning. Specific services can be scaled out accordingly and optimised for either reading, writing or searching, whatever the LLM needs for quick and efficient reasoning and inference.  

Setting up your software architecture for agentic AI systems

Multiple AI agents working efficiently and optimally in a microservices software system.

Just as we use microservices to support our LLMs, the exact principles should apply to the rest of the software architecture. Breaking up the monolith into microservices makes data and functionality accessible through official communication contracts—such as HTTP endpoints (Figure 1), message processing from a queue (Figure 2), or processing files from a file share. We can also increase performance as needed by scaling out individual microservices.

All of this makes the architecture far more accommodating for AI agents. For example, through event-driven communication, a message placed on a queue can trigger an AI agent without the sender needing to know who or how many components are listening or how they’ll respond. This decoupling enables scalable, autonomous behavior, as multiple agents can subscribe to and act on specific events.

Example: Using event-driven architecture with agentic AI for sales

In this scenario, your microservices system emits a cancel order event. An AI agent subscribed to this event examines the user's interface activity leading up to the cancellation. Using the LLM’s ability to interpret language and sentiment, the agent inferred that the user cancelled after reading negative reviews.

In response, the AI agent:

• Composes an email with positive reviews to counterbalance the negative ones

• Adds promotional codes to the email

• Emails the user with this extra information

Unlike traditional rule-based systems, the AI agent uses language understanding and sentiment analysis to respond to nuanced, unpredictable user behavior. Its ability to act this way depends on microservices design patterns that support event-driven and autonomous workflows.

But the proactive AI agent doesn't stop there. When it detects a new order event from the same customer matching the original cancellation, it records the successful outcome of its tactic to its long-term memory via the long-term memory microservice. Over time, this feedback loop allows the agent to refine and improve its sales tactics for cancelled orders to win back customers. 

Conclusion: Build the right foundation for Agentic AI, first

More than 80% of AI projects fail, typically due to lack of proper preparation. Yes, it may be daunting to divert from your AI ambitions to first create a microservices-based system. But the truth is, it’s not a diversion: it’s the first step on your Agentic AI journey. Without it, your AI Agents won’t have the functionality and data they need to achieve their goals—and by extension, yours.

Make sure you’re familiar with microservices design patterns, such as event-driven communication and asynchronous communication, and then come up with a strategy to implement it. Remember, it’s always easier to build a strong foundation before you put a house on top of it. 

Resources for learning design patterns and transitioning your infrastructure

Microservices design patterns

In this article, we covered event-driven communication, but this is just one of many microservice design patterns that can help you effectively adopt agentic AI. Rather than cover them all in the scope of this article, I’d highly recommend checking out my course: "Microservices Architectural Design Patterns Playbook.” In it, I dive into all these design patterns in far more detail.

It’s impossible to cover all of this in one article, which is why it’s best to check out the course! 

If you’re completely new to microservices in general, don’t worry: I’ve also got a course for that. It’s called “Microservices: The Design Principles,” and it teaches you microservice fundamentals from scratch. 

Transitioning your legacy software architecture

If you’re reading this article, you’re likely in a situation where you need to transition your existing software architecture to a microservices-based approach. Check out my course on “Microservices Transition Strategies and Techniques.” In it, I cover how you can transition from a monolithic software architecture to microservices incrementally at your own pace, while still carrying out day-to-day new development and bug fixes. 

One of the methods I cover is how to use the Strangler application pattern to transition to a microservices-based system.