Skip to content

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.

What is Spring AI, and how to use it: A beginner's guide

SpringAI is a powerful tool that helps Java application developers work with LLMs in a platform-agnostic way. Here's how to start using it.

Feb 20, 2024 • 6 Minute Read

Please set an alt value for this image...
  • Software Development
  • AI & Machine Learning

Applications everywhere are investigating how artificial intelligence can enhance their application. In this article, you’ll learn about a new Spring project that affords Java developers a convenient API abstraction for working with large-language models (LLMs) like ChatGPT. By the end, we’ll have a simple program that can answer questions about the Swiftie Bowl based on information retrieved from Wikipedia.

The Origin of Spring AI

Spring AI --- a Spring project, not quite GA as of this writing --- represents a groundbreaking initiative aimed at simplifying the integration of artificial intelligence features into applications across the JVM family of languages. It was introduced officially to the world in the fall of 2023 at SpringOne by Mark Pollack, its creator. In true Spring fashion, the project introduces a common Spring-like pane of glass across the disparate LLM APIs cropping up, among them OpenAI and Azure OpenAI.

While it is in the experimental phase, it’s not considered production-ready; however, it is certainly worth a look so you can get a jump on learning the kinds of problems AI can help you solve.

AI Concepts

Before creating our sample application, let’s go over a few basic terms:

  • Models: AI models are algorithms that mimic human cognitive functions to generate predictions, text, images, or other outputs from large datasets.

  • Prompts: Prompts are language-based inputs that guide AI models to produce specific outputs; often this requires skillful crafting (sometimes called Prompt Engineering) to be effective.

  • Prompt Templates: These templates use text-based engines to create prompts by substituting parts of the request with user-specific values.

  • Embeddings: Embeddings transform text into numerical vectors, allowing AI models to process and understand language.

  • Tokens: Tokens are the basic units of language processed by AI models, with usage directly linked to the cost of AI services.

We’ll refer to each of these concepts while creating the application.

Initializing a Spring AI Application

Spring AI is so new that it isn’t part of the Spring Intializr. While we could use to create a skeleton and then manually add the starters afterward, I’m going to use the Spring CLI which already has Spring AI support.

Once you’ve installed the CLI, navigate to a directory where you want the application to be, start the Spring Shell, and run:

      spring:> boot new spring-ai-solar-eclipse ai
Getting project from
Created project in directory 'spring-ai-solar-eclipse'
spring:> exit

This creates a Hello World project for interacting with ChatGPT.

Then, take your ChatGPT API key and export it like so:

      export SPRING_AI_OPENAI_API_KEY=your-api-key

Finally, run the application using mvnw:

      ./mvnw spring-boot:run

Then you can send it a command like “Write a haiku”:

      http :8080/ai/simple?message=”Please write me a haiku”

and see ChatGPT’s response:

	"completion": "Beneath moon's soft glow,\nWhispers of the gentle breeze,\nNature's calm, bestowed."


Inspecting the Code

Now that we’ve seen how to run the application, let’s dive into some of the details of the application to learn about Spring AI’s API.

In SimpleAiController, you can see AiClient as an injected dependency:

public class SimpleAiController {

    private final ChatClient chatClient;

    public SimpleAiController(ChatClient chatClient) {
        this.chatClient = chatClient;



Similar to other XXXClient classes in the Spring portfolio like WebClient and LdapClient, ChatClient represents the interface by which you’ll interact with any LLM.

Its simplest usage is similar to what you may have already experienced when using the ChatGPT web interface; you give it a string and it gives one back. You can see the above controller doing just that in its one method:

public Completion completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
    return new Completion(;


So, when we sent “Please write me a haiku”, Spring received that, sent it to this rest controller which subsequently transmitted it to OpenAI via its AIClient implementation.

But, because the model is trained only periodically on recent content, it doesn’t know the answer to recent questions like the following:

      http :8080/ai/simple?”Who won the Swiftie Bowl? If you don’t know, just say ‘I don’t know’”

It will give the simple answer:

	"completion": "I don’t know"


Can Spring AI help us with this? Let’s take a look.

Stuffing Prompt with Additional Information

One technique as far as effective prompting is stuffing it with information in addition to the question or message that the user is sending.

At the most basic (and often inefficient) level, this looks like giving OpenAI the text of an article along with a question like so:

      ?message=”Based on this article: ‘{text of the article}’, please answer me this question: {your question}”

The OpenAI API can handle this just fine, but as I said, it’s inefficient since OpenAI charges you by the token. Since it’s quite likely that the answer to your question exists in only a portion of that article, sending fewer parts will result in you spending less money to get the same answer.

Pulling Relevant Information from Wikipedia

If we download the latest Wikipedia article about the Super Bowl, it should help ChatGPT to be able to answer our question.

Place it in the src/main/resources directory of the project.

Storing in a Vector Database

Next, you’ll load the contents of that pdf into a VectorStore. You can use a local store like Redis, but for the purposes of this intro article, you’ll use SimpleVectorStore, which is an in-memory store.

NOTE: This demo uses the `OpenAIEmbeddingClient` by default, using your OpenAI credits. See the documentation for other embedding clients that run locally. Additionally, since we are using `SimpleVectorStore`, the embedding happens again at startup each time. For a production application, you would certainly want to store these vectors somewhere outside of the application.

Since we’re working with a PDF, add the following dependency to your project:



This adds the needed APIs for parsing the PDF file.

Then in Application, publish the vector store, including the PDF as its contents:

VectorStore vectors(EmbeddingClient embedding, @Value("Super_Bowl_LVIII.pdf") Resource pdf) {
    	SimpleVectorStore vectors = new SimpleVectorStore(embedding);
    	var reader = new PagePdfDocumentReader(pdf);
    	var splitter = new TokenTextSplitter();
    	var documents = splitter.apply(reader.get());
    	return vectors;


What this does is split up the document into smaller chunks and then store them locally as a set of vectors. In the next step, we’ll have Spring AI do some math to figure out which of those document chunks to send to OpenAI as part of our question.

Then, let’s update SimpleAiController to also be dependent on the VectorStore like so:

      private final VectorStore vectors;

public SimpleAiController(ChatClient chat, VectorStore vectors) { = chat;
    this.vectors = vectors;


Finding Relevant Information to Include

Now we’re ready to add a new endpoint to SimpleAiController. This time, it will have two parameters, one for the stuffed prompt and one for the message:

public Completion completion(
    @RequestParam(value = "prompt", defaultValue = "Based on the following: {documents}") String prompt,
    @RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
    // … content  to follow


But instead of simply calling like the first time, we’re going to:

  1. Search the vector store for relevant chunks, and
  2. Construct a prompt that includes those chunks

You can search the vector store by the user’s message like this:

      var documents = this.vectors.similaritySearch(message);
var inlined =;


Then you can send both those documents and the user’s message so OpenAI has more context to work with:

      var system = new SystemPromptTemplate(prompt).createMessage(Map.of("documents", inlined));
var user = new UserMessage(message);
return new Completion( Prompt(List.of(system, user))).getResult().getOutput().getContent());


Trying It Out

With that endpoint in place, start up the application and try it out!

      http :8080/ai/stuffed?message=”Who won the Swiftie Bowl? If you don’t know, just say ‘I don’t know’”

	“completion”:  "The Kansas City Chiefs won the Swiftie Bowl, defeating the San Francisco 49ers with a final score of 17-10."



Spring AI is a powerful tool for working with LLMs in a platform-agnostic way. Today, you caught a glimpse of what the ChatClient and VectorStore APIs are capable of, allowing you in a few keystrokes to create a chatbot that references a set of data that you provide.

Choosing Spring AI allows you to work within the Spring programming model. Looking ahead, choosing something platform-agnostic like Spring AI allows you either to switch platforms or use a multiplicity of platforms without changing the way you interface with them.

Other learning resources

Other learning resources

Want to learn more about using the Spring Framework? Pluralsight offers a wide range of learning paths that can help you become a master of all things Spring. Here's some you should check out:

Josh Cummings

Josh C.

Like many software craftsmen, Josh eats, sleeps, and dreams in code. He codes for fun, and his kids code for fun! Right now, Josh works as a full-time committer on Spring Security and loves every minute. Application Security holds a special place in his heart, a place diametrically opposed to and cosmically distant from his unending hatred for checked exceptions.

More about this author