Guided: Building AI-Augmented Endpoints in ASP.NET Core 10 Web APIs

Table of Contents

1. Challenge

   Integrate IChatClient into an existing Web API

   Welcome to the lab: Building AI-Augmented Endpoints in ASP.NET Core 10 Web APIs.

   In this lab, you’ll extend an existing ASP.NET Core 10 Web API with AI-augmented endpoints that integrate an LLM and retrieve contextual data using vector search. You’ll focus on applying AI patterns to build practical, production-ready AI endpoints. Along the way, you’ll learn how to integrate an LLM using the IChatClient abstraction, implement a basic Retrieval-Augmented Generation (RAG) flow by querying a vector store for contextual data, stream AI-generated responses to clients using TypedResults.ServerSentEvents and expose a server-side capability as an AI tool. First, you'll start with adding a basic endpoint that accepts user prompts and returns AI-generated output. Once finished, you will verify that the endpoint works in the lab environment. You will create a basic endpoint that accepts user prompts and returns AI-generated output using the IChatClient abstraction. info> If you get stuck, you can refer to the provided solution code for each task, available in the solution folder. Congratulations! You created a basic IChatClient endpoint. You learned how to:

   • Add support for AI-generated responses using the IChatClient abstraction.
   • Implement a basic endpoint that accepts user prompts and returns AI-generated output Now you will test the basic IChatClient endpoint that you built:

   1. Inside the Terminal tab type cd src and press Enter.
   2. Inside the Terminal tab type dotnet run and press Enter or click the Run button.
   3. Inside the Web Browser tab navigate to localhost:5000/swagger.
   4. Inside the Web Browser tab, click on the icon with the arrow Open in new browser tab. This will open an external browser tab with the application on your local computer for easier viewing and navigation.
   5. Verify that you see ai/chat HTTP POST endpoint of ProductCatalogApi.
   6. Expand the endpoint view and click the button "Try it Out".
   7. Replace the string in the prompt {"prompt": "string"} with a question, for example "What is a good running shoe?", and click the "Execute" button.
   8. Verify that the endpoint returns a status code 200 with a relevant reply in the response body.
2. Challenge

   Create a Retrieval-Augmented Generation (RAG) endpoint

   In this step you will create a Retrieval-Augmented Generation (RAG) endpoint that uses Large Language Models (LLMs) in your own specific data by providing relevant documents as context before generating a response. This endpoint performs a semantic search against the InMemoryVectorStore to identify documents relevant to the user's prompt. It then constructs a structured request by combining these retrieved documents into a context block and wrapping them in a system instruction that forces the AI to answer using only the provided information. Finally, the code forwards the augmented prompt to IChatClient and returns both the AI-generated answer and the specific source documents to the user. Congratulations! You created a Retrieval-Augmented Generation (RAG) endpoint. You learned how to:

   • Introduce a vector store or simulated semantic search component.
   • Convert user queries into embeddings and retrieve relevant product data.
   • Inject retrieved context into LLM prompts to produce grounded, accurate responses. Now you will test the RAG endpoint that you built:

   1. Inside the Terminal tab type cd src and press Enter.
   2. Inside the Terminal tab type dotnet run and press Enter or click the Run button.
   3. Inside the Web Browser tab navigate to localhost:5000/swagger.
   4. Inside the Web Browser tab, click on the icon with the arrow Open in new browser tab. This will open an external browser tab with the application on your local computer for easier viewing and navigation.
   5. Verify that you see ai/chat/rag HTTP POST endpoint of ProductCatalogApi.
   6. Expand the endpoint view and click the button "Try it Out".
   7. Replace the string in the prompt {"prompt": "string"} with "Which computer has more than 10GB RAM?", and click the "Execute" button.
   8. Verify that the endpoint returns a status code 200 with a response "This laptop includes 16GB RAM and is ideal for developers."

   Note: Replies are augmented by the vector store which is already created in Data\InMemoryVectorStore.cs. You can call the endpoint with a different question. To get a correct reply you need to include at least one keyword from documents inside the vector store.

3. Challenge

   Add a streaming endpoint with Server-Sent Events

   Next, you will create a streaming endpoint with Server-Sent Events (SSE). This endpoint implements a streaming chat response using Server-Sent Events (SSE), which improves the perceived performance and responsiveness of the application. Instead of waiting for the entire response to be generated, the code calls GetStreamingResponseAsync to retrieve the AI's output in incremental chunks as they are produced. By returning these chunks directly to the client via TypedResults.ServerSentEvents, the user interface can display the text in real-time. Congratulations! You created a Streaming endpoint with server-sent events. You learned how to:

   • Create an endpoint that streams AI-generated responses using Server-Sent Events.
   • Use TypedResults.ServerSentEvents to deliver incremental output to clients.
4. Challenge

   Expose a server-side capability as an AI tool

   In this final step, you will expose a server-side capability as an AI tool. This step demonstrates the implementation of AI Function Calling, allowing the Large Language Model to interact with external backend services to satisfy user requests. The code first wraps a local method productService.GetProductsUnderPrice into an AI-compatible tool, which is then provided to IChatClient via ChatOptions. When a user asks a question, the model evaluates whether it needs data from that tool. If it does, it returns a FunctionCallContent request. The endpoint then intercepts this request, validates the extracted arguments, executes the actual product service call, and returns the live data back to the user. This functionality bridges the gap between the AI's natural language understanding and your application's business logic. Now you will test the AI Tool endpoint that you built:

   1. Inside the Terminal tab type cd src and press Enter.
   2. Inside the Terminal tab type dotnet run and press Enter or click the Run button.
   3. Inside the Web Browser tab navigate to localhost:5000/swagger.
   4. Inside the Web Browser tab, click on the icon with the arrow Open in new browser tab. This will open an external browser tab with the application on your local computer for easier viewing and navigation.
   5. Verify that you see ai/chat/tools HTTP POST endpoint of ProductCatalogApi.
   6. Expand the endpoint view and click the button "Try it Out".
   7. Replace the string in the prompt {"prompt": "string"} with "Show me products under $50", and click the "Execute" button.
   8. Verify that the endpoint returns a status code 200 with a response "toolUsed": "get_products_under_price", "result": [ "Budget Headphones - $25" ]"
   9. Replace the string in the prompt {"prompt": "string"} with "What is the capital of the United States?", and click the "Execute" button.
   10. Verify that the endpoint returns a status code 200 with a response "The capital of the United States is Washington, D.C."

   Note: The prompt in #7 was a question specific to ProductCatalogApi so AI determined that it should use tool get_products_under_price and return a matching product. The prompt in #9 was a general question so AI determined that it should not use the tool and instead answer the question directly.

   By completing this step you learned how to:

   • Define an API-backed tool.
   • Allow the LLM to invoke this tool via structured function calls.

   Congratulations! You completed Building AI-Augmented Endpoints in ASP.NET Core 10 Web APIs.

   By completing this lab you developed the following skills:

   1. Integrate an LLM into ASP.NET Core 10 Web API using the IChatClient abstraction
   2. Implement a Retrieval-Augmented Generation (RAG) flow by querying a vector store for contextual data
   3. Stream AI-generated responses to clients using ServerSentEvents
   4. Expose a server-side capability as an AI tool with validation