ChatGPT AI Agents

Table of Contents

1. Challenge

   Set up the AI API

   Start by opening up ChatGPT_AI_Agents.ipynb. Then the first and only configuration you will have to change is swapping out the <YOUR API KEY HERE> with the API Key generated at the top center of your screen. The AZURE_OPENAI_API_KEY variable is the variable you are meant to define.

   In the same cell is a small call to GPT to ensure you can connect to the API properly, the response should take no more than about 100 of the tokens needed for the lab.

2. Challenge

   RTCF system prompt

   RTCF or Role, Task, Context, Format, is a structure of system prompt used for agents to ensure they understand:

   • R — Role Defines who the agent is (its identity, expertise, and behavior style)

   This is specifically important to help reduce the scope of knowledge for LLMs down to the proper subset, since something like finance, which is broad and can be vastly different depending on context, still requires precision.

   • T — Task Defines what the agent must accomplish

   This can include metrics or specified needs in the returned text from the agent

   • C — Context Provides relevant information, constraints, or background

   This further reduces the scope of knowledge for the model making sure it works within the correct domain but also can provide relevant context the model may be unaware of if not within the training dataset.

   • F — Format Specifies how the output should be structured

   Structures the return to ensure it's not difficult to understand how results were achieved. Your RTCF has already been defined, if you wanted to change it to see how different system prompts affect the agents abilities, feel free to do so.

   The prompt in this case was left intentionally vague, yet the format is important. The format for the prompt is a list of expected aspects of the prompt, as opposed to a particular metric optimization, with focus on explainability. AI agents suffer from hallucinations, misclassifications, and other common problems that normal LLMs have. Due to the unreliability and non deterministic nature of agents it is highly recommended to always include information such as tool usage or logic summaries to help trace models decision making.

3. Challenge

   Local tools

   Tools the agents can use consist of two main types, functions and knowledge bases. Not all models need both but a model with just a knowledge base will share similar functionality of a RAG, with far less functionality.

   Functions allow the model to actually execute code to achieve something, like draw real time data of inventory stock, or in your case use a calculator.

   Whereas knowledge bases contain specific index information that the agent may need to solve the scope of its tasks. This lab will be using a very basic calculator function. This function will take strings to express different mathematical functions and apply them onto the number included. It is important that the function can interact and return strings as that is primarily how agents communicate.

   For the knowledge base, you will have a small dictionary with a few key knowledge foundations the agent needs to understand the tasks you will declare farther along. The knowledge base is a dictionary in this case due to the fast lookup time of the information. You also have a kb_lookup which is the function the model will use to query the knowledge base to find information it needs for specific tasks.

   Azure OpenAI tool schemas

   The tool schemas depend on the platform being used and are primarily json. In this case the format adheres to Azure Open AI schema requirements including key features such as type to specify the tool is a function that could be run and parameters to specify the input requirements needed to run the function.

   Bonus: The model does not actually run the code, and the code does not have to be python. The kernel in your notebook runs the functions allowing you to isolate and version different libraries and requirements.

4. Challenge

   Agent loop

   The agent loop is simply the loop the agent will follow until solving the task given to it. The format of this loop is fairly basic and likely wouldn't change between agents whereas the underlying tools and prompts would.

   The entire agent loop progresses as follows:

   1. The agent is given the RTCF prompt and the user task.
   2. The agent generates a response, and the response is checked if there is a request for a tool.
   3. If there is no request for a tool, the response is returned otherwise the tool requested in run.
   4. The model is returned the results from the tool, where the parameters for the tool are defined by the AI model.
   5. The loop repeats from step 2.

   In this way the model is allowed to loop as long as needed to generate a good response. Additional features such as metric requirements before responses are returned or a maximum amount of loops allowed are possible to integrate as well.

5. Challenge

   Agent tasks

   The agent tasks are quite simple, you provide the agent with the task, in this case they are provided as a list for the agent to roll through but would likely just be called as needed in production. The agent simply takes a prompt to try to solve the task and will use tools provided to it before returning the response in the format you specified above.

   Feel free to read through the responses and see how changing the format aspect of the RTCF changes the agent's responses. You can also uncomment a print statement within the agent loop to see when tools are being called.