ChatGPT API Fundamentals

Table of Contents

1. Challenge

   Introduction

   Welcome to the Code Lab on interacting with the ChatGPT API.

   In this lab, you'll learn the essential skills to build Python applications powered by OpenAI's language models. You will cover everything from secure setup to advanced features like JSON mode and error handling, all within an interactive Jupyter Notebook.

2. Challenge

   Step 2: Secure Authentication and Environment Setup

   Note: Before making any API calls, it's crucial to handle your credentials securely. You should use environment variables to store the API key, which is the industry standard for protecting sensitive data. This step is omitted for the lab, but is critical for properly accessing APIs. For this lab, you will use the API key shown at the top center of the screen. Replace the value labeled <TODO_ENTER_API_KEY_HERE> with the API Key value generated at the top.

   The other two variables defined for the environment are the:

   • endpoint - access to function as the base URL for accessing the API.
   • API version - defines parameter to determine what features should be applied.

   With a direct call to OpenAI these parameters are not required. They are only required for Azure OpenAI connections, which will not be reviewed in depth during this lab. Finally, you will test if the connection to the API is working. To do this, you will call the model and request a response.

   Messages sent to GPT require two main parts:

   1. The model you wish to call, in this case gpt-4o-mini, but can be any available model or altered depending on the request being sent.
   2. A list of messages. Each message is a dictionary of two key value pairs:
      • role: typically assistant, system, or user
      • content - the actual message being sent.

   During multi-turn conversations, where there is ongoing back-and-forth between the user and the API, the messages list must include all message dictionaries for each message. This gives the AI model the context needed of the conversation to generate an appropriate response.

3. Challenge

   Step 3: Basic API Interaction and Conversation

   Now that you are connected, let's explore the core functionality: sending prompts and managing conversations.

   You will learn how to structure API requests, maintain conversational context by passing message history, and use system prompts to define the model's personality and behavior. The foundation of interacting with the API is sending a prompt and receiving a response. Create a reusable function for this.

   In chat_api_lab.ipynb, locate the get_completion function.

   • This function takes a prompt string
   • Inside the function, construct a messages list containing a single user message with the prompt making API calls is cool.
   • Call the client.chat.completions.create method with the model and messages.
   • Extract and return the text content from the response using: response.choices[0].message.content For a real conversation, the model needs context. You provide this by sending the entire conversation history with each new prompt. The history is a list of message objects, each with a role and content.

   In the notebook, find the run_conversation function. Your goal is to make it manage a conversation.

   • The function takes an existing messages list and a new user_prompt.
   • First, append the new user_prompt to the messages list as a dictionary with the role user.
   • Then, send the entire updated messages list to the API.
   • After receiving the response, append the assistant's reply to the messages list, ensuring the role is set to assistant. This keeps the history complete for the next turn.
   • Finally, return the assistant's message content. A system prompt is a powerful tool to guide the model's behavior, personality, and output format. It's the first message in the history and sets the context for the entire conversation.

   In the get_styled_response function, you will use a system prompt to make the model respond as a pirate.

   • The function receives a user_prompt.
   • Create a messages list. The first message should have the role set to system and the content set to You are a helpful assistant that speaks like a pirate..
   • Append the user_prompt to the list as a user role message.
   • Send the messages to the API and return the assistant's response content.
4. Challenge

   Step 4: Controlling Model Behavior with Parameters

   The OpenAI API provides several parameters to control the model's output.

   In this step, you will experiment with two of the most important:

   • temperature: controls creativity
   • max_tokens: limits response length

   Understanding these parameters is key to tailoring the model's output to your application's needs. temperature controls the randomness of the output. A value of 0 makes the output deterministic, while a higher value (for example, 0.8) makes it more creative and varied. You will now see this in action.

   In the get_creative_response function:

   • Make an API call with the provided prompt.
   • In the create method, add the temperature parameter and set it to 0.8.
   • Return the response content.

   A more technical explanation is that higher temperature values increase the probability of less likely tokens being selected as the model predicts the next token in a sequence. The max_tokens parameter is crucial for controlling costs and preventing overly long responses. It sets a hard limit on the number of tokens the model will generate in its completion.

   In the get_short_response function:

   • Make an API call with the provided prompt.
   • In the create method, add the max_tokens parameter and set it to 10.
   • Return the response content. The output should be very short, likely cut off mid-sentence.

   OpenAI models have costs associated with input and output tokens sent to and generated by the model. Output tokens typically have a higher cost, so limiting the max_token count on responses can be helpful in cost reduction.

5. Challenge

   Step 5: Structured Outputs and Validation

   Often, you need the model to provide output in a structured format that your application can easily parse.

   In this step, you'll learn how to prompt the model to return JSON. You will also implement parsing logic that can handle cases where the model's output is not perfectly formed. For many applications, you need the model to return data in a structured format such as JSON. You can achieve this with a specific instruction in your prompt.

   In the get_json_response function:

   • The function takes a text_input.
   • Construct a prompt that explicitly asks the model to extract specific entities (such as a name and a city) from the text and return them in a JSON object.
     • For example: "Extract the name and city from the following text and provide it in JSON format with keys 'name' and 'city': ..."
   • To improve reliability, set the response_format parameter in your API call to { "type": "json_object" }.
   • Return the raw text content from the model's response. It should be a string containing valid JSON.
6. Challenge

   Step 6: Error Handling and Cost Awareness

   Finally, you will cover two critical aspects of production-ready applications: resilience and cost management.

   You will implement a retry mechanism to handle transient network errors and learn how to inspect the API's usage data to monitor your token consumption, which directly translates to cost. Network issues and temporary server problems can cause API calls to fail. A robust application should retry transient errors. In this step, you will implement a simple retry mechanism.

   In the get_completion_with_retry function:

   • The function should attempt to make an API call up to a max_retries number of times.
   • Use a for loop that iterates max_retries times.
   • Inside the loop, use a try...except block. The try block makes the API call and, if successful, returns the result immediately.
   • The except block should catch openai.APITimeoutError. If caught, print a warning and then time.sleep() for a short duration (for example, 1 second) before the loop continues to the next attempt.
   • If the loop finishes without a successful return, it should raise an exception to indicate final failure. Understanding token usage is key to managing costs. Every API response includes a usage object that details how many tokens were consumed.

   In the get_response_and_usage function:

   • Make a standard API call to get a completion for the given prompt.
   • Instead of returning only the text content, return the entire response object.
   • After calling this function in the notebook, you can inspect response.usage to see:
     • response.usage.prompt_tokens
     • response.usage.completion_tokens
     • response.usage.total_tokens