GitHub Copilot and the Agentic Loop

Table of Contents

1. Challenge

   Introduction

   In this lab, you will use the GitHub Copilot CLI to improve a simple Python shopping cart application. The application allows customers to add items, calculate totals, and apply discounts, but contains some errors. You will use Copilot to identify these issues, implement corrections, generate tests, and verify the results. Run the application to observe its initial behavior.

   1. In the Terminal, run

      python3 shopping_cart.py
      

   The expected cart total is $1050, with a discounted total of $945 after applying a 10% discount. Compare these expected values against the application's output; they do not match.

2. Challenge

   Launch the GitHub Copilot CLI

   Copilot CLI can inspect files, modify code, and execute commands to complete development tasks. At the >  Copilot prompt, you will now enter your first prompt.

   1. Enter the following prompt into the Copilot CLI session:

      Review shopping_cart.py and improve its reliability.
      
      Your goals:
      1. Identify and fix errors in cart total calculations.
      2. Ensure percentage discounts are calculated correctly.
      3. Add validation for negative prices, invalid quantities,
         and discounts outside the 0-100% range.
      4. Generate unit tests for the shopping cart functionality.
      5. Run the tests and verify the results.
      6. If tests fail, investigate and correct the implementation.
      
      Explain your plan before making changes.
      Summarize the changes and test results when finished.
      

   2. Select the Yes options when prompted.
      • Do this when prompted later on, too, unless otherwise specified.
      • Outside of this lab, you'd want to more carefully consider what you would allow Copilot to do.

   Observe how Copilot approaches the task. It should inspect the existing application, identify calculation errors, modify the code, generate tests, and verify its implementation.

3. Challenge

   Copilot agentic workflow

   As Copilot executes, identify each stage of the agentic development process.

   • Plan: Copilot identifies calculation errors and determines necessary changes.
   • Act: Copilot modifies the shopping cart and generates tests.
   • Verify: Copilot executes tests to confirm correct calculations and validation.
   • Self-correct: Copilot investigates failed tests and revises its implementation.

   Note: Copilot may successfully complete the task without encountering a test failure. If this occurs, use an additional prompt to encourage further verification and correction.

4. Challenge

   Identify human intervention signals

   GitHub Copilot can autonomously inspect code, implement changes, run tests, and correct errors. However, not every problem should be resolved without developer input. The key distinction is between technical problems Copilot can verify and decisions requiring human judgment, authorization, or business context.

   • Autonomous resolution: Syntax errors, failing unit tests, incorrect calculations, and straightforward refactoring.
   • Human intervention: Ambiguous business requirements, security-sensitive changes, destructive operations, and decisions involving conflicting priorities.
   • Permission requests: Copilot may request approval before executing terminal commands or accessing resources. These are checkpoints for human oversight, even when the underlying task is routine.
   • Repeated failures: If Copilot repeatedly attempts the same unsuccessful solution, the developer may need to provide additional context or redirect its approach.

   Continue working with the shopping cart application.

   1. In Copilot, enter the following:

      Update the shopping cart to support customer
      loyalty discounts.
      
      Loyal customers should receive better discounts
      than regular customers.
      
      Review the existing code and explain your proposed
      implementation before making changes.
      
      Identify any business rules that need
      clarification.
      

      The prompt intentionally leaves several requirements undefined. For example, what qualifies a customer as loyal, how much of a discount should they receive, and can loyalty discounts be combined with other promotions?
      Does Copilot request clarification, identify assumptions, or proceed with an implementation based on its own interpretation?
      
      If Copilot begins making assumptions, you can press Esc once or more to cancel the run, and intervene with the following:

      Do not implement loyalty discounts yet.
      The business requirements are not sufficiently defined.
      List the decisions that require human input and
      explain why those decisions cannot be determined
      through code inspection or automated testing.
      

      This demonstrates that an agent may be technically capable of implementing a feature while lacking the business context necessary to implement it correctly.

   2. Now provide Copilot with a requirement that has clearly defined acceptance criteria.

      Implement the following loyalty discount rules:
      - Regular customers receive no loyalty discount.
      - Silver customers receive 5%.
      - Gold customers receive 10%.
      - Invalid loyalty tiers raise ValueError.
      - Loyalty discounts cannot be combined with other discounts.
      Implement the feature, generate unit tests, and
      run the tests.
      
      If tests fail, investigate and correct the errors
      without requesting assistance, unless a business
      requirement is unclear.
      

      Observe how Copilot moves from requiring human input to independently implementing and validating the feature.

5. Challenge

   Manage context and custom instructions in sessions

   As an Agent Mode session grows, Copilot accumulates conversation history, code changes, tool outputs, and previous instructions. Over time, this can make it harder for the agent to prioritize the most relevant information. Context drift occurs when an agent gradually moves away from the original requirements, coding conventions, or intended scope. Common signals include the following:

   • Forgetting previously established requirements.
   • Introducing functionality outside the requested scope.
   • Generating code that conflicts with existing conventions.
   • Repeatedly inspecting irrelevant files or revisiting completed work.

   Two techniques that help manage these challenges are maintaining persistent custom instructions, and controlling session context. You'll implement the former next. GitHub Copilot CLI supports repository-level instructions; Copilot's specific instructions are stored in .github/copilot-instructions.md. These provide project-specific guidance that Copilot can use across prompts and sessions. Copilot will automatically generate information about the repo but will not generate instructions on its limitations without prompting.

   You will now add more to the markdown file (the .md file) which clarifies things.

   1. Beside the existing Terminal, click the +, select New tab, then in the pop-up click Terminal.
      
      This opens a second Terminal at the Bash prompt, in which you'll next append to the .md file.

   2. In the second Terminal, the one with the workspace $ prompt, run the following command:

      cat >> ~/workspace/.github/copilot-instructions.md
      

      This will allow input to be added to the .md file.

   3. Paste in the following clarifying instructions:

      # Shopping Cart Project Instructions
      
      ## Project Context
      - This project is a Python shopping cart application.
      - The application supports items, quantities, and discounts.
      - Maintain compatibility with existing functionality.
      
      ## Coding Standards
      - Use Python standard libraries whenever possible.
      - Keep functions simple and readable.
      - Avoid unnecessary dependencies.
      - Preserve existing public method names.
      
      ## Development Requirements
      - Add unit tests for new functionality.
      - Run tests after modifying application logic.
      - Do not remove existing functionality without approval.
      - Ask for clarification when business rules are ambiguous.
      - Do not add unrelated features.
      

   4. Press Enter, then press Control+D.

      • This adds in a final newline, and ends the input session, allowing the text to be appended to the .md file.
      • The added instructions establish consistent expectations for Copilot without requiring you to repeat them in every prompt.

   5. Back in Copilot, in the first Terminal with the >  prompt, enter the following:

      /instructions
      

      • During one test, the .md file was not noticed; AI can be inconsistent. Starting a new session using /clear forced Copilot to notice the file. Do note this resets the conversation, which at this point is okay as changes were made, and instructions stored in the .md file.

   It will list .github/copilot-instructions.md.