Skip to content

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.

Mastering LLM Deployment

Course Summary

Mastering LLM Deployment is designed for software engineers and data scientists looking to deploy large language models (LLMs) efficiently and cost-effectively. Participants will learn essential techniques for model distillation, quantization, and pruning to optimize LLMs. The course includes hands-on experience deploying these models into AWS ECS using Docker and strategic insights into cost-saving measures. By the end of the course, participants will have the skills to deploy optimized LLMs in a production environment, ensuring efficient resource usage and cost optimization.


Purpose
Acquire the skills to deploy optimized LLMs in a production environment.
Audience
Software engineers and data scientists with basic familiarity with TensorFlow, Keras, and AWS, who are interested in deploying and optimizing large language models. An understanding of NLP and Deep Learning and familiarity with Python are essential prerequisites.
Role
Data Scientist | Software Engineer
Skill level
Advanced
Style
Lecture | Case Studies | Labs
Duration
2 days
Related technologies
AWS ECS | Docker | Tensorflow | Keras | Python

 

Learning objectives

  • Distill, quantize, and prune large language models.

  • Analyze and optimize the resource requirements for LLM deployment.

  • Deploy optimized LLMs into AWS ECS using Docker.

  • Implement TensorFlow Serving and Flask API for LLM deployment.

  • Understand and apply cost-saving strategies for LLM deployment.

What you'll learn:

In this course, you'll learn:
  • Introduction and Optimization Techniques
  • Quick Recap of TensorFlow and Keras
    • Overview of TensorFlow and Keras
      • Brief refresher on TensorFlow and Keras functionalities relevant to LLMs.
    • Lab: Basic Keras and TensorFlow Exercises
      • Hands-on exercises to familiarize participants with essential TensorFlow and Keras functions.
  • Course Introduction and Case Study
    • Overview of LLM Deployment Challenges and Objectives
      • Introduction to the course structure, objectives, and key challenges in LLM deployment.
    • Case Study: Successful LLM Deployment
      • Detailed analysis of a real-world LLM deployment case, highlighting challenges, solutions, and outcomes.
  • Model Distillation
    • Introduction to Model Distillation
      • Overview of model distillation and its benefits for LLMs.
    • Lab: Distilling a Pre-trained LLM using TensorFlow
      • Hands-on exercise to distill a given LLM, using the SQuAD dataset.
      • Participants will learn to reduce the model size and improve inference speed.
  • Model Quantization
    • Understanding Model Quantization
      • Introduction to quantization techniques and their benefits.
    • Lab: Quantizing an LLM using TensorFlow
      • Practical lab to quantize a pre-trained LLM, using the IMDB dataset for sentiment analysis.
      • Participants will convert the model to lower precision to save memory and improve performance.
  • Model Pruning
    • Fundamentals of Model Pruning
      • Explanation of pruning methods and their benefits.
    • Lab: Pruning an LLM using TensorFlow
      • Hands-on exercise to prune an LLM, using the SST-2 dataset for sentiment analysis.
      • Participants will learn to remove redundant neurons and weights to optimize the model.
  • Deployment and Cost Optimization
  • Preparing for Deployment
    • Introduction to TensorFlow Serving and Flask API
    • Overview of serving models using TensorFlow Serving and Flask.
    • Lab: Setting Up Docker for Deployment
    • Hands-on lab to create Docker containers for LLM deployment.
    • Participants will learn to package the optimized LLMs into Docker containers.
  • Deploying to AWS ECS
    • Overview of AWS ECS and Deployment Strategies
      • Introduction to AWS ECS services and deployment options.
    • Lab: Deploying LLMs with TensorFlow Serving on AWS ECS
      • Practical exercise to deploy an LLM using TensorFlow Serving on AWS ECS.
      • Participants will learn to set up ECS tasks and services.
    • Lab: Deploying LLMs with Flask API on AWS ECS
      • Hands-on lab to deploy an LLM using Flask API on AWS ECS.
      • Participants will implement and test REST API endpoints for model inference.
  • Final Hackathon (3 hours)
  • Project: Text Summarization using CNN/DailyMail Dataset
    • Participants will work individually to deploy a finetuned LLM for text summarization using the CNN/DailyMail dataset.
    • They will apply distillation, quantization, and pruning techniques, and deploy the model using Docker and AWS ECS

    

Mastering LLM Deployment

real world content

Real-World Content

Project-focused demos and labs using your tool stack and environment, not some canned "training room" lab.
expert practitioners

Expert Practitioners

Industry experts that bring their battle scars into the classroom.
experiential learning

Experiential Learning

More coding than lecture, coupled with architectural and design discussions.
fully customized

Tailored Outlines

Once-size-fits-all doesn't apply to training teams. That's where we come in!

Dive in and learn more

When transforming your workforce, it’s important to have expert advice and tailored solutions. We can help. Tell us your unique needs and we'll explore ways to address them.

Let's chat

By filling out this form and clicking submit, you acknowledge our privacy policy.