descriptionLong TextWant to try for yourself? Find the code here → https://ibm.biz/~pDRvDsIfj
Want to run an LLM on your own hardware? Cedric Clyburn demonstrates self-hosted AI inference with vLLM, from loading an open-weight model to serving it through an API. Learn how quantization, GPU memory, and model serving shape practical local AI deployments.
00:00 - Why Self-Host an LLM?
00:21 - What Is vLLM and Why It Matters
00:57 - Continuous Batching & Paged Attention Explained
02:16 - Two Ways to Use vLLM: Python vs API Server
02:45 - Choosing Open Models for AI Inference
03:23 - Running Offline Inference with Python
05:21 - Serving LLMs Through an API Endpoint
07:45 - Understanding Quantization and GPU Memory
08:37 - Using Compressed Models Efficiently
10:08 - Deploying and Evaluating AI Models with vLLM
Explore more technical tutorials, articles, and learning resources here → https://ibm.biz/~KvxBUv6yY
AI was used in the creation of the transcript and metadata for this video.
#vllm #selfhostedai #opensource #opensourceai
-----------------------------------------------------------------------------------------------------------
Find us on YouTube:
🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates
🔵 IBM Technology: https://www.youtube.com/@IBMTechnology
🔵 IBM: https://www.youtube.com/@IBM
Follow our Developer social channels:
🟣https://www.linkedin.com/showcase/ibmdeveloper
🟣 https://x.com/ibmdeveloper