Efficient Fine-Tuning for Llama-v2-7b on a Single GPU
The first problem you’re likely to encounter when fine-tuning an LLM is the “host out of memory” error. It’s more difficult for fine-tuning the 7B parameter Llama-2 model which requires more memory. In this talk, we are having Piero Molino and Travis Addair from the open-source Ludwig project to show you how to tackle this problem.
The good news is that, with an optimized LLM training framework like , you can get the host memory overhead back down to a more reasonable host memory even when training on multiple GPUs.
In this hands-on workshop, we‘ll discuss the unique challenges in finetuning LLMs and show you how you can tackle these challenges with open-source tools through a demo.
By the end of this session, attendees will understand:
- How to fine-tune LLMs like Llama-2-7b on a single GPU
- Techniques like parameter efficient tuning and quantization, and how they can help
- How to train a 7b param model on a single T4 GPU (QLoRA)
- How to deploy tuned models l
1 view
447
210
4 months ago 00:11:00 1
2024 Bentley Continental GT Speed - Wild luxury Sport Coupe!
4 months ago 00:24:17 1
266 Massive Heavy Load Logistics Industrial Giant Transportation and Heavy Machinery
4 months ago 00:00:56 2
Mech Engineer 1.0 Launch Trailer
5 months ago 00:01:44 1
Speechelo Review | Text to Speech Voice Over Software @20xincome-wx9zy
5 months ago 00:01:38 1
Whatsapp Marketing Software 🔥 How Can I Extract All Contacts From WhatsApp Group?
5 months ago 00:07:00 1
How To Use Roto Brush 3.0 in After Effects
5 months ago 00:01:55 1
Rebecca Ferguson Says Silo Season 3 and 4 Might Film Back-to-Back With Latter Ending Series
6 months ago 00:01:00 1
Vocal Compressor Plugin Settings – What are best? #musicproducer #musicproduction #studiolife
6 months ago 00:02:06 1
Revolutionize Customer Service with Alepo ’s AI Virtual Agent
6 months ago 00:31:15 1
PERFECT 30 MINUTE FULL BODY WORKOUT
6 months ago 00:13:24 5
2024 Audi S8 - Ultra Luxury Sport Sedan!
6 months ago 00:02:35 1
Whatsapp Marketing Software 🔥 How WawPro Revolutionizes WhatsApp Marketing?