This INCREDIBLE trick will speed up your data processes.
In this video we discuss the best way to save off data as files using python and pandas. When you are working with large datasets there comes a time when you need to store your data. Most people turn to CSV files because they are easy to share and universally used. But there are much better options out there! Watch as Rob Mulla, Kaggle grandmaster, discusses some alternative ways of saving data files: pickle, parquet and feather files. I run some benchmarks to show that you can save time, space and keep the important metadata about your files in the process!
Timeline
00:00 Intro
00:49 Creating our Data
02:08 CSVs
04:39 Setting dtypes for CSVs
06:15 Pickle Files
07:16 Parquet ❤️
09:07 Feather
10:31 Other Options
11:02 Benchmarking
12:19 Takeaways
12:43 Outro
Code Gist:
Follow me on twitch for live coding streams:
Other Videos:
1 view
78
15
1 month ago 00:28:30 1
“The Most Unique Home You’ve Ever Seen: Project Blow Your Mind
1 month ago 01:13:11 1
The State of Rogue Core (On The Horizon 2024)
2 months ago 00:14:36 1
TOP 40❗FAILS 4X4 THE CRAZIEST OFF ROAD ACCIDENTS ❌ INSANE FAILS AND WINS AMAZING VEHICLES 2024
2 months ago 00:16:17 1
The Ultimate Email Extractor in 2024 🚀
2 months ago 00:03:18 1
“Dancing on the Edge“ — War Thunder official cinematic trailer
2 months ago 00:17:26 1
Crossing From China🇨🇳 to Russia🇷🇺 I S2, EP98
2 months ago 00:52:21 1
The Nefertiti Scandal: Was Egypt’s Most Famous Bust Forged? | Full Documentary
2 months ago 00:20:50 1
Shocking Natural Disasters: Incredible Footage of Nature’s Fury
2 months ago 00:39:19 1
Historian Answers Witchcraft Questions | Tech Support | WIRED
2 months ago 00:12:42 1
Top 15 Upcoming City Building Games 2024 - 2025
3 months ago 00:00:31 1
Walking On Water #polarbear #northpole
3 months ago 00:04:09 1
(Everything I Do) I Do It For You - Bryan Adams (Boyce Avenue ft. Connie Talbot acoustic cover)
3 months ago 01:16:30 1
#1 Neurosurgeon: How to Manifest Anything You Want & Unlock the Unlimited Power of Your Mind
3 months ago 00:01:00 1
AWESOME Space Saving Furniture | Murphy Bed Ideas for Small Spaces Furniture Innovations
3 months ago 00:03:49 1
LINKIN PARK Epic Symphony Show - Somewhere I Belong
3 months ago 00:01:47 5
LINKIN PARK Epic Symphony Show - Teaser
3 months ago 00:02:43 1
She Keeps Bees - Burn (Session)
3 months ago 00:11:22 1
The Italian American pasta that drove the world crazy! Easy and delicious spaghetti meatballs recipe
3 months ago 02:12:31 1
Ludovico Einaudi FULL LIVE CONCERT - Arena di Verona, Italy (10/07/2024)