This INCREDIBLE trick will speed up your data processes.
In this video we discuss the best way to save off data as files using python and pandas. When you are working with large datasets there comes a time when you need to store your data. Most people turn to CSV files because they are easy to share and universally used. But there are much better options out there! Watch as Rob Mulla, Kaggle grandmaster, discusses some alternative ways of saving data files: pickle, parquet and feather files. I run some benchmarks to show that you can save time, space and keep the important metadata about your files in the process!
Timeline
00:00 Intro
00:49 Creating our Data
02:08 CSVs
04:39 Setting dtypes for CSVs
06:15 Pickle Files
07:16 Parquet ❤️
09:07 Feather
10:31 Other Options
11:02 Benchmarking
12:19 Takeaways
12:43 Outro
Code Gist:
Follow me on twitch for live coding streams:
Other Videos:
1 view
78
15
4 months ago 00:54:49 1
The Art & Science of Cymatics - John Stuart Reid
4 months ago 01:04:00 1
Dec 20, 2023 - REBROADCAST What is the “Big Event“ That Tall Whites Predict for Earth
4 months ago 00:03:51 1
Thirty Seconds To Mars - Hail to the Victor (Official Music Video)
4 months ago 00:05:12 1
What happens to your body at the top of Mount Everest - Andrew Lovering
4 months ago 00:00:14 1
Lovebird Helps Open Sugar Packet || ViralHog
4 months ago 03:53:49 1
NO MORE Insomnia ★︎ FALL INTO SLEEP INSTANTLY ★︎ Calm Night on a Secluded Island
4 months ago 00:03:43 1
Snuff Out the Light - Kingdom of the Sun (Fan Reconstruction & Storyboard)
4 months ago 00:03:20 1
Frizform - Incredible
4 months ago 00:00:34 1
Princess Daisy from Super Mario Movie 👸 👑 Live at WONDERCON 2023 Real Nintendo World Cosplay
4 months ago 00:03:14 1
Я беру капусту, куриную грудку и готовлю этот невероятно вкусный быстрый ужин! Рекомендую рецепт
4 months ago 00:01:00 1
The Ibex🐐 - More Than Just a Mountain Goat
4 months ago 00:00:44 1
Amazing lifehack with mango 🥭🥭🥭
4 months ago 00:04:46 1
Category 7 - In Stitches (Official Video)
4 months ago 00:03:51 2
CDK - Somebody That I Used To Know by Gotye
4 months ago 00:04:45 1
NORD LIVE: Jesus & Molina’s Brothers - Nissi
4 months ago 00:01:23 1
EU Mega Launch! We’re expanding to 7 new markets | Oriflame
4 months ago 00:23:36 1
Oil Massage to Remove Wrinkles around Eyes(Crow feet) and Lift up Droopy Eyelids!
4 months ago 00:19:28 1
Face Lifting Oil Massage to Remove Eye Bags & Laugh Lines(Nasolabial folds)
4 months ago 00:04:30 1
Mötley Crüe - Girls, Girls, Girls (The End, Live In Los Angeles)
4 months ago 00:03:17 1
How To Get Unlimited Points in Shein Hack 2024 (Android/iOS)
4 months ago 00:09:51 1
This 51 year old Japanese Skater Is Incredible
4 months ago 00:24:59 1
A ’Jolly Milkman’: Faith, Family, Farm | Justus Walker | Real Life in the Rural Outback of Russia🇷🇺
4 months ago 00:32:52 1
Revolutionary Technology Defies the Laws of Physics #podcast #randallcarlson #science #ancient