Building a Real-Time Analytics Database • Tim Berglund • GOTO 2023
This presentation was recorded at GOTO Chicago 2023. #GOTOcon #GOTOchgo
Tim Berglund - VP DevRel at StarTree & Author of “Gradle Beyond the Basics“ @tlberglund @startreefilms
ORIGINAL TALK TITLE
Building a Real-Time Analytics Database: A ’Choose Your Own Adventure’ Journey
RESOURCES
Tim
ABSTRACT
Have you ever stopped to think about how to build a database? The thing is, there isn’t just one way, as we can see by the massive number of data infrastructure options we have to choose from. It’s a nonstop series of tradeoffs, each motivated by the constraints the database wants to satisfy. An in-memory transactional database would be one thing. A general-purpose, single-server relational database would be another. A low-latency, horizontally scalable analytics database would journey we’re going to take.
In this talk, we’ll start by picking a data model, make decisions about serialization and storage, choose indexing strategies, pick a query language, and figure out how to scale, eventually ending up with something that looks remarkably like Apache Pinot, a real-time analytics database. Pinot was built on a journey like this, always optimized for ultra low-latency, user-facing analytics at scale. In the real world, Pinot is used by applications like LinkedIn and UberEats to expose the state of the system not just to internal decision-makers, but to the users of the system itself, including all of us people who consumers of analytical queries. By focusing on the internals of Pinot and the tradeoffs made along the way to build a database of its kind, we’ll see how it enables a new class of applications that every user of a system into a decision maker. [...]
Download slides and read the full abstract here:
RECOMMENDED BOOKS
Tim Berglund • Gradle Beyond the Basics •
Tim Berglund & Matthew McCullough • Building and Testing with Gradle •
Mark Needham • Building Real-Time Analytics Systems •
Gwen Shapira, Todd Palino, Rajini Sivaram & Krit Petty • Kafka: The Definitive Guide •
Adi Polak • Scaling Machine Learning with Spark •
#ApachePinot #Analytics #RealTime #RealTimeAnalytics #TimBerglund #StarTree #StarTreeCloud #Cloud #ApachePinotTutorial #ApachePinotTraining #OLAP #OLTP #LowLatency #ApacheZooKeeper #ApacheHelix #Hadoop #ApacheSpark
Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at
Sign up for updates and specials at
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
1 view
0
0
6 months ago 00:01:29 1
Need for Speed Unbound - VOL 2 Content Update Trailer
6 months ago 03:13:24 1
Full video: Girl Living Alone in the Forest - Set Traps and Ambush Wild Boars, Big Sharp Trap...
6 months ago 00:25:52 1
Heavenly Sounds - The Organ and its Fascinating Versatility | Music Documentary
6 months ago 00:01:00 1
Juggler vs Juggler! The ultimate showdown! | Cirque du Soleil #shorts
6 months ago 00:13:08 1
Just Insert an Electrical Plug on the TV and Watch all the Channels in the World in Full HD!
6 months ago 00:05:16 1
Орск. Крупнейшее наводнение в 1957 / Orsk. Largest flood in 1957
6 months ago 00:02:23 1
Starship | Third Flight Test
6 months ago 00:01:41 1
Runa | Official Announcement Trailer | April 16 on Kickstarter
6 months ago 11:54:59 1
Mystical Treasure - Brings you miracles, money, luck and prosperity - 432hz
6 months ago 11:54:59 1
This week you will become very rich | 432 Hz music attracts money, prosperity and luck!
6 months ago 00:18:25 1
Curious George in Space 🐵 Curious George 🐵Kids Cartoon 🐵 Kids Movies 🐵Videos for Kids