Build next generation Big Data applications with Delta Lake

Track: Frameworks

Abstract

Delta Lake (https://delta.io/) is an open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark and also APIs for Scala/Java, Python and Rust. A Lakehouse is a modern data architecture that reimagines data warehouses in response to the availability of affordable and highly reliable storage solutions.

Delta Lake provides key benefits and will fit perfectly in your Big Data architecture:
- ACID Transactions
- Schema Evolution
- Time Travel
- Audit History
- Handle petabyte-scale tables
- Platform Agnostic (Cloud, On-prem, or locally)
- DML operations through its SQL and Spark API

In this presentation, I will provide an introduction to Delta Lake, explaining how it works, and its key features and benefits. Whether you're a data scientist, data engineer, or business analyst, this session is for you.

Theo Lebrun

Theo Lebrun is a seasoned Data Engineer and Technology Consultant with a vast experience in cloud computing, databases, and software development. With expertise in a range of tools and technologies including AWS, Kafka, Databricks, and Python, Theo has helped numerous clients optimize their data pipelines meeting and exceeding their goals. Theo's technical expertise are matched by his passion for sharing his knowledge with others. He has authored multiple blog posts covering and breaking down technical topics such as high-scaled streaming development with Kafka or Databricks Tips and Tricks. He has also presented in multiple conferences such as DevNexus and RVATech Data Summit.