A Guide to Delta Lake Sessions at Data+AI Summit
The Data+AI Summit returns to San Francisco from June 10-13, 2024. With over 160 countries represented, the Data+AI Summit is a global event that brings together thousands of data practitioners, leaders and visionaries from more than 160 countries to explore the latest in lakehouse, open source, AI/ML and more. As we gear up for this monumental event, we are excited to spotlight a series of Delta Lake sessions.
With a carefully curated lineup of over 500 sessions, the Data+AI Summit schedule is jam-packed with everything from data warehousing, to governance and the latest in generative AI. From optimizing data pipelines to revolutionizing data engineering with Rust to navigating the complexities of multi-petabyte data migrations, we’ve put together a list of Delta Lake sessions you won’t want to miss.
Delta Lake Sessions at Data+AI Summit
Innovations in Data Architecture and Operations
Explore the foundational changes and advancements in data architecture facilitated by Delta Lake. These sessions focus on how Delta Lake integrates with other systems, enhances data architecture, and pushes the boundaries of what's possible in data operations.
Delta Lake is the unified storage layer for the data lakehouse. Join us to learn about the newest features added in Delta Lake 4.0, the next major release of the project.
Best Features of Delta Lake: Love Your Open Tables
Delve into the world of Delta Lake, the revolutionary open-source storage system designed for both Spark and non-Spark analyses. This session will guide you through Delta Lake’s standout features such as Liquid Clustering, UniForm, sophisticated column operations, and the game-changing capabilities of time travel and schema evolution. Learn how to leverage these tools across various analysis engines including Spark, Trino, Flink, and Pandas, to enhance interoperability and query performance.
Delta Kernel: Anatomy of a Delta Lake Connector
In this session, we'll talk about the Delta Kernel project, which simplifies creation and maintenance of Delta Lake connectors. Join us to learn how to create a connector, architectural and engineering decisions you need to make, and how this is being used across the Delta Lake community.
Scaling and Efficiency in Data Processing
Learn strategies and best practices for scaling data processing tasks efficiently with Delta Lake. These sessions will dive into real-world examples and advanced techniques that enable organizations to handle massive datasets seamlessly.
Processing a Trillion Rows Per Day with Delta Lake at Adobe
Get an insightful update on Adobe’s scaling journey, navigating through 8000+ self-managed Delta tables. Discover the strategies and practices enabling the Adobe Experience Platform to manage petabytes of customer data, powering diverse marketing scenarios across multiple channels. The session will cover scaling challenges, JVM agnostic locking, anti-patterns, and much more.
Drastically Reducing Processing Costs with Delta Lake
Discover how Amadeus, a global technology leader in the travel sector, leverages Delta Lake to transform data across various business units into efficient, cost-effective star schemas. This session dives into cutting-edge optimizations and collaborative successes with Databricks, focusing on overcoming read-and-write amplification challenges with innovative features like Predictive I/O and Photon.
Migrating a Multi-Petabyte Data Platform to Delta Lake at Zalando
Join Zalando's journey through a complex, company-wide migration of its core data platforms to Delta Lake. This session covers the technological and transformational challenges encountered during the migration of multi-petabyte-scale data platforms, offering key insights and lessons learned valuable for organizations contemplating similar transitions.
Enhancing Data Management with Advanced Technologies
Dive into the latest advancements in data management with Delta Lake and other state-of-the-art technologies. These sessions will showcase innovative tools and methods to optimize data ingestion, streaming, and versioning processes.
Fast, Cheap and Easy Data Ingestion with AWS Lambda and Delta Lake
Explore how AWS Lambda can revolutionize your data ingestion and processing workflows. This practical session will demonstrate using AWS Lambda with Delta tables for efficient, event-driven data management. Prepare to unlock new levels of efficiency and flexibility in your AWS-based data platforms.
Effective Lakehouse Streaming with Delta Lake and Friends
Learn the secrets to achieving scalable, reliable streaming data systems within the lakehouse architecture. This session will unravel the core principles of streaming data capture, schema enforcement, data quality, and workflow orchestration, showcasing why Delta Lake is instrumental in building a robust data ecosystem.
Power Up Your Lakehouse with Git Semantics & Delta Lake
Learn how to enhance your lakehouse architecture with git-like semantics through the integration of Delta Lake and lakeFS. This talk explores the revolutionary approaches to data versioning, enabling improved auditability, rollback capabilities, and the creation of economical, ad hoc development and testing environments.
Emerging Tools and Languages for Data Engineering
Stay ahead of the curve by exploring the latest tools and languages revolutionizing data engineering. These sessions highlight innovative approaches and practical applications of emerging technologies with Delta Lake.
Let's Do Some Data Engineering With Rust and Delta Lake!
Step into the future of data engineering with a session that introduces Rust's capabilities in handling data with Delta Lake. Whether you're new to Rust or looking to expand your skills, this talk offers valuable insights into creating high-performance data pipelines and applications, emphasizing real-world problem-solving in data ingestion, enrichment, and management.
Semi-Structured Communication Data with Delta Lake and Microservices
Learn about the challenges and solutions for managing semi-structured communication data in the context of eDiscovery. This session showcases a microservices-based approach, utilizing delta-rs, Arrow Flight, and DataFusion for efficient data storage, querying, and analysis, offering insights valuable to engineers, architects, and legal professionals.
Data Intelligence and Strategy
Enhance your data strategy with sessions focusing on advanced data intelligence and strategic approaches using Delta Lake in Databricks. Learn about the latest innovations and how they can transform your data management practices.
Deep Dive Into Delta Lake and Uniform on Databricks
This session is an introductory deep dive into Delta Lake, the cutting-edge open source storage layer that revolutionizes data lakes by ensuring reliability, performance, governance, and quality. This session will cover crucial aspects such as the architecture of Delta Lake, designed to seamlessly handle both batch and streaming data, and its integration with Unity Catalog. Learn about Delta Lake’s wide adoption across various industries, explore real-world use cases, and hear about the exciting features on the horizon.
Introducing Data Intelligence to Delta Lake with DatabricksIQ
DatabricksIQ makes your Delta Lake tables faster, cheaper and easier to use. This session will dive into how AI-driven capabilities like Predictive Optimization and Liquid Clustering can transform your data management — making your tables not only faster and cheaper but also simpler to operate. Discover firsthand how users are leveraging these tools to automate maintenance and tuning, and get a sneak peek at the thrilling new optimizations on our roadmap.
Delivering Portability to Open Data Lakes with Delta Lake UniForm
Discover how Delta Lake UniForm is addressing the challenges of reliability, performance, and governance in data lakes across multiple environments. This session will provide a deep dive into the technology behind UniForm, demonstrating its impact on data lake portability and query performance through live demos.
Towards Multi-Statement Transactions in Delta Lake
Get an exclusive look at the future of transactions in Delta Lake with the introduction of a new commit protocol. This session will discuss the innovations enabling multi-statement transactions and the enhancement of data governance through managed-commits.
Why Attend Data+AI Summit?
By attending the Data+AI Summit, you'll not only gain invaluable insights from Delta Lake sessions but you’ll also have the opportunity to:
- Engage with the latest advancements in Delta Lake, AI, ML, and lakehouse architectures.
- Network with a diverse and vibrant community of data and AI professionals.
- Grow your professional network in social meetups or on the Expo floor or at the Data+AI Summit event party.
- Learn from leading experts, researchers, and open source contributors in the data and AI spheres.
Join us at Data+AI Summit!
Whether you're looking to strengthen your technical expertise, create new professional relationships, or learn about the latest industry trends, Data+AI Summit has it all. Register for Data+AI Summit today, and prepare to be inspired, educated, and connected with the forefront of the data and AI community.