A reflective lake

An Introduction to Modern Data Lake Storage Layers

In recent years we’ve seen a rise in new storage layers for data lakes. In 2017, Uber announced Hudi - an incremental processing framework for data pipelines. In 2018, Netflix introduced Iceberg - a new table format for managing extremely large cloud datasets. And in 2019, Databricks open-sourced Delta Lake - originally intended to bring ACID transactions to data lakes. 📹 If you’d like to watch a video that discusses the content of this post, I’ve also recorded an overview here....

 · 13 min