Database

Introduction The data warehouse landscape is experiencing a tectonic shift. After decades of dominance by traditional vendor-locked solutions, a new way of thinking is emerging: data warehouses built on open table formats. This architectural approach isn’t just another incremental improvement—it represents a fundamental reimagining of how organizations store, manage, and analyze their critical data assets. Open table formats like Apache Iceberg, Delta Lake, and Apache Hudi are transforming what’s possible in data warehousing. By decoupling storage from compute and leveraging cloud-native technologies, these formats enable data architectures that are more flexible, cost-effective, and powerful than their traditional counterparts. ...

Introduction As data engineers and analysts, we rely heavily on SQL databases to store and query our data efficiently. To speed up our queries, we often create indexes on frequently filtered columns. However, there’s a common gotcha that can cause our queries to run slower than expected, even with appropriate indexes in place. In this post, we’ll explore how applying functions to indexed columns in the WHERE clause can prevent SQL optimizers from utilizing those indexes effectively. ...

DuckDB Inside Postgres: The Unlikely Duo Supercharging Analytics If you work in data engineering, you know that the field moves at a dizzying pace. New tools and technologies seem to pop up daily, each promising to revolutionize how we store, process, and analyze data. Amidst this constant change, two names have remained stalwarts: Postgres, the tried-and-true relational database, and DuckDB, the talented new kid on the block for analytics workloads. ...