DuckDB Meets Postgres

A growing number of organizations are moving historical data from Postgres to S3 and adopting Apache Iceberg, a table format enabling S3 data to be queried like SQL tables. While query engines like Trino, Spark, or Flink integrate with Iceberg, Postgres does not. ParadeDB has introduced Iceberg table support for S3 and Google Cloud Storage, replacing DataFusion with DuckDB for accelerated engineering efforts. Iceberg, designed for analytics over large datasets, organizes metadata around files like Parquet, offering features like ACID transactions and schema evolution. pg_lakehouse adds Iceberg support to Postgres using the foreign data wrapper API, pushing most queries down to DuckDB for enhanced analytical performance. Initially preferring DataFusion for its extensibility and growing adoption, ParadeDB switched to DuckDB due to its out-of-the-box integrations, familiarity to end users, and better performance in benchmarks. Future plans for pg_lakehouse include adding write support for copying Postgres tables into external object stores. Iceberg is supported on pg_lakehouse version 0.8.0 or later, available for installation or through a Docker image.