Connect to Apache Iceberg

Connect RisingWave to your Apache Iceberg tables.

This guide explains how to connect RisingWave to your Apache Iceberg tables for batch data ingestion. RisingWave supports reading data from Iceberg tables stored on S3-compatible storage (including AWS S3 and MinIO) and Google Cloud Storage (GCS).

Prerequisites

• An existing Apache Iceberg table.
• Access credentials for the underlying storage system (e.g., S3 access key and secret key).
• Network connectivity between RisingWave and your storage system.
• Know your Iceberg catalog type.

Connect to Iceberg

RisingWave supports connecting to Iceberg using CREATE SOURCE. Currently, RisingWave does not support creating a table backed by an Iceberg table using CREATE TABLE ... WITH (connector=...). If you create a table to store data, and ingest from the Iceberg source, the table is not an Iceberg table.

Basic connection example (S3)

The following example shows how to connect to an Iceberg table stored on S3:

CREATE SOURCE iceberg_source
WITH (
    connector = 'iceberg',
    warehouse.path = 's3://your-bucket/path/to/iceberg/warehouse',
    database.name = 'your_iceberg_db',
    table.name = 'your_iceberg_table',
    s3.endpoint = 'http://your-s3-endpoint:port', -- e.g., 'http://minio:9000'
    s3.access.key = 'YOUR_ACCESS_KEY',
    s3.secret.key = 'YOUR_SECRET_KEY',
    s3.region = 'your-s3-region'  -- Optional if endpoint is specified
) ;

Replace the placeholders with your actual values.

The example shows connecting to S3 compatible storage systems. For different catalog types and storage systems, please refer to Apache Iceberg configuration options.

Query data

You can select from an Iceberg source to query data from the Iceberg table in RisingWave.

SELECT * FROM iceberg_source;

Create streaming jobs

Support for creating streaming jobs directly from Iceberg sources was added in v2.3.0. If your Iceberg source was created before this version, you must recreate it to enable streaming job creation.

You can create a materialized view or a sink from an Iceberg source to incrementally process and transform its data, similar to how you would with other supported sources.

Note that streaming ingestion only works for append-only iceberg tables. Any delete or update operations will be ignored.

Data type mapping

RisingWave converts data types from Iceberg to RisingWave according to the following table:

Iceberg Type	RisingWave Type
boolean	boolean
integer	int
long	bigint
float	real
double	double
string	varchar
date	date
timestamptz	timestamptz
timestamp	timestamp
decimal	decimal

Time travel

RisingWave’s Iceberg source supports time travel. Please refer to Apache Iceberg configuration options for more details.

System tables

RisingWave offers system tables to query Iceberg metadata. Please refer to Apache Iceberg configuration options for more details.

What’s next?

• All configuration options: Apache Iceberg configuration options