Debezium PostgreSQL Connector — Configuration That Matters

The default Debezium connector config produces usable events, but the defaults leave important gaps — before values are missing on UPDATEs and DELETEs, all tables are captured including system tables, and each table gets its own Kafka topic. This post covers the configuration knobs that close those gaps: replica identity, table filtering, publication modes, and single-topic routing with the ByLogicalTableRouter transform.

Replica Identity — Getting the “before” Value

With the default connector from the previous post, UPDATE and DELETE events have before: null. The previous row state is missing because PostgreSQL’s default replica identity only writes the primary key to the WAL on UPDATEs and DELETEs — not the full row. Debezium can only produce what the WAL gives it.

Seeing the Problem

UPDATE orders SET status = 'shipped' WHERE id = 1;

The resulting event:

{
  "before": null,
  "after": {
    "id": 1,
    "customer": "alice",
    "product": "Widget A",
    "amount": 29.99,
    "status": "shipped"
  },
  "op": "u"
}

There is no way to tell what status was before the update. For audit trails, sync pipelines, or any use case that needs to know what changed (not just what the current state is), this is a problem.

Fixing It

Set the table’s replica identity to FULL so PostgreSQL writes the entire old row to the WAL:

ALTER TABLE orders REPLICA IDENTITY FULL;

UPDATE orders SET status = 'cancelled' WHERE id = 1;

Now the event has before populated:

{
  "before": {
    "id": 1,
    "customer": "alice",
    "product": "Widget A",
    "amount": 29.99,
    "status": "shipped"
  },
  "after": {
    "id": 1,
    "customer": "alice",
    "product": "Widget A",
    "amount": 29.99,
    "status": "cancelled"
  },
  "op": "u"
}

DELETEs also benefit — a DELETE on a FULL table produces before with the deleted row’s data and after: null.

Automating It

Instead of manually running ALTER TABLE ... REPLICA IDENTITY FULL on each table, Debezium can do it automatically:

"replica.identity.autoset.values": "public.*:FULL"

This tells the connector to set REPLICA IDENTITY FULL on all tables in the public schema matching the pattern. You can verify it took effect:

SELECT relname, relreplident
FROM pg_class
WHERE relname IN ('orders', 'customers');
-- 'f' = FULL, 'd' = DEFAULT

When You Don’t Need FULL

If you only care about the current state of a row — for example, replicating a table to another database where you overwrite the target row with the latest after — DEFAULT replica identity is fine. FULL adds overhead because PostgreSQL writes more data to the WAL on every UPDATE and DELETE.

Table Filtering

The default connector captures all tables in the database. In practice, you want to exclude system tables (migration tracking, job queues, framework-internal tables) and capture only domain tables.

table.include.list

An allowlist of fully-qualified table names:

"table.include.list": "public.orders,public.customers"

Any table not in the list is ignored. With this config, inserts into other tables produce no CDC events and no Kafka topics are created for them.

table.exclude.list

A denylist — capture everything except these tables:

"table.exclude.list": "public.flyway_schema_history,public.schema_migrations"

You can use one or the other, not both.

The Alternative: Publication Filtering

Instead of filtering at the Debezium level, you can control it at the PostgreSQL level through publications. The publication.autocreate.mode setting controls how Debezium manages publications:

Mode	What it does
all_tables	Auto-creates a publication for all tables. Simple, good for development
filtered	Auto-creates a publication only for tables in table.include.list
disabled	Expects a publication to already exist — you create it yourself via a migration

For production, disabled with a manually-created publication gives the most explicit control. The publication is created in a database migration, not by the connector:

-- In a migration script
CREATE PUBLICATION cdc_publication FOR TABLE orders, customers, products;

Then the connector references it:

"publication.autocreate.mode": "disabled",
"publication.name": "cdc_publication"

This avoids surprises when new tables are added to the schema — they are only captured when explicitly added to the publication.

Single-Topic Routing with ByLogicalTableRouter

By default, Debezium creates one Kafka topic per table (learn.public.orders, learn.public.customers, etc.). The ByLogicalTableRouter transform routes events from multiple tables into a single topic.

"transforms": "route-to-single-topic",
"transforms.route-to-single-topic.type": "io.debezium.transforms.ByLogicalTableRouter",
"transforms.route-to-single-topic.topic.regex": ".*",
"transforms.route-to-single-topic.topic.replacement": "learn.v0.cdc",
"transforms.route-to-single-topic.key.enforce.uniqueness": "false"

The route-to-single-topic name is an alias you choose — it is just a label to reference the transform in the config. The .type is the actual Java class. Everything else under that name is config passed to that class.

After applying this, all events from all captured tables land on learn.v0.cdc. The consumer distinguishes them by the source.table field in the event envelope.

Why One Topic

• Simpler topic management — one topic to configure, monitor, and set retention on
• Downstream sinks — an S3 sink or data warehouse connector only needs to subscribe to one topic
• Ordering across tables — events from the same transaction across different tables land on the same topic (though not guaranteed to be on the same partition)

The trade-off: consumers need to handle mixed event types and filter by source.table.

Chaining Transforms

You can chain multiple transforms — they run in order on each event:

"transforms": "filter,route",
"transforms.filter.type": "...",
"transforms.route.type": "..."

Snapshot Modes

The snapshot.mode setting controls what happens when the connector starts:

Mode	Behavior
initial	Snapshot all existing rows on first start, then stream. On subsequent starts, just stream from where the slot left off
never	No snapshot — only stream changes from the WAL. Existing rows are not captured
when_needed	Snapshot if the replication slot does not exist or the slot’s position is no longer available in the WAL
initial_only	Snapshot existing rows and then stop — no streaming

initial is the most common choice. It gives you the full current state of the database as op: "r" events, followed by real-time changes as op: "c"/"u"/"d".

never is useful when you only care about changes going forward — for example, if the historical data has already been loaded by another mechanism.

Tombstone Records

"tombstones.on.delete": "false"

When set to true (the default), Debezium produces a second event after every DELETE with a null value (a Kafka tombstone). Tombstones tell Kafka’s log compaction to remove the key entirely. If your downstream consumers do not rely on log compaction, setting this to false removes the extra event.

A Full Production-Oriented Config

Putting it all together — a connector config that filters tables, sets replica identity, routes to a single topic, and uses explicit publication management:

{
  "name": "orders-cdc-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "postgres",
    "database.port": "5432",
    "database.user": "cdc_reader",
    "database.password": "${file:/secrets/db-password}",
    "database.dbname": "orders_db",
    "topic.prefix": "orders",
    "plugin.name": "pgoutput",
    "slot.name": "orders_cdc_slot",
    "publication.name": "orders_publication",
    "publication.autocreate.mode": "disabled",
    "table.include.list": "public.orders,public.customers,public.products",
    "snapshot.mode": "initial",
    "tombstones.on.delete": "false",
    "replica.identity.autoset.values": "public.*:FULL",
    "transforms": "route",
    "transforms.route.type": "io.debezium.transforms.ByLogicalTableRouter",
    "transforms.route.topic.regex": ".*",
    "transforms.route.topic.replacement": "orders.v0.cdc",
    "transforms.route.key.enforce.uniqueness": "false",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": "false",
    "database.ssl.mode": "require",
    "errors.retry.timeout": "600000",
    "producer.override.max.request.size": "20971520"
  }
}

Config	Why
publication.autocreate.mode: disabled	Explicit control — publication created via migration, not by Debezium
replica.identity.autoset.values	Ensures before-values on UPDATE/DELETE for audit history
database.ssl.mode: require	Encrypted connections
errors.retry.timeout: 600000	Retry transient errors for 10 minutes before failing
producer.override.max.request.size: 20971520	Handles large payloads (big text fields, JSONB columns) up to 20 MB

Available Debezium Transforms

The transforms that come up most often in practice:

Transform	Type class	What it does
ByLogicalTableRouter	io.debezium.transforms.ByLogicalTableRouter	Routes multiple tables to a single topic via regex
EventRouter	io.debezium.transforms.outbox.EventRouter	Routes outbox table rows to dynamic topics based on a field — used with the transactional outbox pattern
ExtractNewRecordState	io.debezium.transforms.ExtractNewRecordState	Flattens the envelope — strips before/source/op, emits just the after payload
Filter	io.debezium.transforms.Filter	Drops events based on scripting expressions
ContentBasedRouter	io.debezium.transforms.ContentBasedRouter	Routes to different topics based on expressions evaluated against event content
TimezoneConverter	io.debezium.transforms.TimezoneConverter	Converts timestamps between timezones

ByLogicalTableRouter and EventRouter cover the majority of production use cases. ExtractNewRecordState is useful when the downstream consumer wants a flat record instead of the full envelope — for example, when sinking directly into a relational table where each column maps to a field in after.