Data Engineering

Postgres Change Data Capture: How WAL, Logical Decoding and Replication Slots Work

Understand PostgreSQL change data capture through WAL, logical decoding, publications and replication slots, including setup choices and production risks.

1Postgres
2WAL
3CDC reader
4Stream
5Analytics

Direct answer

Postgres change data capture (CDC) is the process of reading committed row changes from PostgreSQL's write-ahead log and turning them into an ordered stream that another system can consume. The normal production path uses logical decoding, a publication that selects tables, a replication slot that tracks consumer progress and a CDC client such as Debezium or a managed connector. The stream can then feed Kafka, another database, a warehouse or a real-time analytics platform.

CDC is useful when a system needs inserts, updates and deletes sooner than a scheduled export can provide them. It avoids repeatedly scanning large operational tables and catches changes that timestamp polling often misses, especially deletes. It also creates operational responsibilities: retained WAL can fill storage, schema changes need a policy, consumers may see duplicates after recovery, and failover must preserve a valid position.

PostgreSQL provides the log and logical-decoding infrastructure. It does not, by itself, provide the full delivery pipeline, schema registry, retries, monitoring or analytical model. Those responsibilities belong to the connector and downstream systems.

How Postgres CDC works from commit to consumer

PostgreSQL writes changes to the write-ahead log, or WAL, before modified data pages are considered durable. Physical replication uses those records to reproduce storage-level changes. Logical decoding interprets the WAL as higher-level changes that a client can understand, such as an insert into a named table or a delete identified by a key.

A transaction is not emitted as a set of final changes until PostgreSQL knows it committed. This matters because downstream consumers should not build state from work that was rolled back. The CDC client connects through PostgreSQL's replication protocol, reads from a logical replication slot, converts the records to an event format and acknowledges progress. The slot retains the position, expressed as a log sequence number or LSN, needed by that consumer.

The important boundary is acknowledgement. If the client has not confirmed that it safely processed a position, PostgreSQL may retain the associated WAL so it can be read later. That protects recoverability, but a stalled slot can retain enough WAL to exhaust disk. CDC therefore changes “Did the export run?” into a more continuous operational question: “Is the slot advancing, and how far behind is the destination?”

Publications, slots, output plugins and connectors

A publication defines which tables and operations are eligible for logical replication. It is a selection rule, not a queue. A replication slot stores the consumer's progress and prevents required WAL from being discarded too early. An output plugin converts logical changes into a protocol. PostgreSQL's built-in pgoutput plugin is the common choice for modern connectors and native logical replication.

The connector is the active client. Debezium, for example, reads the logical stream, creates structured change events and normally publishes them to Kafka topics. Managed services wrap similar mechanics with hosted operation, destination integrations and monitoring. Each independent consumer should have its own slot. Sharing one slot between competing connector instances can divide events rather than broadcast a complete copy to both.

Initial state needs separate thought. A stream begins at a position; it does not magically contain every row that existed before capture started. Many tools take an initial snapshot, establish a consistent position and then continue with live changes. A production plan must document how that snapshot is created, how writes during the snapshot are reconciled and how a full rebuild would work.

What must be enabled in PostgreSQL?

The server needs wal_level = logical, enough max_replication_slots capacity for the planned consumers and enough max_wal_senders capacity for replication connections. The connector identity needs replication privileges and access to the captured tables. With pgoutput, it also needs a publication, whether created automatically or managed explicitly.

Hosted PostgreSQL changes the controls, not the fundamentals. On Amazon RDS for PostgreSQL, AWS documents enabling the static rds.logical_replication parameter and rebooting the instance for it to take effect. Teams should test the change in a representative environment, because enabling logical replication can alter WAL generation and the parameter group also influences slot and sender capacity.

Tables need a stable way to identify rows. Primary keys make update and delete handling much clearer. If a table has no suitable key, REPLICA IDENTITY FULL can include the old values of all columns, but that increases WAL volume and should be an explicit per-table decision rather than a blanket switch.

What a Postgres change stream contains

A useful CDC envelope normally includes the operation, source schema and table, a transaction or log position, source timestamp and before/after row values where available. Connector-specific metadata may also identify the snapshot phase and schema version. Downstream consumers should preserve the source position even if they reshape the row, because it is valuable for ordering, deduplication and incident investigation.

Updates are not ordinary append-only business events. Five updates to one order can produce five versions of the same logical entity. Deletes may arrive as delete records or tombstones depending on the connector and serialization. An analytical destination therefore needs a declared model: keep the full change history, maintain only the latest row state, or support both through separate raw and serving layers.

Do not assume commit order is identical to every business-time field. An application may write an old updated_at value, clocks can disagree and bulk jobs can modify historical records. Prefer the WAL/connector position for transport ordering and use domain timestamps for domain questions.

CDC is safer than polling only when it is operated

Polling an indexed watermark column is simpler when tables are append-only, deletes do not matter and freshness measured in minutes or hours is acceptable. CDC earns its complexity when updates and deletes matter, data changes continuously, repeated scans create load or several downstream consumers need a durable event stream.

The major production risks are measurable. Monitor retained WAL bytes, slot activity, source-to-destination lag, event throughput, rejected schemas, connector restarts, duplicate rate and reconciliation differences. Set alerts before the database is close to its storage limit. A quiet application also needs heartbeats so the connector can report progress and operators can distinguish “no changes” from “not reading.”

PostgreSQL documentation explains that after a crash a logical slot can resume from an earlier persisted position and emit some recent changes again. That is why robust consumers are idempotent. The pipeline should prefer a duplicate it can remove over an event it can never recover.

From Postgres changes to real-time analytics

A common architecture is Postgres → CDC connector → Kafka-compatible topic → analytical destination. Keep the raw change envelope long enough to replay and debug it. Then project it into a query-friendly table that treats keys, versions and deletes consistently. Analytical endpoints should query prepared state rather than decoding connector envelopes on every request.

Tinybird documents this pattern for keeping an analytical system current while PostgreSQL remains the source of truth. Its PostgreSQL migration guide describes consuming CDC through Kafka-compatible infrastructure, storing raw events and modeling updates and deletes before serving APIs. The companion practical Postgres CDC guide shows a Confluent-based implementation.

For new events that exist only for analytics—clickstream, product usage, logs or telemetry—writing them directly to an event ingestion API may be cleaner than first inserting them into PostgreSQL and extracting them again. CDC is best for changes whose source of truth genuinely is the transactional database.

Postgres CDC implementation checklist

  • Define the tables, operations, freshness target and recovery point objective.
  • Confirm primary keys or choose replica identity per table.
  • Enable logical WAL and size slots, senders and storage headroom.
  • Create a dedicated connector identity, publication and unique slot.
  • Plan the initial snapshot and the repeatable rebuild procedure.
  • Preserve source position and make every downstream write idempotent.
  • Model updates, deletes and schema evolution before serving analytics.
  • Monitor WAL retention, lag, throughput, failures and reconciliation counts.
  • Test connector outage, destination outage and database failover.

FAQ

Does PostgreSQL have native CDC?

PostgreSQL provides WAL, logical decoding, publications and replication slots—the core database capabilities used for log-based CDC. A connector or replication client is still needed to consume, deliver and operate the change stream.

Does CDC slow down Postgres?

It avoids repeated table scans, but it is not free. Logical WAL generation, replication connections, replica identity choices and stalled-slot retention all have costs. Measure them under realistic write volume.

Can Postgres CDC lose or duplicate events?

A well-operated slot protects unread WAL, but failures and incorrect slot handling can still cause gaps. Recovery can also replay recent events. Use durable offsets, idempotent consumers, monitoring and reconciliation rather than relying on a label such as “exactly once.”

Sources checked September 5, 2026: PostgreSQL documentation on logical decoding concepts; Debezium's PostgreSQL connector reference; AWS guidance for logical replication on RDS for PostgreSQL; and Tinybird's PostgreSQL integration guidance.