wal2json
wal2json
wal2json : Changing data capture in JSON format
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 9630 | wal2json
|
wal2json
|
2.6 |
ETL
|
BSD 3-Clause
|
C
|
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
--s----
|
No
|
Yes
|
No
|
No
|
no
|
no
|
| Relationships | |
|---|---|
| See Also | pglogical
wal2mongo
decoderbufs
decoder_raw
kafka_fdw
pglogical_origin
pglogical_ticker
pg_failover_slots
|
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PGDG
|
2.6 |
18
17
16
15
14
|
wal2json |
- |
| RPM | PGDG
|
2.6 |
18
17
16
15
14
|
wal2json_$v |
- |
| DEB | PGDG
|
2.6 |
18
17
16
15
14
|
postgresql-$v-wal2json |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
el8.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
el8.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
el9.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.5
|
PGDG 2.5
|
PGDG 2.5
|
el9.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
el10.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
el10.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
d12.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
d12.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
d13.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
d13.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
u22.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
u22.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
u24.x86_64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
u24.aarch64
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
PGDG 2.6
|
Source
Install
Make sure PGDG repo available:
pig repo add pgdg -u # add pgdg repo and update cacheInstall this extension with pig:
pig install wal2json; # install via package name, for the active PG version
pig install wal2json -v 18; # install for PG 18
pig install wal2json -v 17; # install for PG 17
pig install wal2json -v 16; # install for PG 16
pig install wal2json -v 15; # install for PG 15
pig install wal2json -v 14; # install for PG 14This extension does not need CREATE EXTENSION DDL command
Usage
A logical decoding output plugin that produces JSON-formatted change data capture from the PostgreSQL WAL.
Configuration
In postgresql.conf:
wal_level = logical
max_replication_slots = 10
max_wal_senders = 10Using with Streaming Protocol (pg_recvlogical)
# Create a replication slot
pg_recvlogical -d postgres --slot test_slot --create-slot -P wal2json
# Start consuming changes
pg_recvlogical -d postgres --slot test_slot --start -o pretty-print=1 -f -
# Drop the slot when done
pg_recvlogical -d postgres --slot test_slot --drop-slotUsing with SQL Functions
-- Create a logical replication slot
SELECT * FROM pg_create_logical_replication_slot('test_slot', 'wal2json');
-- Peek at changes (does not consume)
SELECT data FROM pg_logical_slot_peek_changes('test_slot', NULL, NULL);
-- Get and consume changes
SELECT data FROM pg_logical_slot_get_changes('test_slot', NULL, NULL,
'pretty-print', '1');
-- Drop the slot
SELECT pg_drop_replication_slot('test_slot');Output Format v1 (JSON per transaction)
{
"change": [
{
"kind": "insert",
"schema": "public",
"table": "my_table",
"columnnames": ["a", "b"],
"columntypes": ["integer", "text"],
"columnvalues": [1, "hello"]
},
{
"kind": "delete",
"schema": "public",
"table": "my_table",
"oldkeys": {
"keynames": ["a"],
"keytypes": ["integer"],
"keyvalues": [1]
}
}
]
}Output Format v2 (JSON per tuple)
Enable with: 'format-version', '2'
Key Parameters
include-xids- add transaction ID (default: false)include-timestamp- add timestamp (default: false)include-schemas- add schema name (default: true)include-types- add column types (default: true)include-pk- add primary key info (default: false)include-lsn- add WAL LSN (default: false)include-not-null- add NOT NULL info (default: false)include-default- add default expressions (default: false)pretty-print- format JSON output (default: false)filter-tables- comma-separated list of tables to includeadd-tables- same as filter-tablesfilter-msg-prefixes- filter logical messages by prefixformat-version- 1 (per-transaction) or 2 (per-tuple)actions- filter by action type: insert, update, delete, truncate
Last updated on