pgspider_ext
pgspider_ext : foreign-data wrapper for remote PGSpider servers
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 8540 | pgspider_ext
|
pgspider_ext
|
1.3.0 |
FDW
|
PostgreSQL
|
C
|
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
--s-d-r
|
No
|
Yes
|
No
|
Yes
|
yes
|
no
|
| Relationships | |
|---|---|
| See Also | plproxy
wrappers
multicorn
postgres_fdw
citus
mysql_fdw
oracle_fdw
mongo_fdw
|
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PGDG
|
1.3.0 |
18
17
16
15
14
|
pgspider_ext |
- |
| RPM | PGDG
|
1.3.0 |
18
17
16
15
14
|
pgspider_ext_$v |
- |
| DEB | PIGSTY
|
1.3.0 |
18
17
16
15
14
|
postgresql-$v-pgspider-ext |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
el8.x86_64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
el8.aarch64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
el9.x86_64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
el9.aarch64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
el10.x86_64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
el10.aarch64
|
MISS
|
PGDG 1.3.0
|
PGDG 1.3.0
|
PGDG 1.3.0
|
MISS
|
d12.x86_64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
d12.aarch64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
d13.x86_64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
d13.aarch64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
u22.x86_64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
u22.aarch64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
u24.x86_64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
u24.aarch64
|
MISS
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
PIGSTY 1.3.0
|
MISS
|
Source
pig build pkg pgspider_ext; # build debInstall
Make sure PGDG repo available:
pig repo add pgdg -u # add pgdg repo and update cacheInstall this extension with pig:
pig install pgspider_ext; # install via package name, for the active PG version
pig install pgspider_ext -v 17; # install for PG 17
pig install pgspider_ext -v 16; # install for PG 16
pig install pgspider_ext -v 15; # install for PG 15Create this extension with:
CREATE EXTENSION pgspider_ext;Usage
pgspider_ext: Foreign data wrapper for remote PGSpider servers
PGSpider Extension turns PostgreSQL into a distributed query engine by creating virtual partitioned tables that transparently query data across multiple remote nodes in parallel.
Setup Child Servers
First, create servers for each data source using their respective FDWs:
CREATE EXTENSION pgspider_ext;
CREATE EXTENSION postgres_fdw;
CREATE SERVER pgsrv1 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5433', dbname 'postgres');
CREATE SERVER pgsrv2 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5434', dbname 'postgres');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv1
OPTIONS (user 'user', password 'pass');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv2
OPTIONS (user 'user', password 'pass');Create Child Foreign Tables
CREATE FOREIGN TABLE t1_pg1_child (i int, t text)
SERVER pgsrv1 OPTIONS (table_name 't1');
CREATE FOREIGN TABLE t1_pg2_child (i int, t text)
SERVER pgsrv2 OPTIONS (table_name 't1');Create PGSpider Partitioned Table
Create a PGSpider server and a partitioned parent table with an extra partition key column:
CREATE SERVER spdsrv FOREIGN DATA WRAPPER pgspider_ext;
CREATE USER MAPPING FOR CURRENT_USER SERVER spdsrv;
CREATE TABLE t1 (i int, t text, node text)
PARTITION BY LIST (node);
CREATE FOREIGN TABLE t1_pg1 PARTITION OF t1
FOR VALUES IN ('node1') SERVER spdsrv;
CREATE FOREIGN TABLE t1_pg2 PARTITION OF t1
FOR VALUES IN ('node2') SERVER spdsrv
OPTIONS (child_name 't1_pg2_child');By default, PGSpider finds child tables using the pattern [parent_table_name]_child. Use child_name to specify a different name.
Query Across All Nodes
SELECT * FROM t1;
i | t | node
----+---+-------
10 | a | node1
11 | b | node1
20 | c | node2
21 | d | node2Queries automatically fan out to all child nodes in parallel. WHERE clauses and aggregate functions are pushed down to child nodes:
SET enable_partitionwise_aggregate TO on;
SELECT count(*), node FROM t1 GROUP BY node;Note: Only SELECT operations are supported; INSERT, UPDATE, and DELETE are not supported.