aws_s3
aws_s3
aws_s3 : aws_s3 postgres extension to import/export data from/to s3
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 8800 | aws_s3
|
aws_s3
|
0.0.1 |
FDW
|
Apache-2.0
|
SQL
|
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
----d-r
|
No
|
No
|
No
|
Yes
|
yes
|
no
|
| Relationships | |
|---|---|
| See Also | pg_parquet
hdfs_fdw
file_fdw
duckdb_fdw
wrappers
pg_bulkload
columnar
pg_analytics
|
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PIGSTY
|
0.0.1 |
18
17
16
15
14
|
aws_s3 |
- |
| RPM | PIGSTY
|
0.0.1 |
18
17
16
15
14
|
aws_s3_$v |
- |
| DEB | PIGSTY
|
0.0.1 |
18
17
16
15
14
|
postgresql-$v-aws-s3 |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
el8.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
el8.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
el9.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
el9.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
el10.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
el10.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
d12.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
d12.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
d13.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
d13.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
u22.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
u22.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
u24.x86_64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
u24.aarch64
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
PIGSTY 0.0.1
|
Source
pig build pkg aws_s3; # build rpm/debInstall
Make sure PGDG and PIGSTY repo available:
pig repo add pgsql -u # add both repo and update cacheInstall this extension with pig:
pig install aws_s3; # install via package name, for the active PG version
pig install aws_s3 -v 18; # install for PG 18
pig install aws_s3 -v 17; # install for PG 17
pig install aws_s3 -v 16; # install for PG 16
pig install aws_s3 -v 15; # install for PG 15
pig install aws_s3 -v 14; # install for PG 14Create this extension with:
CREATE EXTENSION aws_s3;Usage
aws_s3: PostgreSQL extension to import/export data from/to S3
Setup Credentials
Configure AWS credentials via PostgreSQL session variables:
SET aws_s3.access_key_id TO 'your_access_key';
SET aws_s3.secret_key TO 'your_secret_key';
SET aws_s3.session_token TO 'optional_session_token'; -- if using temporary credentialsFor local development with LocalStack:
SET aws_s3.endpoint_url TO 'http://localhost:4566';Import Data from S3
CREATE EXTENSION aws_s3;
CREATE TABLE animals (
name text,
age int
);
SELECT aws_s3.table_import_from_s3(
'animals',
'',
'(FORMAT CSV, DELIMITER '','', HEADER true)',
'my-bucket',
'animals.csv',
'us-east-1'
);
SELECT * FROM animals;Parameters: table name, column list (empty string for all), COPY options, S3 bucket, S3 key, AWS region.
Export Data to S3
SELECT * FROM aws_s3.query_export_to_s3(
'SELECT * FROM animals',
'my-bucket',
'export/animals.csv',
'us-east-1',
options := 'FORMAT CSV, HEADER true'
);Parameters: SQL query, S3 bucket, S3 key, AWS region, COPY options.
Features
- Files with
Content-Encoding=gzipmetadata are automatically decompressed during import - Credentials can be passed directly as function arguments or via session variables
- Uses the same COPY format options as PostgreSQL (CSV, TEXT, BINARY, with all related settings like DELIMITER, HEADER, NULL, etc.)
Last updated on