2017-05-02 16:26:32 -04:00
---
2020-01-17 19:18:09 -05:00
layout: docs
page_title: PostgreSQL - Database - Secrets Engines
2017-05-02 16:26:32 -04:00
description: |-
2017-09-20 16:05:00 -04:00
PostgreSQL is one of the supported plugins for the database secrets engine.
This plugin generates database credentials dynamically based on configured
roles for the PostgreSQL database.
2017-05-02 16:26:32 -04:00
---
2023-06-30 19:22:07 -04:00
# PostgreSQL database secrets engine
2017-05-03 01:24:31 -04:00
2017-09-20 16:05:00 -04:00
PostgreSQL is one of the supported plugins for the database secrets engine. This
plugin generates database credentials dynamically based on configured roles for
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk
* fix issue of storing pointers and now copy
* update to use copy structure
* Remove file, put Item struct def. into other file
* add link
* clean up docs
* refactor internal data structure to hide heap method implementations. Other cleanup after feedback
* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods
* updates after feedback
* refactoring/renaming
* guard against pushing a nil item
* minor updates after feedback
* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC
* Initial Combined database backend implementation of static accounts and automatic rotation
* vendor updates
* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation
* add lock and setup of rotation queue
* vendor the queue
* rebase on new method signature of queue
* remove mongo tests for now
* update default role sql
* gofmt after rebase
* cleanup after rebasing to remove checks for ErrNotFound error
* rebase cdcr-priority-queue
* vendor dependencies with 'go mod vendor'
* website database docs for Static Role support
* document the rotate-role API endpoint
* postgres specific static role docs
* use constants for paths
* updates from review
* remove dead code
* combine and clarify error message for older plugins
* Update builtin/logical/database/backend.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups from feedback
* code and comment cleanups
* move db.RLock higher to protect db.GenerateCredentials call
* Return output with WALID if we failed to delete the WAL
* Update builtin/logical/database/path_creds_create.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* updates after running 'make fmt'
* update after running 'make proto'
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update comment and remove and rearrange some dead code
* Update website/source/api/secret/databases/index.html.md
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups after review
* Update sdk/database/dbplugin/grpc_transport.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* code cleanup after feedback
* remove PasswordLastSet; it's not used
* document GenerateCredentials and SetCredentials
* Update builtin/logical/database/path_rotate_credentials.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* wrap pop and popbykey in backend methods to protect against nil cred rotation queue
* use strings.HasPrefix instead of direct equality check for path
* Forgot to commit this
* updates after feedback
* re-purpose an outdated test to now check that static and dynamic roles cannot share a name
* check for unique name across dynamic and static roles
* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials
* remove commented out code
* refactor to have loadstaticwals filter out wals for roles that no longer exist
* return error if nil input given
* add nil check for input into setStaticAccount
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* add constant for queue tick time in seconds, used for comparrison in updates
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* code cleanup after review
* remove misplaced code comment
* remove commented out code
* create a queue in the Factory method, even if it's never used
* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type
* document new method
* move rotation things into a specific file
* rename test file and consolidate some static account tests
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update code comments, method names, and move more methods into rotation.go
* update comments to be capitalized
* remove the item from the queue before we try to destroy it
* findStaticWAL returns an error
* use lowercase keys when encoding WAL entries
* small cleanups
* remove vestigial static account check
* remove redundant DeleteWAL call in populate queue
* if we error on loading role, push back to queue with 10 second backoff
* poll in initqueue to make sure the backend is setup and can write/delete data
* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false
* add code comments on read-only loop
* code comment updates
* re-push if error returned from find static wal
* add locksutil and acquire locks when pop'ing from the queue
* grab exclusive locks for updating static roles
* Add SetCredentials and GenerateCredentials stubs to mockPlugin
* add a switch in initQueue to listen for cancelation
* remove guard on zero time, it should have no affect
* create a new context in Factory to pass on and use for closing the backend queue
* restore master copy of vendor dir
2019-06-19 15:45:39 -04:00
the PostgreSQL database, and also supports [Static
2023-01-25 19:12:15 -05:00
Roles](/vault/docs/secrets/databases#static-roles).
2017-05-03 01:24:31 -04:00
2023-01-25 19:12:15 -05:00
See the [database secrets engine](/vault/docs/secrets/databases) docs for
2017-09-20 16:05:00 -04:00
more information about setting up the database secrets engine.
2017-05-03 01:24:31 -04:00
2022-06-09 17:37:14 -04:00
The PostgreSQL secrets engine uses [pgx][pgxlib], the same database library as the
2023-01-25 19:12:15 -05:00
[PostgreSQL storage backend](/vault/docs/configuration/storage/postgresql). Connection string
2022-06-09 17:37:14 -04:00
options, including SSL options, can be found in the [pgx][pgxlib] and
[PostgreSQL connection string][pg_conn_docs] documentation.
2020-03-20 11:24:02 -04:00
2020-05-01 17:05:05 -04:00
## Capabilities
2020-12-17 16:53:33 -05:00
2024-10-07 11:51:30 -04:00
| Plugin Name | Root Credential Rotation | Dynamic Roles | Static Roles | Username Customization | Credential Types |
| ---------------------------- | ------------------------ | ------------- | ------------ | ---------------------- | ---------------------------- |
| `postgresql-database-plugin` | Yes | Yes | Yes | Yes (1.7+) | password, gcp_iam |
2020-05-01 17:05:05 -04:00
2017-09-20 16:05:00 -04:00
## Setup
2017-05-03 01:24:31 -04:00
2020-01-17 19:18:09 -05:00
1. Enable the database secrets engine if it is not already enabled:
2017-05-03 01:24:31 -04:00
2022-08-18 15:38:30 -04:00
```shell-session
2017-09-20 16:05:00 -04:00
$ vault secrets enable database
Success! Enabled the database secrets engine at: database/
```
2017-05-03 01:24:31 -04:00
2017-09-20 16:05:00 -04:00
By default, the secrets engine will enable at the name of the engine. To
enable the secrets engine at a different path, use the `-path` argument.
2017-05-03 01:24:31 -04:00
2020-01-17 19:18:09 -05:00
1. Configure Vault with the proper plugin and connection information:
2017-05-03 01:24:31 -04:00
2022-08-18 15:38:30 -04:00
```shell-session
2017-09-20 16:05:00 -04:00
$ vault write database/config/my-postgresql-database \
2022-08-18 15:38:30 -04:00
plugin_name="postgresql-database-plugin" \
2017-09-20 16:05:00 -04:00
allowed_roles="my-role" \
2022-09-06 08:10:50 -04:00
connection_url="postgresql://{{username}}:{{password}}@localhost:5432/database-name" \
2020-05-01 17:05:05 -04:00
username="vaultuser" \
2023-03-21 15:12:53 -04:00
password="vaultpass" \
password_authentication="scram-sha-256"
2017-09-20 16:05:00 -04:00
```
2017-05-03 01:24:31 -04:00
2020-01-17 19:18:09 -05:00
1. Configure a role that maps a name in Vault to an SQL statement to execute to
create the database credential:
2017-05-03 01:24:31 -04:00
2022-08-18 15:38:30 -04:00
```shell-session
2017-09-20 16:05:00 -04:00
$ vault write database/roles/my-role \
2022-08-18 15:38:30 -04:00
db_name="my-postgresql-database" \
2017-09-20 16:05:00 -04:00
creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
default_ttl="1h" \
max_ttl="24h"
Success! Data written to: database/roles/my-role
```
2017-05-03 01:24:31 -04:00
2017-09-20 16:05:00 -04:00
## Usage
After the secrets engine is configured and a user/machine has a Vault token with
the proper permission, it can generate credentials.
2020-01-17 19:18:09 -05:00
1. Generate a new credential by reading from the `/creds` endpoint with the name
of the role:
2017-09-20 16:05:00 -04:00
2022-08-18 15:38:30 -04:00
```shell-session
2017-09-20 16:05:00 -04:00
$ vault read database/creds/my-role
Key Value
--- -----
lease_id database/creds/my-role/2f6a614c-4aa2-7b19-24b9-ad944a8d4de6
lease_duration 1h
lease_renewable true
2020-10-19 17:58:09 -04:00
password SsnoaA-8Tv4t34f41baD
username v-vaultuse-my-role-x
2017-09-20 16:05:00 -04:00
```
2017-05-03 01:24:31 -04:00
2024-10-07 11:51:30 -04:00
## Rootless Configuration and Password Rotation for Static Roles
<EnterpriseAlert product="vault" />
The PostgreSQL secrets engine supports using Static Roles and its password rotation mechanisms with a Rootless
DB connection configuration. In this workflow, a static DB user can be onboarded onto Vault's static role rotation
mechanism without the need of privileged root accounts to configure the connection. Instead of using a single root
connection, multiple dedicated connections to the DB are made for each static role. This workflow does not support
dynamic roles/credentials.
~> Note: It is **highly recommended** that the DB users being onboarded as static roles
have the minimum set of privileges. Each static role will open a new connection into the DB.
Granting minimum privileges to the DB users being onboarded ensures that multiple
highly-privileged connections to an external system are not being made.
~> Note: Out-of-band password rotations will cause Vault to be out of sync with the state of
the DB user, and will require manually updating the user's password in the external PostgreSQL
DB in order to resolve any errors encountered during rotation.
1. Enable the database secrets engine if it is not already enabled:
```shell-session
$ vault secrets enable database
Success! Enabled the database secrets engine at: database/
```
By default, the secrets engine will enable at the name of the engine. To
enable the secrets engine at a different path, use the `-path` argument.
1. Configure connection to DB without root credentials and enable the rootless
workflow by setting the `self_managed` parameter:
```shell-session
$ vault write database/config/my-postgresql-database \
plugin_name="postgresql-database-plugin" \
allowed_roles="my-role" \
connection_url="postgresql://{{username}}:{{password}}@localhost:5432/database-name" \
self_managed=true
```
1. Configure a static role that creates a dedicated connection to a user in the DB with
the `self_managed_password` parameter:
```shell-session
$ vault write database/static-roles/my-static-role \
db_name="my-postgresql-database" \
username="staticuser" \
self_managed_password="password" \
rotation_period="1h"
```
1. Read static credentials:
```shell-session
$ vault read database/static-creds/static-test
Key Value
--- -----
last_vault_rotation 2024-09-11T14:15:13.764783-07:00
password XZY42BVc-UO5bMsbgxrW
rotation_period 1h
ttl 59m55s
username staticuser
```
2024-09-09 13:34:16 -04:00
## Client x509 certificate authentication
This plugin supports using PostgreSQl's [x509 Client-side Certificate Authentication](https://www.postgresql.org/docs/16/libpq-ssl.html#LIBPQ-SSL-CLIENTCERT).
To use this authentication mechanism, configure the plugin to consume the
PEM-encoded TLS data inline from a file on disk by prefixing with the "@"
symbol. This is useful in environments where you do not have direct access to
the machine that is hosting the Vault server. For example:
```shell-session
$ vault write database/config/my-postgresql-database \
plugin_name="postgresql-database-plugin" \
allowed_roles="my-role" \
connection_url="postgresql://{{username}}:{{password}}@localhost:5432/database-name?sslmode=verify-full" \
username="vaultuser" \
private_key=@/path/to/client.key \
tls_certificate=@/path/to/client.pem \
tls_ca=@/path/to/client.ca
```
Note: `private_key`, `tls_certificate`, and `tls_ca` map to [`sslkey`][sslkey_docs],
[`sslcert`][sslcert_docs], and [`sslrootcert`][sslrootcert_docs] configuration
options from PostgreSQL with the exception that the Vault parameters are the
contents of those files, not filenames.
[sslkey_docs]: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLKEY
[sslcert_docs]: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLCERT
[sslrootcert_docs]: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLROOTCERT
Alternatively, you can configure certificate authentication in environments
where the TLS certificate data is present on the machine that is running the
Vault server process. Set `sslmode` to be any of the applicable values as
outlined in the PostgreSQL documentation and set the SSL credentials in the
`sslrootcert`, `sslcert` and `sslkey` connection parameters as paths to files.
For example:
```shell-session
$ export SSL="sslmode=verify-full&sslrootcert=/path/to/ca.pem&sslcert=/path/to/client.pem&sslkey=/path/to/client.key"
$ vault write database/config/my-postgresql-database \
plugin_name="postgresql-database-plugin" \
allowed_roles="my-role" \
connection_url="postgresql://{{username}}:{{password}}@localhost:5432/database-name?sslmode=verify-full&${SSL}" \
username="vaultuser"
```
2017-05-03 01:24:31 -04:00
## API
The full list of configurable options can be seen in the [PostgreSQL database
2023-01-25 19:12:15 -05:00
plugin API](/vault/api-docs/secret/databases/postgresql) page.
2017-05-03 01:24:31 -04:00
2017-09-20 16:05:00 -04:00
For more information on the database secrets engine's HTTP API please see the
2023-01-25 19:12:15 -05:00
[Database secrets engine API](/vault/api-docs/secret/databases) page.
2022-06-09 17:37:14 -04:00
[pgxlib]: https://pkg.go.dev/github.com/jackc/pgx/stdlib
[pg_conn_docs]: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING
2023-09-08 17:03:50 -04:00
## Authenticating to Cloud DBs via IAM
### Google Cloud
Aside from IAM roles denoted by [Google's CloudSQL documentation](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users#creating-a-database-user),
the following SQL privileges are needed by the service account's DB user for minimum functionality with Vault.
Additional privileges may be needed depending on the SQL configured on the database roles.
```sql
-- Enable service account to create roles within DB
ALTER USER "<YOUR DB USERNAME>" WITH CREATEROLE;
```
### Setup
1. Enable the database secrets engine if it is not already enabled:
```shell-session
$ vault secrets enable database
Success! Enabled the database secrets engine at: database/
```
By default, the secrets engine will enable at the name of the engine. To
enable the secrets engine at a different path, use the `-path` argument.
1. Configure Vault with the proper plugin and connection information. Here you can explicitly enable GCP IAM authentication
and use [Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc#how-to) to authenticate:
```shell-session
$ vault write database/config/my-postgresql-database \
plugin_name="postgresql-database-plugin" \
allowed_roles="my-role" \
connection_url="host=project:us-west1:mydb user=test-user@project.iam dbname=postgres sslmode=disable" \
2024-06-25 15:17:13 -04:00
use_private_ip="false" \
2023-09-08 17:03:50 -04:00
auth_type="gcp_iam"
```
You can also configure the connection and authenticate by directly passing in the service account credentials
as an encoded JSON string:
```shell-session
$ vault write database/config/my-postgresql-database \
plugin_name="postgresql-database-plugin" \
allowed_roles="my-role" \
connection_url="host=project:region:instance user=test-user@project.iam dbname=postgres sslmode=disable" \
2024-06-25 15:17:13 -04:00
use_private_ip="false" \
2023-09-08 17:03:50 -04:00
auth_type="gcp_iam" \
service_account_json="@my_credentials.json"
```
Once the connection has been configured and IAM authentication is complete, the steps to set up a role and generate
credentials are the same as the ones listed above.