Support for RDS AWS IAM Auth#8084
Conversation
- add IAM token generation for Aurora/RDS PostgreSQL authentication - wire IAM auth configuration through postgres consumers - add tests, docs, and optional RDS CA certificate image support
There was a problem hiding this comment.
Code Review
This pull request introduces opt-in AWS RDS IAM authentication for PostgreSQL connections across several services, utilizing short-lived IAM tokens (SigV4) instead of static passwords. It includes updates to Docker configurations, environment variables, Slonik patches, and the introduction of a connection string provider. The review feedback highlights several critical improvements: avoiding direct access to process.env in favor of Zod-parsed configurations, removing unnecessary type assertions, and eliminating the use of the non-null assertion operator (!) outside of test files to prevent potential runtime errors.
| password: postgres.POSTGRES_PASSWORD, | ||
| ssl: postgres.POSTGRES_SSL === '1', | ||
| awsIamAuthEnabled: postgres.POSTGRES_AWS_IAM_AUTH_ENABLED === '1', | ||
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, |
There was a problem hiding this comment.
Avoid accessing process.env directly. Use the parsed environment variable from configs.base.data instead.
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, | |
| awsRegion: postgres.POSTGRES_AWS_REGION ?? configs.base.data?.AWS_REGION, |
References
- Do not access
process.envdirectly in your code. Define environment variables inpackages/services/*/src/environment.tsusing Zod schemas. (link)
| password: postgres.POSTGRES_PASSWORD, | ||
| ssl: postgres.POSTGRES_SSL === '1', | ||
| awsIamAuthEnabled: postgres.POSTGRES_AWS_IAM_AUTH_ENABLED === '1', | ||
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, |
There was a problem hiding this comment.
Avoid accessing process.env directly. Use the parsed environment variable from configs.base.data instead.
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, | |
| awsRegion: postgres.POSTGRES_AWS_REGION ?? configs.base.data?.AWS_REGION, |
References
- Do not access
process.envdirectly in your code. Define environment variables inpackages/services/*/src/environment.tsusing Zod schemas. (link)
| password: postgres.POSTGRES_PASSWORD, | ||
| ssl: postgres.POSTGRES_SSL === '1', | ||
| awsIamAuthEnabled: postgres.POSTGRES_AWS_IAM_AUTH_ENABLED === '1', | ||
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, |
There was a problem hiding this comment.
Avoid accessing process.env directly. Use the parsed environment variable from configs.base.data instead.
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, | |
| awsRegion: postgres.POSTGRES_AWS_REGION ?? configs.base.data?.AWS_REGION, |
References
- Do not access
process.envdirectly in your code. Define environment variables inpackages/services/*/src/environment.tsusing Zod schemas. (link)
| password: postgres.POSTGRES_PASSWORD, | ||
| ssl: postgres.POSTGRES_SSL === '1', | ||
| awsIamAuthEnabled: postgres.POSTGRES_AWS_IAM_AUTH_ENABLED === '1', | ||
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, |
There was a problem hiding this comment.
Avoid accessing process.env directly. Use the parsed environment variable from configs.base.data instead.
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, | |
| awsRegion: postgres.POSTGRES_AWS_REGION ?? configs.base.data?.AWS_REGION, |
References
- Do not access
process.envdirectly in your code. Define environment variables inpackages/services/*/src/environment.tsusing Zod schemas. (link)
| password: postgres.POSTGRES_PASSWORD, | ||
| ssl: postgres.POSTGRES_SSL === '1', | ||
| awsIamAuthEnabled: postgres.POSTGRES_AWS_IAM_AUTH_ENABLED === '1', | ||
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, |
There was a problem hiding this comment.
Avoid accessing process.env directly. Use the parsed environment variable from configs.base.data instead.
| awsRegion: postgres.POSTGRES_AWS_REGION ?? process.env.AWS_REGION, | |
| awsRegion: postgres.POSTGRES_AWS_REGION ?? configs.base.data?.AWS_REGION, |
References
- Do not access
process.envdirectly in your code. Define environment variables inpackages/services/*/src/environment.tsusing Zod schemas. (link)
| ? () => | ||
| generateRdsIamAuthToken( | ||
| { | ||
| region: env.postgres.awsRegion!, |
| ? () => | ||
| generateRdsIamAuthToken( | ||
| { | ||
| region: env.postgres.awsRegion!, |
There was a problem hiding this comment.
| ? () => | ||
| generateRdsIamAuthToken( | ||
| { | ||
| region: env.postgres.awsRegion!, |
There was a problem hiding this comment.
| ? () => | ||
| generateRdsIamAuthToken( | ||
| { | ||
| region: env.postgres.awsRegion!, |
There was a problem hiding this comment.
| ? () => | ||
| generateRdsIamAuthToken( | ||
| { | ||
| region: env.postgres.awsRegion!, |
There was a problem hiding this comment.
|
@mish-elle Can you please rebase this on the |
Background
Self-hosters running Hive on AWS with Aurora/RDS PostgreSQL currently have no way to use IAM-based authentication for PostgreSQL connections, which forces them to use static passwords.
This PR adds opt-in AWS IAM authentication support for PostgreSQL connections across services that connect to Postgres.
This PR is part of the following issue. We will have separate PRs for each IAM support to help decrease the scope per PR.
Description
RDS IAM token generation logic lives in a new module inside
service-common:service-common/src/iam-rds.ts: Aurora/RDS-specific IAM token generation using@aws-sdk/rds-signer.When Postgres IAM auth is enabled:
Implementation notes:
connectionStringProviderinpackages/internal/postgres/src/connection-string.tsthat returns a fresh connection string on each call.tokenGeneratorfunction is provided, each invocation first callsawait tokenGenerator()to fetch a fresh, short-lived RDS IAM token, then builds the connection string with that token as the password.postgres-database-pools, we updatedcreatePostgresDatabasePoolso thatconnectionParameterscan now be aConnectionStringProviderfunction as well.New environment variables introduced
AWS_REGIONPOSTGRES_AWS_IAM_AUTH_ENABLED1to enable IAM authentication for Aurora/RDS PostgreSQL.POSTGRES_AWS_REGIONAWS_REGION).Environment Variable Validation
When
POSTGRES_AWS_IAM_AUTH_ENABLED=1, environment validation enforces:POSTGRES_SSL=1(RDS IAM requires TLS)POSTGRES_AWS_REGIONorAWS_REGIONmust be setPnpm-lock file generation
Like previous IAM PRs, CI will require maintainer to help generate lockfile will be needed due to our internal constraints. Thank you!
Checklist