CloudBeaver on ECS: Why Password Authentication Fails Intermittently

The CloudBeaver ECS tasks started crash-looping yesterday. We spent a few hours debugging an intermittent “password authentication failed” error in CloudBeaver running on ECS with an RDS PostgreSQL backend. The root cause turned out to be an undocumented interaction between CloudBeaver’s config parser and special characters in auto-generated passwords. Here’s the full investigation.

The Symptom

The RDS PostgreSQL logs showed a puzzling pattern — from the same ECS task IP, some connections succeeded and some failed:

2026-08-12 03:00:05 UTC:192.x.x.x(47986):xyz@cloudbeaver: connection authenticated: identity="xyz" method=md5
2026-08-12 03:00:05 UTC:192.x.x.x(47986):xyz@cloudbeaver: connection authorized: user=xyz database=cloudbeaver SSL enabled
2026-08-12 03:00:06 UTC:192.x.x.x(47998):xyz@cloudbeaver: connection authenticated: identity="xyz" method=md5
2026-08-12 03:00:06 UTC:192.x.x.x(47998):xyz@cloudbeaver: connection authorized: user=xyz database=cloudbeaver SSL enabled
2026-08-12 03:00:06 UTC:192.x.x.x(48000):xyz@cloudbeaver: FATAL: password authentication failed for user "xyz"

Same IP, same second — two connections authenticated successfully, then a third failed. This ruled out wrong credentials across the board.

The Investigation

Step 1: Identify the Two Connection Types

Looking at the ECS task logs, the successful connections were followed by CloudBeaver’s internal metadata queries:

INSERT INTO qm.QM_DBEAVER_HOST (MAC_ADDRESS, HOST_NAME, IP_ADDRESS, UPDATE_TIME) VALUES($1,$2,$3,CURRENT_TIMESTAMP)
INSERT INTO qm.QM_DBEAVER_DEPLOYMENT (HOST_ID, INSTALL_PATH, PORT_NUMBER, UPDATE_TIME) VALUES($1,$2,$3,CURRENT_TIMESTAMP)
INSERT INTO qm.QM_DBEAVER_WORKSPACE (HOST_ID, WORKSPACE_PATH, UPDATE_TIME) VALUES($1,$2,CURRENT_TIMESTAMP)
INSERT INTO qm.QM_DBEAVER_RUN (DEPLOY_ID, VERSION_ID, WORKSPACE_ID, USER_ID, START_TIME) VALUES($1,$2,$3,$4,CURRENT_TIMESTAMP)

These are CloudBeaver’s startup registration queries — it registers itself in its own metadata schema (qm.*). This is the management database connection defined in cloudbeaver.conf, which reads the password from an environment variable. This connection works.

The failing connection came immediately after — a separate connection attempt using a different password source.

Step 2: Understand CloudBeaver’s Config Hierarchy

CloudBeaver has two config layers:

The official documentation states:

Use /opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf when the server is already running — it has higher priority and overrides settings from cloudbeaver.conf.

(Reference: CloudBeaver Server Configuration)

Step 3: The Runtime Conf Override

In our setup, the workspace is stored on S3 (with versioning enabled). I found that .cloudbeaver.runtime.conf contained a database password field that was overriding the environment variable reference in cloudbeaver.conf.

The flow:

1. cloudbeaver.conf says password: "${CLOUDBEAVER_DB_PASSWORD}" → reads env var → correct password → management DB connects ✅
2. .cloudbeaver.runtime.conf in S3 has the password stored as a literal string. If the password contains $, {, or }, the config parser misinterprets it as a variable reference — for example, treating it as ${CLOUDBEAVER_DB_PASSWORD:default-password} if no matching environment variable is detected → mangled password → connection fails ❌

But why was the stored password wrong?

Step 4: The Root Cause — Variable Substitution in Passwords

CloudBeaver’s config parser uses ${...} syntax for variable substitution. This substitution is always active when reading config files — it cannot be disabled for server-level configuration.

Our password was generated by AWS Secrets Manager using GenerateSecretString in CDK:

generateSecretString: {
  secretStringTemplate: JSON.stringify({ username: 'xyz' }),
  generateStringKey: 'password',
  passwordLength: 32,
  includeSpace: false,
  excludeCharacters: '"@/\\\'',
}

The generated password contained $, {, or } characters. When CloudBeaver first wrote this password to .cloudbeaver.runtime.conf, the literal value was stored correctly. However, when CloudBeaver read it back on the next startup, the config parser’s variable substitution logic broke the password.

The parser treats ${...} as an environment variable reference — but the issue is not limited to the full ${...} pattern. Even a single } character anywhere in the password can break the parser, because it interprets it as the closing delimiter of a substitution expression that was never opened, corrupting the parsed value. Similarly, a lone $ or { can trigger unexpected parsing behaviour.

This is an undocumented behaviour. CloudBeaver does not warn about or escape these characters in passwords stored in its runtime configuration files.

The Fix

Immediate fix: Manually fix the .cloudbeaver.runtime.conf file in the S3 workspace bucket by removing $, {, } characters from the password entry if there are any, then restart the ECS task.

Permanent fix: Update the CDK GenerateSecretString to exclude characters that trigger variable substitution:

generateSecretString: {
  secretStringTemplate: JSON.stringify({ username: 'xyz' }),
  generateStringKey: 'password',
  passwordLength: 32,
  includeSpace: false,
  excludeCharacters: '"@/\\\'${}',  // added ${}
}

IMPORTANT: Changing excludeCharacters on an existing secret will regenerate the password. You must manually update the user’s password in the database.

Then force a new ECS deployment to pick up the new value.

(Reference: AWS::SecretsManager::Secret CloudFormation documentation)

Key Takeaways

1. Always exclude $, {, and } from generated passwords when the consuming application uses ${...} variable substitution in its config files. This applies to CloudBeaver, Spring Boot, and many Java-based applications.
2. CloudBeaver’s .cloudbeaver.runtime.conf overrides cloudbeaver.conf and persists in the workspace (S3/EFS). A stale or mangled password there will survive ECS task restarts and redeployments.
3. CloudBeaver opens two separate connection pools to the same database — one for its internal management schema (configured via env var) and one that reads from the runtime conf. This is why you see mixed success/failure from the same container.
4. S3 bucket versioning on the workspace bucket creates a new version of .cloudbeaver.runtime.conf on every startup. Useful for debugging — compare versions to find when the password was last written.

References

• CloudBeaver Server Configuration (GitHub Wiki)
• CloudBeaver User Credentials Storage
• AWS::SecretsManager::Secret CloudFormation Resource
• DBeaver Issue #6034 – Special characters in database password