Configuration

This guide covers how to configure ClickHouse and Keeper clusters using the ClickHouse operator.

ClickHouse Operator configuration guide

This guide covers how to configure ClickHouse and Keeper clusters using the operator.

ClickHouseCluster configuration

Basic configuration

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: my-cluster
spec:
  replicas: 3           # Number of replicas per shard
  shards: 2             # Number of shards
  keeperClusterRef:
    name: my-keeper     # Reference to KeeperCluster
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 10Gi

Replicas and shards

• Replicas: Number of ClickHouse instances per shard (for high availability)
• Shards: Number of horizontal partitions (for scaling)

spec:
  replicas: 3  # Default: 3
  shards: 2    # Default: 1

A cluster with replicas: 3 and shards: 2 will create 6 ClickHouse pods total.

Keeper integration

Every ClickHouse cluster must reference a KeeperCluster for coordination:

spec:
  keeperClusterRef:
    name: my-keeper
    # namespace: keeper-system  # Optional, defaults to the ClickHouseCluster namespace

When keeperClusterRef.namespace is set, the operator must watch both namespaces. If WATCH_NAMESPACE is configured, include the ClickHouse and Keeper namespaces in that list.

KeeperCluster configuration

apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: my-keeper
spec:
  replicas: 3  # Must be odd: 1, 3, 5, 7, 9, 11, 13, or 15
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 5Gi

Storage configuration

Configure persistent storage:

spec:
  dataVolumeClaimSpec:
    storageClassName: fast-ssd  # Optional: consider your storage class based on the installed CSI
    resources:
      requests:
        storage: 100Gi

Operator can modify existing PVC only if the underlying storage class supports volume expansion.

Pod configuration

Automatic topology spread and affinity

Distribute pods across availability zones:

spec:
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname

Ensure your Kubernetes cluster has enough nodes in different zones to satisfy the spread constraints.

Manual configuration

Arbitrary pod affinity/anti-affinity rules and topology spread constraints can be specified.

spec:
  podTemplate:
    affinity:
      <your-affinity-rules-here>
    topologySpreadConstraints:
      <your-topology-spread-constraints-here>

See API Reference for all supported Pod template options.

Pod disruption budgets

The operator creates a PodDisruptionBudget (PDB) for each cluster so that voluntary disruptions — node drains, rolling upgrades, autoscaler evictions — cannot take down enough pods to lose quorum or break availability.

For ClickHouse clusters with more than one shard, one PDB is created per shard so a disruption in one shard cannot count against another.

Defaults

The operator picks safe defaults based on the cluster size so that a fresh apply already protects against accidental quorum loss.

Resource	Topology	Default PDB
ClickHouseCluster	replicas: 1 (single-replica shard)	maxUnavailable: 1 — disruption is allowed for a single-node cluster so that node drains are not blocked
ClickHouseCluster	replicas: 2+ (multi-replica shard)	minAvailable: 1 — at least one replica per shard must stay up
KeeperCluster	replicas: 1	maxUnavailable: 1 — disruption is allowed for a single-node cluster so that node drains are not blocked
KeeperCluster	replicas: 3+	maxUnavailable: replicas/2 — preserves the RAFT quorum for a 2F+1 cluster (3 replicas tolerate 1 down, 5 replicas tolerate 2 down)

For a 3-shard ClickHouseCluster with replicas: 3, the operator creates three PDBs, one per shard, each with minAvailable: 1.

Overriding the defaults

Use spec.podDisruptionBudget to override either minAvailable or maxUnavailable (exactly one):

spec:
  replicas: 3
  shards: 2
  podDisruptionBudget:
    minAvailable: 2   # keep at least 2 of 3 replicas in every shard up during a disruption

Or the maxUnavailable form, with a percentage:

spec:
  replicas: 5
  podDisruptionBudget:
    maxUnavailable: 40%

Setting both minAvailable and maxUnavailable is rejected by the validating webhook. Pick one — Kubernetes itself does not allow both either.

You can also pass the unhealthyPodEvictionPolicy field through to the generated PDB — useful when you need to allow eviction of pods that are still in NotReady:

spec:
  podDisruptionBudget:
    minAvailable: 2
    unhealthyPodEvictionPolicy: AlwaysAllow

Policies

spec.podDisruptionBudget.policy lets you choose how aggressively the operator manages PDBs:

Policy	Behavior
Enabled (default)	The operator creates and updates the PDB on every reconcile. This is the safe production default.
Disabled	The operator does not create PDBs and deletes any existing ones with matching labels. Useful for development clusters where every voluntary disruption should be allowed.
Ignored	The operator neither creates nor deletes PDBs. Existing PDBs are left alone. Use this when another system (e.g. policy admission, GitOps tool) owns PDB management for you.

Example — disable PDB management completely on a development cluster:

spec:
  podDisruptionBudget:
    policy: Disabled

Example — keep your hand-crafted PDB next to the cluster and stop the operator from touching it:

spec:
  podDisruptionBudget:
    policy: Ignored

Cluster-wide opt-out

PDB management can also be disabled cluster-wide via the operator’s ENABLE_PDB environment variable. With ENABLE_PDB=false, the operator skips the PDB reconcile step for every ClickHouseCluster and KeeperCluster regardless of their spec.podDisruptionBudget.policy, and does not watch PodDisruptionBudget resources at all. The operator’s ServiceAccount therefore does not need RBAC permissions on poddisruptionbudgets.policy/v1, which is useful when running the operator under a restricted ServiceAccount that intentionally omits those permissions.

# in the operator Deployment spec
env:
- name: ENABLE_PDB
  value: "false"

This is intended for environments that ship their own disruption policies (e.g. through Gatekeeper / Kyverno) and want the operator out of the loop entirely.

Container configuration

Custom image

Use a specific ClickHouse image:

spec:
  containerTemplate:
    image:
      repository: clickhouse/clickhouse-server
      tag: "25.12"
    imagePullPolicy: IfNotPresent

Container resources

Configure CPU and memory for ClickHouse containers:

# default values
spec:
  containerTemplate:
    resources:
      requests:
        cpu: "250m"
        memory: "512Mi"
      limits:
        cpu: "1"
        memory: "512Mi"

Environment variables

Add custom environment variables:

spec:
  containerTemplate:
    env:
    - name: CUSTOM_ENV_VAR
      value: "1"

Volume mounts

Add additional volume mounts:

spec:
  containerTemplate:
    volumeMounts:
    - name: custom-config
      mountPath: /etc/clickhouse-server/config.d/custom.xml
      subPath: custom.xml

It is allowed to specify multiple volume mounts to the same mountPath. Operator will create projected volume with all specified mounts.

See API Reference for all supported Container template options.

TLS/SSL configuration

Configure secure endpoints

Pass a reference to a Kubernetes Secret containing TLS certificates to enable secure endpoints

spec:
  settings:
    tls:
      enabled: true
      required: true # Insecure ports are disabled if set
      serverCertSecret:
        name: <certificate-secret-name>

SSL certificate secret format

It is expected that the Secret contains the following keys:

• tls.crt - PEM encoded server certificate
• tls.key - PEM encoded private key
• ca.crt - PEM encoded CA certificate chain

This format is compatible with cert-manager generated certificates.

ClickHouse-Keeper communication over TLS

If KeeperCluster has TLS enabled, ClickHouseCluster would use secure connection to Keeper nodes automatically.

ClickHouseCluster should be able to verify Keeper nodes certificates. If ClickHouseCluster has TLS enabled, is uses ca.crt bundle for verification. Otherwise, default CA bundle is used.

User may provide a custom CA bundle reference:

spec:
    settings:
        tls:
          caBundle:
            name: <ca-certificate-secret-name>
            key: <ca-certificate-key>

External Secret

By default the operator creates and owns a Secret containing the cluster’s internal credentials (interserver password, management password, keeper identity, cluster secret, named-collections key). The Secret is named after the cluster and lives in the cluster’s namespace.

If you want to manage these credentials yourself — for example, sourcing them from HashiCorp Vault, AWS Secrets Manager, or External Secrets Operator — point the operator at a pre-existing Secret using spec.externalSecret:

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 2
  keeperClusterRef:
    name: sample
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 10Gi
  externalSecret:
    name: my-clickhouse-credentials
    policy: Observe

The referenced Secret must reside in the same namespace as the ClickHouseCluster. The operator never deletes a Secret it did not create.

Required keys

The Secret must contain the following keys:

Key	Format	When required
interserver-password	plaintext password	Always
management-password	plaintext password	Always
keeper-identity	clickhouse:<password>	Always
cluster-secret	plaintext password	Always
named-collections-key	hex-encoded 16-byte AES key (32 hex chars)	ClickHouse >= 25.12 only

A complete Secret looks like this:

apiVersion: v1
kind: Secret
metadata:
  name: my-clickhouse-credentials
  namespace: sample
type: Opaque
stringData:
  interserver-password: "a-strong-random-password"
  management-password: "another-strong-password"
  keeper-identity: "clickhouse:keeper-auth-password"
  cluster-secret: "cluster-internal-secret"
  named-collections-key: "0123456789abcdef0123456789abcdef"   # 32 hex chars = 16 bytes

Policy: Observe vs Manage

spec.externalSecret.policy controls how the operator handles missing required keys:

Policy	Behavior on missing keys
Observe (default)	Reconciliation is blocked until every required key is present. The operator reports each missing key — and the format hint for it — via the ExternalSecretValid status condition and a Warning event.
Manage	The operator generates any missing required keys and writes them back to the same Secret. Useful for bootstrapping: create an empty Secret, let the operator fill it, then optionally tighten access. The operator still never deletes the Secret.

Even with policy: Manage the Secret must already exist in the namespace — the operator never creates the Secret itself, it only writes generated keys into an existing one. If the referenced Secret is missing, reconciliation is blocked with the ExternalSecretNotFound reason regardless of policy.

Pick Observe when an external system (Vault, ESO, sealed-secrets, GitOps) is the source of truth and you want the operator to fail loudly on misconfiguration. Pick Manage when you want self-sufficient bootstrapping but still want to retain ownership of the Secret object itself (for example, to back it up).

Status condition and troubleshooting

The operator exposes a ExternalSecretValid condition on ClickHouseCluster.status.conditions. Inspect it when reconciliation looks stuck:

# Plain kubectl — works out of the box
kubectl describe clickhousecluster sample | sed -n '/Conditions:/,$p'

# Same data as YAML
kubectl get clickhousecluster sample -o yaml | sed -n '/conditions:/,/^[^ ]/p'

# Pretty-printed JSON (requires jq)
kubectl get clickhousecluster sample -o jsonpath='{.status.conditions}' | jq

Possible reasons:

reason	Meaning	Fix
ExternalSecretNotFound	The referenced Secret does not exist in the namespace.	Create the Secret, or fix spec.externalSecret.name.
ExternalSecretInvalid	The Secret exists but lacks required keys (only with Observe). The message lists each missing key together with its expected format.	Add the missing keys, or switch to policy: Manage.
ExternalSecretValid	All required keys are present and the operator is using the Secret.	—

The operator requeues reconciliation while the Secret is invalid, so once you add the missing keys the next reconcile picks them up automatically — no need to bounce pods.

The set of required keys depends on the running ClickHouse version. named-collections-key is only validated once the operator’s version probe has detected ClickHouse 25.12 or newer. On older versions the key may be absent from the Secret.

Additional ports

The operator exposes a fixed set of ports on every ClickHouse Pod and its headless Service: 8123 HTTP, 9000 native, 9009 interserver, 9001 management, 9363 Prometheus metrics, and the TLS variants 8443/9440 when TLS is enabled. To make ClickHouse listen on additional protocols — MySQL, PostgreSQL, gRPC, or any custom port — declare them in spec.additionalPorts:

spec:
  additionalPorts:
    - name: mysql
      port: 9004
    - name: postgres
      port: 9005
    - name: grpc
      port: 9100

The operator adds those ports to the Pod’s containerPorts and to the headless Service. The complete example lives at examples/custom_protocols.yaml.

additionalPorts only opens the ports on the Kubernetes side. It does not configure the ClickHouse server to listen on them. You also have to enable the matching protocol in spec.settings.extraConfig.protocols. Without that, the port is open on the Service but nothing inside the pod is answering.

End-to-end example: MySQL wire protocol

To expose ClickHouse over the MySQL wire protocol on port 9004:

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 1
  keeperClusterRef:
    name: sample
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 2Gi

  # 1) Open the port on the Pod and the headless Service.
  additionalPorts:
    - name: mysql
      port: 9004

  # 2) Tell ClickHouse server to actually listen on it.
  settings:
    extraConfig:
      protocols:
        mysql:
          type: mysql
          port: 9004
          description: "MySQL wire protocol"

After applying, verify from inside the cluster:

kubectl exec sample-clickhouse-0-0-0 -- \
  clickhouse-client --port 9004 --query "SELECT 1"

Field constraints

Field	Rule
name	Must match the DNS_LABEL pattern ^[a-z]([-a-z0-9]*[a-z0-9])?$, max 63 characters. Uniqueness is enforced by the CRD as a list-map key.
port	Integer in [1, 65535]. The webhook rejects duplicate port numbers within the list.

Reserved ports and names

The validating webhook rejects additionalPorts entries that would collide with ports the operator binds itself. All TLS-related ports are reserved unconditionally so that flipping spec.settings.tls.enabled later cannot break a previously valid cluster.

Port	Reserved for
8123	HTTP
8443	HTTPS
9000	native TCP
9440	native TLS
9009	interserver
9001	management
9363	Prometheus metrics

The following names are also rejected — they are the operator’s internal protocol-type identifiers (not the human-readable aliases):

Name
http
http-secure
tcp
tcp-secure
interserver
management
prometheus

A rejected request produces an error such as:

spec.additionalPorts[0].port: 8123 is reserved for the operator-managed HTTP port
spec.additionalPorts[0].name: "http" is reserved by the operator

Version probe and upgrade channel

The operator does two independent things with cluster versions:

1. Version probe — a Kubernetes Job that runs the container image once to detect the running ClickHouse / Keeper version. The detected version is recorded in .status.version and used by other reconciliation steps (e.g. the External Secret named-collections key is only required from ClickHouse 25.12).
2. Upgrade channel — a periodic check against the public ClickHouse release feed (https://clickhouse.com/data/version_date.tsv). The operator reports whether a newer version is available via the VersionUpgraded status condition. It never upgrades the cluster on its own — the user is in control of the image tag.

Choosing a release channel

spec.upgradeChannel selects which set of upstream releases the operator compares against. Same field exists on both ClickHouseCluster and KeeperCluster.

spec:
  upgradeChannel: lts   # or "stable", or "25.8", or omitted

Allowed values (validated by the CRD with the pattern ^(lts|stable|\d+\.\d+)?$):

Value	Behavior
empty (default)	The operator proposes only minor updates within the currently-running major.minor line. A cluster on 25.8.3.1 will be told about 25.8.4.x but not 25.9.x.
stable	Tracks the upstream stable channel — the latest release that ClickHouse Inc. flags as stable on the main release line. Receives major upgrades sooner than the lts channel.
lts	Tracks the upstream lts channel — long-term support releases. Receives major upgrades less frequently, with longer support windows.
25.8 (or any <major>.<minor>)	Pins the channel to a specific major.minor line. Major upgrades beyond it are not proposed even if a newer version exists upstream.

For production, pinning the channel to an explicit <major>.<minor> (e.g. 25.8) is generally preferred. It locks the cluster to the intended major release line and lets the operator surface a WrongReleaseChannel warning if any replica somehow drifts onto a different major — which matters especially when the image is referenced by a digest (@sha256:...) rather than by a human-readable tag. The empty default is fine for development clusters where major-version jumps are not a concern.

Status conditions

Two conditions surface the result of the probe and the upgrade check:

Condition	Reason	Meaning
VersionInSync	VersionMatch	All replicas report the same version as the image
VersionInSync	VersionMismatch	Replicas are running different versions. This reason is suppressed during a planned rolling upgrade. It typically surfaces when a mutable image tag has been pinned (for example latest or a bare major like 26.3) and the underlying registry has shifted between pulls, so different replicas ended up on different patches of the same tag.
VersionInSync	VersionPending	Version probe Job has not finished yet
VersionInSync	VersionProbeFailed	Probe Job failed; the operator cannot determine the running version
VersionUpgraded	UpToDate	The cluster is on the latest version available in the selected channel
VersionUpgraded	MinorUpdateAvailable	A newer patch is available in the same major.minor line
VersionUpgraded	MajorUpdateAvailable	A newer major.minor is available within the chosen channel
VersionUpgraded	VersionOutdated	The running version is out of date and will no longer receive fixes from the selected channel — typically because the major line has been dropped from lts or stable upstream
VersionUpgraded	WrongReleaseChannel	The running image does not belong to the selected upgradeChannel. Example: a cluster running 26.5 with upgradeChannel: lts, since 26.5 is not part of the upstream lts line.
VersionUpgraded	UpgradeCheckFailed	The operator could not reach the upstream release feed

Inspect them with:

kubectl get clickhousecluster sample -o yaml | sed -n '/conditions:/,/^[^ ]/p'

Overriding the version probe Job

The probe is implemented as a regular Kubernetes Job. If your cluster has admission policies that require specific Tolerations, node selectors, security contexts, or you want to limit how long completed probe Jobs linger, override the template via spec.versionProbeTemplate:

spec:
  versionProbeTemplate:
    spec:
      ttlSecondsAfterFinished: 600   # delete completed probe Jobs 10 minutes after completion
      template:
        spec:
          nodeSelector:
            kubernetes.io/arch: amd64
          tolerations:
            - key: dedicated
              operator: Equal
              value: clickhouse
              effect: NoSchedule
          containers:
            - name: version-probe
              resources:
                requests:
                  cpu: 50m
                  memory: 64Mi

The container name version-probe is the operator’s default — the entry under containers: matches it by name, so the operator deep-merges the user-provided fields on top of the defaults.

Operator-wide controls

Two flags on the operator manager control the upgrade-check loop globally:

Flag	Default	Effect
--version-update-interval	24h	How often the operator re-fetches the upstream version list
--disable-version-update-checks	false	Disables the upgrade checker entirely. The VersionUpgraded condition is not set, and no outbound HTTP traffic to clickhouse.com is generated

Set --disable-version-update-checks=true in air-gapped environments or when egress to clickhouse.com is not allowed.

ClickHouse settings

Default user password

Set the default user password:

spec:
  settings:
    defaultUserPassword:
      passwordType: <password-type> # Default: password
      <secret|configMap>:
        name: <resource name>
        key: <password>

It isn’t recommended to use ConfigMap to store plain text passwords.

Create the secret:

kubectl create secret generic clickhouse-password --from-literal=password='your-secure-password'

Using ConfigMap for user passwords

You can also use ConfigMap for non-sensitive default passwords:

spec:
  settings:
    defaultUserPassword:
      passwordType: password_sha256_hex
      configMap:
        name: clickhouse-config
        key: default_password

Custom users in configuration

Configure additional users in configuration files.

Create a ConfigMap and Secret for user:

apiVersion: v1
kind: ConfigMap
metadata:
  name: user-config
data:
  reader.yaml: |
    users:
      reader:
        password:
          - '@from_env': READER_PASSWORD
        profile: default
        grants:
          - query: "GRANT SELECT ON *.*"
---
apiVersion: v1
kind: Secret
metadata:
  name: reader-password
data:
  password: "c2VjcmV0LXBhc3N3b3Jk"  # base64("secret-password")

Add custom configuration to ClickHouseCluster:

spec:
  podTemplate:
    volumes:
      - name: reader-user
        configMap:
          name: user-config
  containerTemplate:
    env:
      - name: READER_PASSWORD
        valueFrom:
          secretKeyRef:
            name: reader-password
            key: password
    volumeMounts:
      - mountPath: /etc/clickhouse-server/users.d/
        name: reader-user
        readOnly: true

Database sync

Enable automatic database synchronization for new replicas:

spec:
  settings:
    enableDatabaseSync: true  # Default: true

When enabled, the operator synchronizes Replicated and integration tables to new replicas.

Server logging

Configure the ClickHouse server log through spec.settings.logger. Every field is optional with a safe default, so a cluster you never touch already logs at trace to both the container console and a rotated file on disk.

spec:
  settings:
    logger:
      logToFile: true   # Default: true. Set false to log only to the console
      jsonLogs: false   # Default: false. Set true for structured JSON log lines
      level: trace      # Default: trace
      size: 1000M       # Default: 1000M. Rotate a log file once it reaches this size
      count: 50         # Default: 50. Number of rotated files to keep

Field	Default	Description
logToFile	true	When false, the operator drops the file targets and the server logs only to the container console.
jsonLogs	false	When true, the operator adds formatting.type: json so each line is a JSON object.
level	trace	Log verbosity. One of test, trace, debug, information, notice, warning, error, critical, fatal.
size	1000M	Maximum size of a single log file before rotation.
count	50	Number of rotated log files the server retains.

The operator always keeps console logging on so that kubectl logs works, and layers file logging on top when logToFile is true. A cluster with the defaults renders this logger block:

logger:
  console: true
  level: trace
  log: /var/log/clickhouse-server/clickhouse-server.log
  errorlog: /var/log/clickhouse-server/clickhouse-server.err.log
  size: 1000M
  count: 50

The same spec.settings.logger block applies to a KeeperCluster; the operator writes its files under /var/log/clickhouse-keeper/ instead.

Console logging stays on regardless of logToFile, so kubectl logs keeps working even when you disable file logging. Set jsonLogs: true when you ship logs to a structured log store that parses JSON.

Custom configuration

Embedded extra configuration

Instead of mounting custom configuration files, you can directly specify additional ClickHouse configuration options.

Add custom ClickHouse configuration using extraConfig:

spec:
  settings:
    extraConfig:
      background_pool_size: 20

Useful links:

• YAML configuration examples
• All server settings

Embedded extra users configuration

You can also specify additional ClickHouse users configuration using extraUsersConfig. This is useful for defining users, profiles, quotas, and grants directly in the cluster specification.

spec:
  settings:
    extraUsersConfig:
      users:
        analyst:
          password:
            - '@from_env': ANALYST_PASSWORD
          profile: "readonly"
          quota: "default"
      profiles:
        readonly:
          readonly: 1
          max_memory_usage: 10000000000
      quotas:
        default:
          interval:
            duration: 3600
            queries: 1000
            errors: 100

The extraUsersConfig is stored in k8s ConfigMap object. Avoid plain text secrets there.

See documentation for all supported ClickHouse users configuration options.

Configuration example

Complete configuration example:

apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: sample
spec:
  replicas: 3
  dataVolumeClaimSpec:
    storageClassName: <storage-class-name>
    resources:
      requests:
        storage: 10Gi
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname
  containerTemplate:
    resources:
      requests:
        cpu: "2"
        memory: "4Gi"
      limits:
        cpu: "4"
        memory: "8Gi"
  settings:
    tls:
      enabled: true
      required: true
      serverCertSecret:
        name: <keeper-certificate-secret>
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: default-user-password
data:
  # secret-password
  password: "..." # sha256 hex of the password
---
apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 2
  dataVolumeClaimSpec:
    storageClassName: <storage-class-name>
    resources:
      requests:
        storage: 200Gi
  keeperClusterRef:
    name: sample
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname
  settings:
    tls:
      enabled: true
      required: true
      serverCertSecret:
        name: clickhouse-cert
    defaultUserPassword:
      passwordType: password_sha256_hex
      configMap:
        key: password
        name: default-password