Installation on On-premises / vanilla Kubernetes

Overview

This guide covers the installation of CAST Imaging on an on-premises (or otherwise non-managed) Kubernetes cluster (i.e. a “vanilla” Kubernetes distribution that you operate yourself) using Helm charts.

The same CAST Imaging Helm chart is used for both cloud-managed clusters (EKS / AKS / GKE) and on-premises clusters. The main differences are:

• The K8SProvider variable in values.yaml must be set to K8S.
• The cluster does not provide managed storage classes, load balancer integration, or ingress controllers out of the box. You are responsible for selecting, installing, and operating these building blocks on your cluster (CSI storage drivers, a load-balancer implementation, an ingress / gateway controller, etc.). The CAST Imaging chart simply consumes whatever you have installed.

This page focuses on the on-premises specifics. For everything else (Keycloak parameters, initial start-up configuration, authentication, start/stop scripts, external PostgreSQL, storage locations, etc.) the chart behaves identically to the cloud-managed variants and the same documentation applies.

Requirements

• Access to Docker Hub registry - CAST Imaging Docker images are available as listed in the table below
• A clone of the latest release branch from the Git repository containing the Helm chart scripts: git clone https://github.com/CAST-Extend/com.castsoftware.castimaging-v3.kubernetessetup (to clone an older release, add the “-b x.x.x” flag with the desired release number).
• A valid CAST Imaging License
• A working Kubernetes cluster with, at minimum:
  • Kubernetes version 1.31 or higher
  • CSI storage drivers installed by your cluster administrator, exposing at least one block / disk storage class and one shared file storage class (required for the shared-datadir volume, which must be accessible by multiple Imaging services). CAST Imaging is agnostic to the specific driver (e.g. local-path-provisioner, longhorn, or vsphere-csi for block/disk; nfs-csi for shared file; rook-ceph for either), provided it meets the following permission requirements (setting this up is the cluster administrator’s responsibility): CAST Imaging pods run as non-root users and rely on the Kubernetes fsGroup security context setting to make their volumes writable. The storage classes must therefore support fsGroup. If your driver does not support it (typically an NFS export with root_squash), the volumes must be pre-provisioned writable by the UID/GID of the pods that use them: see Volume permissions when the storage driver does not support fsGroup.
  • A load-balancer implementation such as MetalLB . Without one, services will remain in <pending> state for their external IP. Setting this up is the cluster administrator’s responsibility.
  • (Optional) an ingress / gateway controller (NGINX Ingress, NGINX Gateway Fabric, Istio, etc.) if you want HTTPS access. See Step 3.
• Optional setup choices:
  • Use an external PostgreSQL instance (by default, an embedded PostgreSQL instance is deployed)
  • Enable the analysis node autoscaler to automatically add and remove analysis nodes based on the job queue (≥ 3.6.9, see Analysis node autoscaling)

Docker images

CAST Imaging is provided in a set of Docker images as follows:

Installation process

Before starting the installation, ensure that your Kubernetes cluster is running, all the CAST Imaging docker images are available in the registry and that helm and kubectl are installed on your system.

Step 1 - Cluster prerequisites

• Make sure your Kubernetes cluster is up and that kubectl is configured to point at it:

$ kubectl get nodes

• Install kubectl (if not already installed) - see https://kubernetes.io/docs/tasks/tools/ 

• Install helm:

  • Binary download: https://github.com/helm/helm/releases 
  • Documentation: https://helm.sh/docs/intro/quickstart 

• Ensure your cluster administrator has installed and tested:

  • One CSI block / disk storage class: note its name (referred to below as <your-disk-storage-class>).
  • One shared file storage class supporting ReadWriteMany. Note its name (referred to below as <your-file-storage-class>).
  • A load-balancer implementation (e.g. MetalLB).

You can list the storage classes available on your cluster with:

$ kubectl get storageclass

Step 2 - Prepare and run the CAST Imaging installation

• Review and adjust the parameter values in the values.yaml file (located at the root of the cloned Git repository branch) in between the section separated with # marks.

• Set the K8SProvider: option to K8S:

K8SProvider: K8S

• Storage classes: disable storage-class creation by the chart and point CAST Imaging at the storage classes already installed on your cluster:

CreateStorageClass: false
DiskClassName: <your-disk-storage-class>   # e.g. local-path, longhorn, rook-ceph-block, ...
FileClassName: <your-file-storage-class>   # e.g. nfs-csi, rook-cephfs, ...  (must support ReadWriteMany, even with a single analysis-node)

• Run helm-install.bat|sh (depending on your base OS) located at the root of the cloned Git repository branch

Keycloak component parameters

CAST Imaging uses the Keycloak system to manage all authentication requirements (see Authentication for more information, and Keycloak accounts and passwords for an explanation of each account provided and how to change its password). You should take note of two parameters that you may wish to customize before running the installation:

KeycloakAdminPassword: <default-password>
KCADMIN_USERNAME: kcadmin

• KeycloakAdminPassword defines the password for the Keycloak admin “service” account used for intra-component communication. Set this if you wish to define the password yourself, e.g. by running echo -n "my-new-password" | base64 to generate a new encrypted password. In releases up to 3.6.5-funcrel this password defaults to “admin” (encrypted in base64);

  Updated in ≥ 3.6.6-funcrel
  a fresh installation generates a random password when you do not set one. This password can also be updated post install (see below).

• KCADMIN_USERNAME defines the name of the account (kcadmin by default with password “admin”) used to login in to the Keycloak UI to administer authentication. Update kcadmin to your preferred account name. Note that this change can only be done BEFORE a new installation.

Step 3 - Configure network settings for console-gateway (main entrypoint) and extendproxy (optional) services

By default, with CreateLoadBalancer: true and none of the ingress/gateway flags enabled, the chart deploys the console-gateway service as a Kubernetes Service of type LoadBalancer, accessible over HTTP on port 8090. For this to result in an actual external IP, a load-balancer implementation such as MetalLB  must be available on the cluster. Without one, the service’s external IP stays <pending>; you can still reach the service via NodePort or by port-forwarding for testing.

If you want HTTPS access, use one of the three options below (they are the same as for cloud-managed Kubernetes). Choose one of the three options below (Gateway API, NGINX Ingress, or Istio). All three options share a common Final steps subsection at the end.

Option A: If you want to use a Gateway API implementation

The Gateway API  is the modern, vendor-neutral successor to the Kubernetes Ingress API. CAST Imaging provides a GatewayAPI.yaml template that creates the Gateway and HTTPRoute resources needed to expose console-gateway-service and (optionally) extendproxy. The template has been tested with NGINX Gateway Fabric  (NGF), but the Gateway / HTTPRoute resources are portable to any conformant implementation (Istio, Envoy Gateway, etc.).

• Set UseGatewayAPI: true in values.yaml:

UseGatewayAPI: true

• Install the Gateway API CRDs on the cluster (if not already installed):

$ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml

• Install a Gateway controller. For NGINX Gateway Fabric:

$ helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway

Verify the installation:

$ kubectl get gatewayclass
$ kubectl get pods -n nginx-gateway

• Configure the Gateway resource in values.yaml. Two options are supported:

  • Option 1 - Create a new Gateway dedicated to CAST Imaging (default):

  Gateway:
    create: true
    className: nginx     # Use "nginx" for NGF, or the gatewayClassName of your controller
  

  • Option 2 - Attach to an existing shared Gateway already deployed on the cluster:

  Gateway:
    create: false
    name: my-shared-gateway
    namespace: my-gateway-namespace
  

  When attaching to an existing Gateway, it must already define listeners named https (and https-extend if ExtendProxy.enable is true with a different hostname for the extend proxy). The shared Gateway’s listener must also allow routes from the CAST Imaging namespace (allowedRoutes.namespaces.from: All or a matching Selector).

• Create TLS Secret(s) using the certificate files associated to the DNS name(s) you are planning to use. The secrets must be created in the CAST Imaging namespace (Gateway listeners read certificates from the Gateway’s own namespace, unless a ReferenceGrant is used):

$ kubectl create secret tls tls-secret-cast --cert=my-cert-folder\myhostname.com\fullchain.pem --key=my-cert-folder\myhostname.com\privkey.pem -n castimaging-v3
# Optional (only needed if you are deploying ExtendProxy):
$ kubectl create secret tls tls-secret-cast-extend --cert=my-cert-folder\myextendhostname.com\fullchain.pem --key=my-cert-folder\myextendhostname.com\privkey.pem -n castimaging-v3
# (fullchain.pem <=> tls.crt ; privkey.pem <=> tls.key)

If you are deploying ExtendProxy (optional) and want to use the same hostname for both services, just create the 2 secrets using the same certificate files.

➡️ Continue with the Final steps (Gateway API, Istio or NGINX Ingress) subsection below.

Option B: If you want to use a Kubernetes NGINX Ingress

• Set CreateIngress: true in values.yaml:

CreateIngress: true

• Install the Ingress driver on the cluster:

$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
$ helm repo update
$ helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace

• Create TLS Secret(s) using the certificate files associated to the DNS name(s) you are planning to use:

$ kubectl create secret tls tls-secret-cast --cert=my-cert-folder\myhostname.com\fullchain.pem --key=my-cert-folder\myhostname.com\privkey.pem -n castimaging-v3
# Optional (only needed if you are deploying ExtendProxy):
$ kubectl create secret tls tls-secret-cast-extend --cert=my-cert-folder\myextendhostname.com\fullchain.pem --key=my-cert-folder\myextendhostname.com\privkey.pem -n castimaging-v3
# (fullchain.pem <=> tls.crt ; privkey.pem <=> tls.key)

If you want to use the same hostname for both services, just create the 2 secrets using the same certificate files.

➡️ Continue with the Final steps (Gateway API, Istio or NGINX Ingress) subsection below.

Option C: If you want to use an Istio Ingress Gateway

• Set CreateIstioGateway: true in values.yaml:

CreateIstioGateway: true

Note that Istio may be already installed on your cluster. Please check with your administrator.

• Install Istio on the cluster (Linux/Mac):

$ curl -L https://istio.io/downloadIstio | sh -
$ cd istio-*
$ export PATH=$PWD/bin:$PATH
$ istioctl install --set profile=default -y

• Install Istio on the cluster (Windows - PowerShell using Chocolatey):

choco install istioctl
istioctl install --set profile=default -y

• Install Istio on the cluster (Windows - PowerShell using manual download and install of a specific version)

$ISTIO_VERSION="1.30.1"  # Check https://github.com/istio/istio/releases for latest
Invoke-WebRequest -Uri "https://github.com/istio/istio/releases/download/$ISTIO_VERSION/istio-$ISTIO_VERSION-win.zip" -OutFile "istio.zip"
Expand-Archive -Path "istio.zip" -DestinationPath "." -Force
cd "istio-$ISTIO_VERSION"
$env:PATH = "$PWD\bin;$env:PATH"
istioctl install --set profile=default -y

• Optional - If you want to use a specific Istio Gateway already deployed on your cluster, set the gateway name in ExistingIstioGatewayName:

UseIstioGateway: true
ExistingIstioGatewayName: "some-namespace/my-gateway"

• Create TLS Secret(s) using the certificate files associated to the DNS name(s) you are planning to use (to be created in the istio-system namespace):

$ kubectl create secret tls tls-secret-cast --cert=my-cert-folder\myhostname.com\fullchain.pem --key=my-cert-folder\myhostname.com\privkey.pem -n istio-system
# Optional (only needed if you are deploying ExtendProxy):
$ kubectl create secret tls tls-secret-cast-extend --cert=my-cert-folder\myextendhostname.com\fullchain.pem --key=my-cert-folder\myextendhostname.com\privkey.pem -n istio-system
# (fullchain.pem <=> tls.crt ; privkey.pem <=> tls.key)

If you want to use the same hostname for both services, just create the 2 secrets using the same certificate files.

➡️ Continue with the Final steps (Gateway API, Istio or NGINX Ingress) subsection below.

Final steps (Gateway API, Istio or NGINX Ingress)

Optional - When Gateway API, Istio or NGINX Ingress is implemented to access the console-gateway service with a certificate that cannot be verified (e.g., self-signed certificate or internal CA), the certificate will need to be stored in CAST auth-service to avoid certificate validation errors:

• set: UseCustomTrustStore: true in values.yaml
• Insert the encoded certificate:
  • directly inside the auth.caCertificate variable in values.yaml
  • or using helm upgrade ... --set-file auth.caCertificate=ca.crt ... to override the variable value with the ca.crt file content

UseCustomTrustStore: true
auth:
  caCertificate: |
    -----BEGIN CERTIFICATE-----
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    -----END CERTIFICATE-----

If you are also deploying ExtendProxy (optional), choose one of the two sub-options below:

• Set hostnames in values.yaml - same-hostname-option: use the same hostname for all services

FrontEndHost: https://myhostname.com
ExtendProxy
  enable: true
  exthostname: myhostname.com

Exposed URLs will be:

• https://myhostname.com (or https://myhostname.com/mycontext if ContextUrl is enabled)

• https://myhostname.com/extendproxy

• Set hostnames in values.yaml - different-hostname-option: use a different hostname for each service:

FrontEndHost: https://myhostname.com
ExtendProxy
  enable: true
  exthostname: myextendhostname.com

Exposed URLs will be:

• https://myhostname.com (or https://myhostname.com/mycontext if ContextUrl is enabled)

• https://myextendhostname.com/extendproxy

• Apply the helm chart changes by running helm-upgrade.bat|sh (depending on your base OS) located at the root of the cloned Git repository branch.

• Create a DNS record pointing at the reverse proxy external IP address. To retrieve the external IP:

# For NGINX Gateway Fabric, use this command:
$ kubectl get gateways -n castimaging-v3
#
# For an NGINX Ingress, use this command:
$ kubectl get ingress -n castimaging-v3
#
# For an Istio Ingress, use this command:
$ kubectl get service istio-ingressgateway -n istio-system

Step 4 - Install Extend Local Server (optional)

To install Extend Local Server:

• In values.yaml (located at the root of the cloned Git repository branch), set ExtendProxy.enable to true and update the ExtendProxy.exthostname variable with the DNS name. In case you want to access it thru its Load Balancer (set “CreateLoadBalancer: true”), use the LoadBalancer external IP address instead or the DNS name (kubectl get service -n castimaging-v3 extendproxy to get the IP):

ExtendProxy:
    enable: true
    exthostname:  myextendhost.com # or LoadBalancer external IP address

• Run helm-upgrade.bat|sh (depending on your base OS) located at the root of the cloned Git repository branch.
• Review the log of the extendproxy pod to find the Extend Local Server administration URL and API key (these are required for managing Extend Local Server and configuring CAST Imaging to use it - you can find out more about this in Extend Local Server). You can get the extendproxy pod name by running kubectl get pods -n castimaging-v3 then run kubectl logs -n castimaging-v3 castextend-xxxxxxxx to display the log.
• In Console System Settings, use this value for the “CAST Extend URL”: http://extendproxy:8085

Step 5 - Initial start up configuration

License and Extend API key steps can be skipped if values were provided for these helm chart variables:

• ImagingLicenseType
• ImagingLicense
• ExtendApiKey

If not, when the install is complete, browse to the public/external URL (on AWS, the FrontEndHost URL) and login using the default local admin/admin credentials.

You will then be prompted to configure:

• your licensing strategy. Choose either a Named Application strategy (where each application you onboard requires a dedicated license key entered when you perform the onboarding), or a Contributing Developers strategy (a global license key based on the number of users):

• CAST Extend settings / Proxy settings (if you chose to install Extend Local Server (see Step 4 above) then you now need to input the URL and API key so that CAST Imaging uses it).

As a final check, browse to the URL below and ensure that you have at least one CAST Imaging Node Service, the CAST Dashboards and the CAST Imaging Viewer components listed:

https://<public or external URL>/admin/services

Step 6 - Configure authentication

Out-of-the-box, CAST Imaging is configured to use Local Authentication via a simple username/password system managed by Keycloak. Default login credentials for CAST Imaging are provided (admin/admin) with the global ADMIN profile so that installation can be set up initially.

CAST recommends configuring CAST Imaging (via the Keycloak component) to use your on-premises enterprise authentication system such as LDAP or SAML Single Sign-on instead before you start to onboard applications. To access the Keycloak UI to configure your authentication system, use the kcadmin account. See Authentication for more information, or Keycloak accounts and passwords for an explanation of each account and how to change its password.

Updating the Keycloak admin service account password post installation

Out-of-the-box Keycloak is provisioned with a service account used for intra-component communication, with the username admin. Its password is set by the KeycloakAdminPassword parameter during installation. If you need to change this password post-installation, follow the steps below:

Change the password via the UI

1. Navigate to the Keycloak account security page using the following URL (replace <public-external-url> with your installation’s external URL). This is the self-service account page for the admin service account, not the Keycloak admin console:

https://<public-external-url>/auth/realms/master/account/account-security/signing-in

2. Log in as the admin service account, using its current password
3. Update the password through the account settings interface

Update the password in the Helm Chart

After changing the password in the Keycloak UI, you must update the Helm chart configuration to match.

1. Encode your new password in base64 format. On a Linux machine, run:

$ echo -n "my-new-password" | base64

2. In your cloned Git repository containing the CAST Imaging Kubernetes installation files, open:

values.yaml

3. Locate this line:

KeycloakAdminPassword: <default-password>

4. Replace the defined password with your base64-encoded password:

KeycloakAdminPassword: <your-base64-encoded-password>

5. Example: If your base64 output is bXktbmV3LXBhc3N3b3Jk, the line should read:

KeycloakAdminPassword: bXktbmV3LXBhc3N3b3Jk

Apply changes

Run Util-ScaleDownAll.bat and helm-upgrade.bat to restart CAST Imaging and apply the changes.

How to start and stop CAST Imaging

Use the following script files (located at the root of the cloned Git repository branch) to stop and start CAST Imaging:

• Util-ScaleDownAll.bat|sh
• Util-ScaleUpAll.bat|sh

Storage class selection

The FileClassName you set in values.yaml must point to a storage class backed by a CSI driver supporting ReadWriteMany (RWX) access mode, because the shared pvc-shared-datadir volume is mounted at the same time by the analysis node(s), console-service and console-control-panel. RWX is therefore required even with a single analysis node. Examples of RWX-capable on-premises options include NFS-based CSI drivers, CephFS (via Rook), GlusterFS, etc. A standard block-storage driver (e.g. local-path-provisioner, plain iSCSI) only supports ReadWriteOnce (RWO) and cannot be used for FileClassName.

Volume permissions when the storage driver does not support fsGroup

CAST Imaging pods run as fixed, non-root users. When a pod starts, Kubernetes uses its fsGroup setting to change the group ownership of the volume so that the pod can write to it (with fsGroupChangePolicy: OnRootMismatch, this is only done when the root folder of the volume does not already have the expected ownership). Some drivers do not apply fsGroup, and NFS servers configured with root_squash (the default) refuse the ownership change. In that case, pods fail to start or fail when writing their files (Permission denied in the pod logs), and the storage administrator must prepare the volumes so that they are owned by the UID/GID used by the pods:

Block / disk drivers formatting the volume with a standard Linux filesystem (ext4, xfs) support fsGroup, so this situation mostly concerns the shared pvc-shared-datadir volume when it is backed by NFS. The example below shows how to pre-provision it on an NFS server, using a static PersistentVolume that the chart’s pvc-shared-datadir claim binds to.

1. On the NFS server, as root, create the folder, give it to UID/GID 10001, and set the setgid bit so that files and folders created inside it keep the 10001 group:

   mkdir -p /exports/castimaging/shared-datadir
   chown -R 10001:10001 /exports/castimaging/shared-datadir
   chmod 2770 /exports/castimaging/shared-datadir
   
   # Check the result: owner and group must be 10001, permissions drwxrws---
   ls -ldn /exports/castimaging/shared-datadir
   

   UID/GID 10001 does not need to exist as a user on the NFS server or on the Kubernetes nodes: only the numeric IDs matter.

2. Export the folder to the Kubernetes nodes. Keep root_squash (the default), as the pods do not run as root. For example, in /etc/exports:

   /exports/castimaging/shared-datadir  10.0.0.0/24(rw,sync,no_subtree_check,root_squash)
   

   Replace 10.0.0.0/24 by the network of your Kubernetes nodes, then reload the exports:

   exportfs -ra
   exportfs -v    # check that the folder is exported with the expected options
   

   If your NFS server uses all_squash, also add anonuid=10001,anongid=10001 to the export options so that all files are created as 10001:10001.

3. Before running the installation, create a PersistentVolume pointing to this folder. Its storageClassName must be the value of FileClassName in values.yaml, its capacity at least size_shared_datadir (default 100Gi), and its claimRef must target the pvc-shared-datadir claim in the CAST Imaging namespace (default castimaging-v3):

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv-castimaging-shared-datadir
   spec:
     capacity:
       storage: 100Gi
     volumeMode: Filesystem
     accessModes:
       - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain
     storageClassName: <your-file-storage-class>   # same value as FileClassName
     mountOptions:
       - nfsvers=4.1
     nfs:
       server: <nfs-server-name-or-ip>
       path: /exports/castimaging/shared-datadir
     claimRef:
       namespace: castimaging-v3
       name: pvc-shared-datadir
   

   kubectl apply -f pv-castimaging-shared-datadir.yaml
   

   After the installation, kubectl get pvc pvc-shared-datadir -n castimaging-v3 must show the claim as Bound to pv-castimaging-shared-datadir.

   This PersistentVolume uses the NFS volume type built into Kubernetes, so no CSI driver is needed, and the storage class named in storageClassName does not need a provisioner. However:

   • The NFS client tools must be installed on every node that can run a CAST Imaging pod (nfs-common on Debian/Ubuntu, nfs-utils on RHEL/Rocky/SUSE), because the volume is mounted by the kubelet from the node itself. Without them, the pods remain in ContainerCreating with a mount failed error in kubectl describe pod.
   • nfsvers=4.1 requires an NFS server supporting NFSv4.1. If your server only supports NFSv3, use nfsvers=3 instead (the nodes then also need rpcbind for file locking), or remove mountOptions to let the client and server negotiate the version.

4. After the installation, check from an analysis node that the shared volume is writable:

   kubectl exec -n castimaging-v3 console-analysis-node-0 -- sh -c 'id; ls -ldn /opt/cast/shared; touch /opt/cast/shared/.write-test && rm /opt/cast/shared/.write-test && echo OK'
   

   The command must display uid=10001 gid=10001, a folder owned by 10001 10001, and OK.

If the volume was instead created dynamically by an NFS provisioner (for example nfs-csi, which creates one sub-folder per volume in its export), apply the same ownership to that sub-folder. Find it from the PersistentVolume bound to the claim:

kubectl get pv $(kubectl get pvc pvc-shared-datadir -n castimaging-v3 -o jsonpath='{.spec.volumeName}') -o jsonpath='{.spec.csi.volumeAttributes}'

Then, on the NFS server, run chown -R 10001:10001 and chmod 2770 on the corresponding sub-folder (for nfs-csi, <share>/<subDir>, by default <share>/<pv name>), and restart the pods using it with Util-ScaleDownAll / Util-ScaleUpAll.

The same procedure applies to the other volumes listed above if their driver does not support fsGroup: use the UID/GID of the corresponding row, and the PVC name and size (size_* values in values.yaml) of that volume in the claimRef and capacity of the PersistentVolume. For the castdir and neo4jdata volumes, which are created per pod, create one PersistentVolume per pod (for example castdir-console-analysis-node-0, castdir-console-analysis-node-1, etc.).

Use an external PostgreSQL instance

If you do not want to use the embedded PostgreSQL instance preconfigured in this helm chart, you can disable it and configure an external PostgreSQL instance instead.

• Setup your external PostgreSQL database

  • PostgreSQL 15 - 8GB RAM minimum recommended
  • Instance must be placed in the same network and the same data center as the Kubernetes cluster nodes: this is required to avoid high network latency, which would significantly impact processing performance in CAST Imaging
  • Define a superuser authenticated by password

• In values.yaml:

  • Set CastStorageService.enable to false (will disable the embedded PostgreSQL installation)
  • Set CustomPostgres.enable to true
  • Set CustomPostgres.host and CustomPostgres.port to match your custom database host name and port number
  • Set CustomPostgres.adminlogin and PostgresDbPassword to match the superuser login name and password of your custom database

• Proceed with the CAST Imaging installation described above

Analysis node autoscaling

• it adds analysis nodes when the job queue holds work that no running analysis node can take
• it removes analysis nodes once they become idle, choosing which node to remove: an analysis node that is running an analysis is never removed

The autoscaler reacts to the depth of the job queue, not to CPU or memory usage. When nothing is queued, the analysis node fleet shrinks to the configured minimum. The feature is disabled by default.

How it works

The autoscaler is a small controller (console-analysis-node-autoscaler Deployment) deployed by the Helm chart when the feature is enabled. It polls the job queue exposed by console-service and resizes the console-analysis-node-core StatefulSet accordingly.

A standard Kubernetes StatefulSet always deletes the highest-numbered pod when scaling down, and that pod may be the one running an analysis. To remove a specific idle pod, the autoscaler relies on the Advanced StatefulSet provided by OpenKruise  (reserveOrdinals feature). When autoscaling is enabled, the console-analysis-node-core StatefulSet is therefore created as an OpenKruise StatefulSet (apps.kruise.io/v1beta1) instead of a standard one (apps/v1).

Analysis capacity per node

The number of analyses an analysis node can run in parallel (its capacity) is defined by MaxParallelAnalysisPerNode in values.yaml, and can then be changed for each node in the CAST Imaging UI. The autoscaler uses both values:

• The capacity set in the UI is the one used to compute the spare capacity of the running nodes and to decide whether a node can be removed.
• When a node is added, a node that was previously removed is brought back first (so that its castdir volume is reused), with the capacity it had in the UI when it was removed. Previously removed nodes with the largest capacity are brought back first. Only a node that never existed starts with MaxParallelAnalysisPerNode.

To remember the capacity of removed nodes, the autoscaler records every capacity that differs from MaxParallelAnalysisPerNode in the imaging.castsoftware.com/ordinal-capacity annotation of the console-analysis-node-core StatefulSet (for example {"1": 4} for node 1 with a capacity of 4). The capacity of a removed node cannot be changed in the UI: it is restored as recorded when the node is brought back.

Requirements

• CAST Imaging ≥ 3.6.9: earlier releases dispatch jobs directly to an analysis node, so the queue is always empty and the autoscaler never scales up.
• OpenKruise operator installed in the cluster: this is an infrastructure prerequisite that is not installed by the CAST Imaging Helm chart (see Install OpenKruise below).
• One schedulable worker node per analysis node: anti-affinity rules give each analysis node its own worker node, and one worker node is used by viewer-neo4j. The practical maximum number of analysis nodes is therefore schedulable worker nodes - 1.
• Each analysis node has its own castdir volume (100Gi by default, see size_castdir). Plan block storage capacity for up to maxNodes × size_castdir.

Install OpenKruise

OpenKruise adds cluster-wide resource types (CRDs), a controller and admission webhooks, so it must be installed by a user with cluster-admin rights:

$ helm repo add openkruise https://openkruise.github.io/charts/
$ helm repo update
$ helm install kruise openkruise/kruise --version 1.9.0

Check that the OpenKruise pods are running before going further:

$ kubectl get pods -n kruise-system

See the OpenKruise installation documentation  for more information.

Check your namespace rights (if you do not have cluster-admin rights)

If CAST Imaging is installed by a user who only has rights on the CAST Imaging namespace, the following commands must all return yes, otherwise the whole helm install / helm upgrade fails (not only the autoscaler):

$ kubectl auth can-i create statefulsets.apps.kruise.io -n castimaging-v3
$ kubectl auth can-i patch  statefulsets.apps.kruise.io -n castimaging-v3
$ kubectl auth can-i delete statefulsets.apps.kruise.io -n castimaging-v3
$ kubectl auth can-i create roles.rbac.authorization.k8s.io -n castimaging-v3
$ kubectl auth can-i create rolebindings.rbac.authorization.k8s.io -n castimaging-v3

The OpenKruise Helm chart grants its rights to the built-in admin and edit cluster roles. A RoleBinding to the built-in admin role in the namespace is therefore sufficient. The edit role is not sufficient, because it cannot create Roles and RoleBindings. If needed, ask your cluster administrator to run:

$ kubectl create rolebinding <your-user>-admin -n castimaging-v3 --clusterrole=admin --user=<your-user>

In this situation, you must also set AnalysisNodeAutoscaling.allowNodeRead: false in values.yaml (see below): with the default value (true), the chart creates a ClusterRole and a ClusterRoleBinding, which cannot be created without cluster-admin rights.

Enable the autoscaler in values.yaml

Set the following in values.yaml:

AnalysisNodeAutoscaling:
  enabled: true
  minNodes: 1
  maxNodes: 3

Then run the CAST Imaging installation (helm-install.bat|sh) or, on an existing deployment, the upgrade (helm-upgrade.bat|sh) as usual.

Settings

All settings are in the AnalysisNodeAutoscaling section of values.yaml:

Clusters with node autoscaling

When the cluster itself adds and removes worker nodes (Cluster Autoscaler on EKS, AKS or GKE, Karpenter, etc.), two autoscalers work side by side: the CAST autoscaler changes the number of analysis node pods based on the job queue, and the cluster autoscaler changes the number of worker nodes when pods are Pending for lack of room. In this case:

• Set allowNodeRead: false. With true, the CAST autoscaler only counts the worker nodes that exist now and never requests a pod that cannot be scheduled, so the cluster autoscaler is never triggered to add a worker node.
• Set maxNodes to the maximum size of the node pool, minus the worker node used by viewer-neo4j if it runs in the same pool.
• Keep stuckPodTimeoutSeconds above the time needed by your cloud provider to add a worker node and pull the analysis node image. Expect a slower scale-up (several minutes) when a worker node must be added.
• Each castdir volume is usually a zonal disk (Amazon EBS, Azure Disk LRS, Google Persistent Disk) that can only be attached in the zone where it was created. If the node pool spans several zones, use a single-zone node pool, a zone-redundant storage class for castdir (e.g. Azure ZRS, GKE regional Persistent Disk), or one node pool per zone.

Starting with 3.6.9, the console-analysis-node, console-postgres and viewer-neo4j pods carry the cluster-autoscaler.kubernetes.io/safe-to-evict: "false" and karpenter.sh/do-not-disrupt: "true" annotations (whether or not the CAST autoscaler is enabled), so a cluster autoscaler never evicts them to remove an under-used worker node and never interrupts a running analysis.

Checking that it works

$ kubectl logs -n castimaging-v3 -l imaging.service=console-analysis-node-autoscaler -f

Each poll logs what the autoscaler observed, what it decided and why, for example:

state: kind=apps.kruise.io replicas=1 ready=1 reserved=[] newNodeCapacity=2 (helm) untainted=4
queue: queued=2 nodes=[0:2/2]
decision: scale-up -> replicas=2 reserve=[]
  reason: 2 job(s) beyond 0 spare thread(s) -> +1 node(s)
  holding: decision is new, need 60s of agreement

• newNodeCapacity is MaxParallelAnalysisPerNode as set in values.yaml: it is the capacity of a node that never existed, and is only used to compute how many nodes to add.
• The queue: line shows the number of queued jobs and, for each running node, node number:analyses in progress/capacity, the capacity being the one set in the UI. It differs from newNodeCapacity when the capacity was changed in the UI.
• When removed nodes have a recorded capacity (see Analysis capacity per node), it is shown at the end of the queue: line, for example reservedCapacity=[1:4].

Common log messages that are expected behavior:

Disabling the autoscaler

Set AnalysisNodeAutoscaling.enabled: false in values.yaml, set AnalysisNodeReplicaCount to the number of analysis nodes you want, and run the upgrade. This removes the autoscaler and all the resources it owns, and restores the standard StatefulSet, which restarts the analysis nodes once (same precautions as when enabling it).

• castdir volumes of analysis nodes numbered AnalysisNodeReplicaCount or higher are kept but no longer used. Delete them if you do not plan to re-enable autoscaling (list them with kubectl get pvc -n castimaging-v3 | grep castdir-console-analysis-node-core).
• Always disable the autoscaler before uninstalling OpenKruise: uninstalling OpenKruise first also deletes the console-analysis-node-core StatefulSet, and the next helm upgrade fails.

Kubernetes storage location information

For information about storage locations used across the containers in a CAST Imaging Kubernetes deployment (for example persistent volumes, ConfigMap-backed configuration files, Secrets, and ephemeral (in-memory) volumes), see Kubernetes storage location information.

values.yaml parameters reference

This section describes each parameter of the values.yaml file provided at the root of the Helm chart, when and how to adjust it, and the risks of changing it on a deployment that is already running. The parameters are listed in the order in which they appear in values.yaml.

Unless stated otherwise, a change in values.yaml is applied by running helm-upgrade.bat|sh. The When can it be changed column uses the following conventions:

Sections in this reference:

• Which values to review before deploying
• General settings
• License and CAST Extend
• Application networking
• Storage
  • Volume sizes
• PostgreSQL
• Passwords and Secrets
  • Using an external secret store
• Optional components
• Extra environment variables
• Analysis settings
• Security and registry access
• Replica counts
• Analysis node autoscaling
• Images
• Resources
• Pod placement

Which values to review before deploying

Most parameters can keep their default values. Use the following overview to focus on the ones that matter for your installation.

1. Always review and set before the initial installation

Review these values for one or more of the following reasons:

• Environment-specific: the default is a placeholder or a generic choice that must be adapted to your cluster (for example FrontEndHost, K8SProvider or the storage classes).
• Security-sensitive: the default is publicly known or exposes CAST Imaging more than you may want (passwords, public load balancer).
• Install-time only: the value can be set now, but never changed once CAST Imaging is deployed.

2. Set only if it applies to your situation

These values must be set when you use the corresponding feature or environment, and can otherwise keep their defaults:

3. Never change, even on a fresh installation (Keep default)

version, postgresqlDataMountPath, anadir.*, and the labelSelector* / podAffinityGroup* affinity rules.

4. Keep the defaults unless advised by CAST Support

*Image.tag and *Image.pullPolicy, the CSI driver names (*_DSdriver, *_FSdriver, install-time only), the log volume sizes, the other replica counts, the other *Resources, postgres_shmsize, AnalysisNode_shmsize, ConsoleEnableBackup, KCADMIN_USERNAME, and the ETL, AIMANAGER and GATEWAYSERVICE environment variables.

General settings

License and CAST Extend

Application networking

Only one of CreateIngress, UseIstioGateway, UseGatewayAPI and UseApplicationGateway can be enabled (the installation fails otherwise). CreateLoadBalancer should be set to false when one of them is used.

Storage

See Kubernetes storage location information for the list of volumes.

Volume sizes

Size the volumes according to the number and size of the applications you plan to analyze: size_castdir, size_shared_datadir, size_db_data and size_neo4jdata are the ones that grow with usage.

PostgreSQL

Enable either CastStorageService (embedded PostgreSQL) or CustomPostgres (external PostgreSQL), not both.

Passwords and Secrets

All passwords are base64-encoded: echo -n "mypassword" | base64 on Linux, or [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes("mypassword")) in PowerShell.

Using an external secret store

existingSecret accepts any Kubernetes Secret, whatever tool creates it. To keep the passwords in an external secret store, use a tool that creates the Secret in the CAST Imaging namespace from that store. The most common one is External Secrets Operator  (ESO), which works the same way on every platform and supports, among others:

Vendor operators (for example HashiCorp Vault Secrets Operator) and GitOps tools that decrypt secrets in the cluster (Sealed Secrets, SOPS) also work, as long as the result is a Secret with the expected keys. The Secrets Store CSI Driver is not suitable: it only synchronizes a Secret while a pod mounts the corresponding CSI volume, and CAST Imaging pods do not.

To use ESO:

1. Install ESO and create a SecretStore (in the CAST Imaging namespace) or a ClusterSecretStore for your store. See the list of ESO providers  for the configuration of your store.

2. In your secret store, create a secret (for example cast-imaging/passwords) containing a JSON object with the keys postgres-db-password, operator-db-password, operator-db-password-crypted2, keycloak-db-password, keycloak-admin-password, neo4j-password and, when SMTP.enable is true, smtp-password. Store the values in plain text, not base64-encoded. For operator-db-password-crypted2, store the complete CRYPTED2:... string generated by cast-imaging-encryption-tool.jar.

3. Create the namespace (kubectl create namespace castimaging-v3) and an ExternalSecret in it:

   apiVersion: external-secrets.io/v1
   kind: ExternalSecret
   metadata:
     name: imaging-passwords
     namespace: castimaging-v3
   spec:
     refreshInterval: 1h
     secretStoreRef:
       kind: ClusterSecretStore      # or SecretStore
       name: my-secret-store
     target:
       name: imaging-passwords       # Secret name to set in existingSecret
     dataFrom:
       - extract:
           key: cast-imaging/passwords
   

   On ESO versions earlier than 0.17, use apiVersion: external-secrets.io/v1beta1. If your store holds one secret per password instead of a JSON object, replace dataFrom with one data entry per key (secretKey set to the expected key name, remoteRef.key set to the name in your store).

4. Check that the Secret is synchronized: kubectl get externalsecret imaging-passwords -n castimaging-v3 must show SecretSynced.

5. Set existingSecret: imaging-passwords in values.yaml and run the installation (or helm-upgrade on an existing deployment, with the passwords currently in use).

Optional components

Extra environment variables

The sections NEO4J, ETL, SERVER, AIMANAGER, MCPSERVER, CONTROLPANEL, ANALYSISNODE, API, AUTHENTICATIONSERVICE, CONSOLESERVICE, DASHBOARDS, GATEWAYSERVICE, SSOSERVICE and EXTENDPROXY add environment variables to the corresponding containers (NAME: value). Only add variables documented by CAST or requested by CAST Support. Changing them restarts the corresponding pods.

Analysis settings

Security and registry access

Replica counts

To stop and start CAST Imaging, use the Util-ScaleDownAll and Util-ScaleUpAll scripts rather than changing the replica counts.

Analysis node autoscaling

The AnalysisNodeAutoscaling parameters (enabled, minNodes, maxNodes, pollIntervalSeconds, scaleUpStabilizationSeconds, scaleDownStabilizationSeconds, stuckPodTimeoutSeconds, dryRun, allowNodeRead, resources) are described in the Analysis node autoscaling section of this page.

Images

Each *Image parameter (InitUtilImage, ConsolePostgresImage, ConsoleSSOServiceImage, ConsoleServiceImage, ConsoleGatewayServiceImage, ConsoleAnalysisNodeImage, ConsoleControlPaneImage, ConsoleAuthenticationServiceImage, ConsoleIntegratedDashboardsImage, ViewerNeo4jImage, ViewerServiceImage, ViewerEtlImage, ViewerAiManagerImage, ViewerApiImage, McpServerImage, ExtendProxyImage) has three settings:

Resources

The *Resources parameters (ConsolePostgresResources, SSOServiceResources, ConsoleServiceResources, GatewayServiceResources, AnalysisNodeResources, ControlPanelResources, AuthenticationServiceResources, IntegratedDashboardsResources, ViewerNeo4jResources, ViewerServerServiceResources, ViewerEtlResources, ViewerAiManagerResources, ViewerApiResources, McpServerResources, ExtendProxyResources, InitContainerResources) define the CPU and memory requests (reserved for the pod) and limits (maximum allowed) of each container.

• Increase AnalysisNodeResources when analyzing large applications or when increasing MaxParallelAnalysisPerNode, and ViewerNeo4jResources (together with the NEO4J heap settings) for large applications in the Viewer.
• The cluster nodes must have enough allocatable CPU and memory for the requests, otherwise pods stay Pending.
• A container exceeding its memory limit is restarted (OOMKilled).

On an existing deployment, changing resources restarts the corresponding pods: for AnalysisNodeResources, ConsolePostgresResources and ViewerNeo4jResources, do it when no analysis is running.

Pod placement