Dashboards are not updated after a job

Configure how long the analysis node waits for the Dashboards service to respond when a job updates the Dashboards

Overview

Available in ≥ 3.6.9-funcrel

When a job updates the Dashboards - at the end of a snapshot, or when results are uploaded to Measure - the analysis node waits for the CAST Imaging Dashboards service to respond. This page explains what happens when the Dashboards service does not respond in time, and how to change the timeout.

What happens when the timeout is reached

If the Dashboards service does not respond within the configured time, for example because it has run out of memory, the analysis node stops waiting:

  • An ERROR is written in the step log, for example in the SNAPSHOT step. The message starts as follows, with the configured timeout:

    The CAST Imaging Dashboards service did not respond within 5 minutes. The Dashboards have not been updated with the results of this job.

  • The same message is displayed as an alert on the job in the console, under A specific error/warning message appeared in log.

  • The step is not failed. The job continues with its remaining steps and ends as completed.

  • The analysis and snapshot results are kept, but the Dashboards are not updated until the Dashboards service is fixed and restarted.

The step log can also contain ERROR Failed to update application information right after the timeout message. This is a consequence of the same problem: it does not raise an alert and needs no separate action.

Before you change the timeout

A longer timeout rarely fixes the problem. If you see the timeout error, check the Dashboards service logs first, for example for an OutOfMemoryError. Increasing the maximum memory of the Dashboards service is usually the right fix - see Increase the memory available to the Dashboards.

Increase the timeout only if your Dashboards normally take more than 5 minutes to load a large application.

Change the timeout

The timeout is controlled by the following analysis node property:

Property Default Unit
dashboards.rest.read-timeout-minutes 5 minutes

The default is built in: the property is not present in the configuration files shipped with CAST Imaging, so you always need to add it. If you have more than one analysis node, repeat these steps on each one.

Step 1 - Set the new value

Edit or create the analysis node application-default.yml file. Do not edit application.yml: it is overwritten when you update CAST Imaging.

%PROGRAMDATA%\CAST\Imaging\CAST-Imaging-Analysis-Node\application-default.yml

Add the following lines, for example to wait 10 minutes:

dashboards:
  rest:
    read-timeout-minutes: 10

Edit or create the following file - see Managing docker-compose.yml files for guidance:

/opt/cast/installation/imaging-node/docker-compose.override.yml

Add the DASHBOARDS_REST_READTIMEOUTMINUTES environment variable to the nodev3 service, for example to wait 10 minutes:

services:
  nodev3:
    environment:
      DASHBOARDS_REST_READTIMEOUTMINUTES: 10

Step 2 - Restart the analysis node

Restart the CAST Imaging Analysis Node Windows service.

Recreate the analysis node container so that it picks up the new environment variable:

$ cd /opt/cast/installation/imaging-node
$ sudo docker compose up -d nodev3