Created by James Hurrell on Jun 14, 2023

Introduction

The process of exporting and importing data into a CAST Imaging instance has previously been a manual process that must be actioned following the completion of either a snapshot or an analysis. This panel provides the settings required to fully automate this data export and import as part of the standard onboarding and rescan processes, so that when these processes are complete, all data is ready to use in CAST Imaging without having to perform any additional manual actions. 

  • Import actions can take a long time for very large applications. Please avoid restarting services during an application import - this may cause corruption in the Neo4j database.
  • Please ensure you are aware of the free disk space requirements on the CAST Imaging host as detailed in Performance issues - tips, particularly if your application is very large.

Prerequisites

CAST Imaging up and running(tick)Your designated CAST Imaging instance MUST be up and running when you attempt this since Console will attempt to connect using the defined settings when the Save button is clicked. Only one CAST Imaging instance can be configured in Console to accept data.
CAST Imaging / ETL releases(tick)

This feature requires the following minimum releases of CAST Imaging or the standalone ETL Tool used with an older release of CAST Imaging:

CAST Imaging≥ 2.2.0
CAST Imaging ETL≥ 2.1.3
CAST Imaging user with the ADMIN role(tick)At least one CAST Imaging user must be configured with the CAST Imaging ADMIN role (see Admin Center - Users panel) - this user should be specified in the Console settings panel as explained below.
CAST Imaging license must be valid(tick)You must ensure that the license applied in CAST Imaging is valid.

Settings

Console ≥ 1.26

When using  Console ≥ 1.26-funcrel an API Key is required to access CAST Imaging. This API Key is only supported in CAST Imaging ≥ 2.5.2-funcrel.

Imaging URL

Enter the URL to your CAST Imaging instance, including the port number if appropriate, for example:

http://my_imaging_server:8083/

In Console ≥ 2.5, if you use "localhost" for your URL (i.e when CAST Imaging is located on the same machine as CAST Console), this will automatically be transformed into "hostname" for the direct links to CAST Imaging that are available in AIP Console - Application Management. This is so that users that are accessing CAST Console from a remote machine will be able to correct access CAST Imaging.

API Key

Paste in your CAST Imaging API Key. You can find out how to generate one in Admin Center - API key generation and usage. Note that the API Key should be generated for a user that has the CAST Imaging ADMIN role (see Admin Center - Users panel).

Note that when you need to update the API Key, you should:

  • use the trash can icon to first remove the existing API Key
  • then paste in the new API Key
  • finally click SAVE.

Pasting in the new API Key without first removing the existing key will cause the key to not be accepted.

Console ≤ 1.25

Click here to expand...


Imaging URL

Enter the URL to your CAST Imaging instance, including the port number if appropriate, for example:

http://my_imaging_server:8083/
Imaging usernameEnter a CAST Imaging user name/password that has the CAST Imaging ADMIN role (see Admin Center - Users panel).
Imaging password
Imaging ETL PortEnter the port number of the Imaging ETL service running on your CAST Imaging instance.. By default the Imaging ETL service (which is responsible for the export/impact of data into CAST Imaging) runs on port 9001 (enter this number unless you have customized your internal services). If the instance is remote to the Console host server as recommended, then you may need to configure the firewall on the CAST Imaging instance instance to accept incoming TCP connections on port 9001.
Imaging ETL Token

Enter the ETL token. The token can be found in the following file:

Microsoft Windows traditional installer:
%APPDATA%\CAST\ImagingSystem\login\application.properties

Docker Installer extension (located in the folder created when unzipping the extension):
login/application.properties

For example, the token that should be used is D5ED6A406775FC71B8D2A978883E8ED4 in the example configuration below:

proxy.config.token.values=D5ED6A406775FC71B8D2A978883E8ED4,D5ED6A406775FC71B8D2A978883E8ED4,D5ED6A406775FC71B8D2A978883E8ED4,D5ED6A406775FC71B8D2A978883E8ED4
Note that in the vast majority of circumstances, the token provided by default in the Console wizard will match what is required.

Saving the settings

Use the Save button to save the settings you have entered - Console will check that the settings are correct at this point. If Console can successfully access the CAST Imaging instance using the settings you have entered, a "success" message will be shown, otherwise a "failure" message will be shown in which you will need to check the settings you have entered:

Failure messages are recorded in the Console front-end log file located in %PROGRAMDATA%\CAST\AipConsole\logs. For example the following suggests that the Imaging ETL Token is incorrect:

ERROR com.castsoftware.webi.common.services.ImagingClient - 403 Forbidden: [{"error":"forbidden"}

Behavior when CAST Imaging is configured

Onboarding with Fast Scan

The configuration of the upload to CAST Imaging is a prerequisite of using the Workflow - Application onboarding with Fast Scan.

Legacy onboarding (without Fast Scan)

The Publish to CAST Imaging option will be enabled in the Add version wizard (for onboarding new applications and rescanning existing applications):

Click to enlarge

When the options in Imaging Settings described above do not exist or are incorrect, the option is unticked and disabled:

The Publish to CAST Imaging option will be enabled in the run analysis/snapshot wizard:

When the options in Imaging Settings described above do not exist or are incorrect, the option is unticked and disabled:

Click to enlarge

All releases

a new corresponding step will be visible in the analysis/snapshot process:

The CAST Imaging icon will be visible in the AIP Console - Application Management screen for both new Applications and rescan of existing Applications:

Editing existing settings

When you have entered settings you can edit them but you cannot remove them.

Behaviour when CAST Imaging settings are incorrect

If the defined CAST Imaging settings are incorrect, for example, the API key is updated in CAST Imaging but is not updated in CAST Console, or the URL changes or is not accessible, then CAST Console (≥ 2.9) will warn you about this:

A system alert is displayed:

When using the Advanced option to run analysis jobs in Fast Scan mode, the CAST Imaging steps will be greyed out and will not be ticked:

When using the Run Analysis button in Fast Scan mode the CAST Imaging related steps will not be run:

Ensuring Console uploads data during the next analysis

To ensure that your data is uploaded to your CAST Imaging instance, ensure the following options are enabled and ticked:

Onboarding with Fast Scan

The configuration of the upload to CAST Imaging is a prerequisite of using the Workflow - Application onboarding with Fast Scan.

Legacy onboarding (without Fast Scan)

What happens when the Application data already exists in CAST Imaging?

When the current application already exists in your designated CAST Imaging instance, whether from a manual export/import, or from a previous automated upload from Console, when you choose data to be uploaded again in a new snapshot/analysis (known as a refresh) the application data will be merged with existing data already imported into CAST Imaging - so new objects may be visible and existing objects may be removed or updated.

What happens if you delete an Application or a snapshot?

If you delete an Application or a snapshot using Console, no corresponding data will be removed from your designated CAST Imaging instance. If you need remove the data from CAST Imaging, please see Admin Center - Application management panel.

Where is the exported data stored?

When the Publish to CAST Imaging option is ticked, the data export/import process will be performed during the analysis/snapshot as explained previously. This process generates a set of files containing the data exported from the Analysis schema for your Application and these are stored in the following location on the CAST Imaging instance:

%PROGRAMDATA%\CAST\ImagingSystem\Neo4j_data\import

Domain and tenant synchronization

Information valid for Console ≥ 1.26-funcrel, which requires CAST Imaging ≥ 2.5.2-funcrel.

In  Console ≥ 1.26-funcrel it is possible to choose to synchronize your Console domains with equivalent CAST Imaging tenants. In other words, if your Application belongs to a domain in Console, you can optionally upload the Application to a tenant in CAST Imaging with the same name as the domain. This is achieved by enabling an option in the aip-node-app.properties file located on the Node responsible for analyzing your Application:

≥ 2.x: 
%PROGRAMDATA%\CAST\AIP-Node\aip-node-app.properties

≥ 1.x:
%PROGRAMDATA%\CAST\AipConsole\AipNode\aip-node-app.properties

Locate the following section and set the synchro.domains.imaging.enabled option to true:

# ==============
CAST Imaging configuration
--------------
set to true if you want that your Console domains becomes CAST Imaging tenant and that your application be imported into
a tenant in Cast Imaging corresponding to the domain in  Console
synchro.domains.imaging.enabled=false

Save the aip-node-app.properties file and then restart the Node package in order for the configuration to be taken into account. All applications managed by the Node in question will now be synchronized to equivalent tenants in CAST Imaging, if they belong to a domain in Console. You should repeat this action on all Nodes where you want the same to be true.

Additional notes about domain and tenant synchronization

  • When synchro.domains.imaging.enabled option is in the default false position, the Application will be uploaded to the default tenant in CAST Imaging, or the current tenant in which the Application is currently stored if it already exists in CAST Imaging, regardless of whether the application belongs to a domain in Console or not.
  • When synchro.domains.imaging.enabled option is in the true position, the following will occur:
    • If the Application belongs to a domain in Console and the Application is not yet present in CAST Imaging, the Application will be uploaded to a tenant in CAST Imaging with the same name. If the equivalent tenant does not already exist in CAST Imaging, the tenant will be created.
    • If the Application belongs to a domain in Console and the Application is already present in CAST Imaging in a tenant with the same name, the Application will be uploaded to that same tenant and merged.
    • If the Application belongs to a domain in Console and the Application is already present in a different tenant or the default tenant in CAST Imaging, the Application will be uploaded to a new tenant matching the domain name in Console. In this case, the Application will exist twice (or more) in CAST Imaging. It is the end user's responsibility to manage domains and tenants.
    • If the Application does not belong to a domain in Console, the Application will be uploaded to the default tenant in CAST Imaging (if it does not already exist in CAST Imaging), or the current tenant in which the Application is currently stored (if it already exists in CAST Imaging).
    • If a domain in console has a ? or / in its name, or if its name is less than 3 characters long, the equivalent tenant will not be created in CAST Imaging (since this naming syntax is not permitted in CAST Imaging) and the Application will be uploaded to the default tenant.