Advanced onboarding for CAST Imaging - add a new version and deliver source code

Introduction
Add a new Version
Deliver source code
What happens when a version is added and source code delivered?

To perform this step, your AIP Console login must have the Application Owner role at a minimum.

Introduction

In the AIP Console, source code for a given Application is organised into versions - the idea being that you can create new Versions when the Application's source code changes and you want to measure these changes. When working with a new Application, an initial version first needs to be created and source code delivered so that it can be analyzed. This page describes how to do this.

Run analysis should be disabled

In an advanced onboarding scenario, we do not want to run an analysis as part of the version creation process, therefore we must ensure that the Run analysis option is disabled when delivering the source code (this option can be enabled in a Standard onboarding scenario where you simply want results quickly) - this option is explained fully below:

Click to enlarge

Add a new Version

To access Console, use the following URL (ensure you specify the correct server and port number): http://<server>:8081/ui/index.html. Then move to the Console screen if you are not already there:

Find the newly created Application - in this example it is called "MEUDON" - and click the Add Version icon:

Click to enlarge

The Delivery wizard is then displayed - see below.

Deliver source code

Add code

Click to enlarge

To add source code:

Drag and drop one .zip or .tar.gz file containing your source code - or click the upload cloud icon to add a file using a traditional "open file" method. Please ensure that your source code archive does NOT contain any other archive files - this can cause the analysis process to fail.
If you have configured a source code folder (see Administration Center - Settings - Source Folder Location) you can click the yellow folder to add code directly from this folder - only one folder in the designated path can be selected:

If you want to deliver source code containing:

.NET source code: you should ensure that you define the location of the .NET Assemblies which is required for a successful analysis. See Configuring source code delivery for .NET.
Maven based source code: you should ensure that you define the location of the Maven repositories so that any associated JAR files can be automatically discovered and that POM dependencies can also be located - which is required for a successful analysis. See Configuring source code delivery for Maven.

When you have made your option choices (see below), click either:

PROCEED, any subsequent steps will be skipped and:
- if the Run Analysis option is disabled, the version will be added and will appear in the Application - Versions screen.
- if the Run Analysis and the Publish to CAST Imaging options are enabled, the analysis/publishing will be actioned immediately.
NEXT, see Manage Exclusions.

Options available

Option

Description

Version Name

Enter a name for the Version. Default name will be set to: Version-YYYY-MM-DDTHH-MM-SS.

Version Date

Enter a date for the Version. Default date will be set to YYYY-MM-DD HH:MM.

Backup application

Unticked by default. Selecting the option will cause the Application to be backed up as part of the actions you choose. When adding a new version for a new Application, enabling this option is not necessary since there is nothing to back up. When the option is enabled, some additional processing time is required while the backup completes. Backups are stored in the following location on the relevant AIP Node and can be managed in Administration Center - Applications - Application Details:

%PROGRAMDATA%\AipConsole\AipNode\backup\<application_name>\YYYYMMDDHHMM.zip

Same configuration as previous version

Only available when adding a new version when a version already exists - i.e. a "rescan" scenario. See Application rescan for CAST Imaging for more information.

Enable Automatic discovery

Only available when adding a new version when a version already exists and when the option Same configuration as previous version (see above) is enabled - i.e. a "rescan" scenario. See Application rescan for CAST Imaging for more information.

Run analysis

Choose whether you want to run an analysis immediately after the source code is delivered. Technically the Run analysis option also includes an internal step called "Prepare analysis data" step, which allows:

Source code to be viewed when validating Dynamic Links
Architecture Models results to be checked before running an analysis

Legacy application onboarding - Standard onboarding

For a Standard onboarding scenario, you should enable this option. Enabling this option includes the following steps that would otherwise need to be manually actioned:

Validate the Version - i.e. the delivered source code
Accept the Version
Set as current version - i.e. the version to be analyzed

Enabling the option will also automatically expose the following options:

Legacy application onboarding - Step-by-step onboarding

For a step-by-step onboarding scenario, you should disable this option. This will allow you to work through the process of validating/accepting the version, running an analysis and publishing to CAST Imaging as separate steps.

Add modules

This option allows you to choose the Module strategy for your results. Modules are used as a means to configure analysis results into meaningful groups or sets for display purposes - indeed CAST Imaging has a dedicated Modules view mode. The content of a module is based on source code.

Disabling the Run analysis option will hide this option.
When you are delivering a new Version of your Application source code (i.e. a Version already exists), and you decide to tick Same configuration as previous version, the option to choose a module strategy is hidden. In other words, the module strategy chosen in the previous version will be retained and used.
You can change the selected Module strategy and create user define modules once an initial analysis has run: see Application - Config - Modules.

CAST offers these options when creating a new version:

Add modules

When selected, you have the choice to create one Module per Technology or one Module per Analysis Unit (see below). However, if you untick the option, one single Full Content module will be created containing all your Application source code.

Per technology

If you select this option, one Module is created per Technology in your Application - these Modules are created as User Defined Modules and can therefore be edited/removed. This option is the default, therefore unless you specifically choose a different strategy, this one will be used. This option causes the Auto generate strategy in Application - Config - Modules to be set to None (Manual Creation). For example:

Per analysis unit

If you select this option, one Module is created per Analysis Unit in your Application - these Modules are created as User Defined Modules and can therefore be edited/removed. This option causes the Auto generate strategy in Application - Config - Modules to be set to None (Manual Creation). For example:

An Analysis Unit can best be described as a set of configuration settings that govern how a perimeter of source code is consistently analyzed. Analysis Units are automatically created - as such they more often than not correspond to Projects discovered by AIP Console.

Publish to CAST Imaging

If you do not see this option, you must enable the Run analysis option.
If this option is visible, but is unavailable for selection, this means that a corresponding CAST Imaging instance has not been configured. See Administration Center - Settings - Imaging Settings for more information.

Choose whether you want to publish the data in your CAST Imaging instance immediately after the source code is delivered. This option is enabled by default:

Legacy application onboarding - Standard onboarding

For a Standard onboarding scenario, you should enable this option.

Legacy application onboarding - Step-by-step onboarding

For a step-by-step scenario, you should disable this option. This will allow you to work through the process of validating/accepting the version, running an analysis and publishing to CAST Imaging as separate steps.

Manage exclusions

Click to enlarge

Manage Exclusions is an optional step in the source code upload process - it allows you to use to exclude specific files and/or folders in the uploaded archive file and manage project exclusion rules (available in AIP Console ≥ 1.26).

Ignore Patterns

In this section, any excluded items will be ignored during the source code analysis. A set of exclusion rules will be predefined via the "default" Exclusion Template which contains the most common items that should be excluded (see Administration Center - Settings - Exclusion templates for more information):

You are free to set whatever exclusion rules you require:

You can remove existing exclusions using the trash icon
You can add new existing exclusions using the Add Expression option (see below)

When you have made your option choices (see below), click:

FINISH, any subsequent steps will be skipped and:
- if the Run analysis option is disabled, the version will be added and will appear in the Application - Versions screen.
- if the Run Analysis and the Publish to CAST Imaging options are enabled, the analysis/publishing will be actioned immediately.
NEXT, see Choose Objectives.

Options available

Overwrite existing exclusion rules	This option is only visible in rescan mode in AIP Console ≥ 1.25, when the Same configuration as previous version option is enabled. When the option is: disabled, all Ignore Patterns options are also disabled and cannot be edited or removed - this ensures that the same ignore patterns are applied to the new source code delivery as were applied in the previous delivery. enabled, all Ignore Patterns options are also enabled and can be edited and removed - this allows you to change the ignore patterns for this new source code delivery. Bear in mind that changing the ignore patterns for a new delivery can impact the subsequent analysis results. Note that this option should be typically left in the disabled position if you have imported an application from CAST Management Studio (see Import an Application managed with CAST Management Studio into AIP Console) and are re-analyzing the source code with AIP Console: exclusion rules are different in CAST Management Studio/Delivery Manager Tool and in AIP Console, and if you use the AIP Console interface to add the exclusion rules, these will overwrite the exclusion rules managed in CAST Management Studio/Delivery Manager Tool.
Retain source code zip in the application upload folder	This option is only visible: in AIP Console ≥ 1.27 when delivering source code for an Application that has been created in Rapid mode, without version history when delivering source code from a Source Code Location (see Administration Center - Settings - Source Folder Location) When ticked, the selected source code folder will be zipped before the delivery/analysis begins, primarily so that any files that are excluded from the analysis can be retrieved if necessary. The ZIP file will be stored in the shared data location, in the Upload folder.
Expression: Add a new pattern using a glob pattern expression	The pattern matching system uses glob patterns (see https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob for examples of how this system works). Enter an expression to match the folders you want to exclude and then click ADD to add the expression to the list of excluded items: For example: .txt will exclude all files with the extension .txt* tests/ will exclude any folders named tests and everything inside them - e.g. root_folder/tests, root_folder/another_folder/tests .Tests/ will exclude any folders whose name includes .Tests* (for example C:\Support\Delivery\Sample.Tests\sample\) patterns starting with / will exclude starting only from the root folder. In other words, /tests/ will exclude everything in the specific folder root_folder/tests but not root_folder/another_folder/tests You can add multiple expressions.
Preview (Available in AIP Console ≥ 2.1)	This option enables you to preview which files/folders will be excluded from the source code analysis. In the example below, an ignore pattern called "sql/" has been added (ignoring any folder named "sql" or "SQL"):
Use exclusion template	This option enables you to choose from a template that has been predefined in the Administration Center - Settings - Exclusion templates panel under Exclusion Templates: When a template is selected, the list of exclusions will be populated with those defined in the template. You can delete items if you prefer:
Using the GUI to select items	Click to access a screen where you can select the files and folder from the uploaded archive file you want to exclude (the archive will be unpacked by AIP Console) - place a check mark in the items that must be excluded, then click SAVE to add them to the list of exclusions:

Project Exclusion Rules

Available in AIP Console ≥ 1.26.

This section enables you to configure the "exclusion" rules for specific projects identified during the source code delivery. When an exclusion rule is matched, then the project in question will be ignored. The aim of these rules is to avoid a situation where multiple projects (and therefore Analysis Units) are generated for a given piece of source code when more than one is not needed. If you are unsure, you should leave the default settings as they are and review them before accepting delivery.

for an Application's first version, all options are selected except Exclude Maven Java projects when an Eclipse project also exists
for an Application's subsequent versions, exclusion rules are pre-selected according to the options chosen in the previous version delivery.
the option Exclude all empty projects refers to projects that do not have associated source code.
the option Exclude Test code will exclude all folders named "test" that are discovered during the source code delivery

Choose Objectives

Objectives is an optional feature that is designed to pre-configure an analysis (install specific extensions, set specific settings etc.) based on the results you require:

When you have made your option choices (see below), click PROCEED:

if the Run Analysis option is disabled, the version will be added and will appear in the Application - Versions screen.
if the Run Analysis and the Publish to CAST Imaging options are enabled, the analysis/publishing will be actioned immediately.

When enabling any of the Objectives, it is recommended to allow Alpha and Beta extensions to be installed via the Extension Strategy option (Administration Center - Extensions Strategy / Administration Center - Settings - Extensions Strategy), because some of the extensions that are installed automatically via the Objectives feature are currently only in Alpha/Beta release. If Alpha/Beta extensions are not permitted to be installed, the results of the selected objectives will not be produced.
When an extension whitelist is in use via the Extension Strategy option (Administration Center - Extensions Strategy / Administration Center - Settings - Extensions Strategy), any extensions that are automatically installed by a selected Objective and which are not present in the white list will cause the analysis to stop.
If you do not wish to use any of the objectives offered, untick all options. This will ensure that no additional extensions (over and above what you have defined and what has been automatically discovered) will be installed and no additional options will be enabled automatically.
If you are performing a rescan by adding a version N+1 (i.e. you have already created a version, run an analysis and uploaded the data to CAST Imaging and are now working on the next version) and you tick the option Same as previous configuration in Step 1, the same objectives will be applied as in the previous version.
If you have run an analysis and enabled various objectives, and you then edit the version and run a new analysis, the same objectives will be applied.

Options available

Option	Default settings	Description
Global risk assessment	Active	This option focuses on risk assessments by adding additional structural rules to the analysis. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): com.castsoftware.qualitystandards com.castsoftware.jeerules com.castsoftware.dotnetweb com.castsoftware.systemlevelrules com.castsoftware.automaticlinksvalidator
Security assessment	Inactive	This option focuses on user input security assessments for JEE/NET technologies. Selecting this option will currently: install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): com.castsoftware.qualitystandards com.castsoftware.automaticlinksvalidator com.castsoftware.securityforjava (JEE only) com.castsoftware.jeerules (JEE only) com.castsoftware.dotnetweb (.NET only) enable the following options: Application - Security Dataflow for the discovered technologies (JEE and/or .NET)
Functional points measurement	Active	This option focuses on function points measurement. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): com.castsoftware.automaticlinksvalidator If you are using a global license that does not include EFP, then this option will not produce any results.
Blueprint design	Inactive	This option focuses on architecture identification and links between layers. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): com.castsoftware.automaticlinksvalidator
Data safety investigation	Inactive	This option focuses on flow of data identification and will deliver associated results. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): com.castsoftware.automaticlinksvalidator com.castsoftware.datacolumnaccess com.castsoftware.mainframe.sensitivedata (in AIP Console ≥ 1.26) GDPR / PCI DSS These options are ONLY currently taken into account for Mainframe technolgies (analyzed via the com.castsoftware.mainframe and SQL technologies (analyzed via the com.castsoftware.sqlanalyzer extension). These options do not provide a certification for GDPR and PCI DSS. Their aim is to help identify sensitive data in particular contexts. It is still possible to provide your own patterns (see Mainframe Sensitive Data and Data Column Access). Two additional options are available (in AIP Console ≥ 1.26) specifically enabling a check of a set of predefined sensitive key words related to GDPR (General Data Protection Regulation) and/or PCI-DSS (Payment Card Industry Data Security Standards) data: Each option corresponds to one .datasensitive file located in the following location on the AIP Node: %PROGRAMDATA%/CAST/AipConsole/AipNode/datasafetychecks In other words, enabling the GDPR option (for example) will force the check using the keywords defined in GDPR_Keywords.datasensitive. When the analysis runs, the predefined .datasensitive file corresponding to the chosen option is sent to the LISA folder (LISA/{appGuid}/DataSafety) and any key words defined in them will be checked. If any key words are found in the source code a flag will be added in the analysis results on the object in question. This can be seen as below: Click to enlarge Click to enlarge

What happens when a version is added and source code delivered?

When a version is added various steps are processed as follows:

Each step is explained below:

Attaching package to version
Unzipping source and Prepare new version	This step unpacks the uploaded archive file containing the version source code to a dedicated location on the AIP Node: %PROGRAMDATA%\CAST\AipConsole\AipNode\upload\<application_name> A new version is then created. Technically this means that corresponding configuration files will be generated in: %PROGRAMDATA%\CAST\AipConsole\AipNode\delivery\<application_guid>\data\<version_guid>
Content discovery	During this step, the source code will be scanned and a list of technologies contained in the version will be determined based on the extension of each file. In addition some source code "discoverers" may also be installed automatically based on the list of file extensions found.
Install extensions	Extensions are automatically installed for EVERY single source code Version you deliver - this means that each Version will have a specific set of extensions enabled and installed, tailored to the source code that needs to be analyzed. During this step, CAST AIP extensions will be installed as follows: All automatically active / shipped extensions All extensions discovered based on the information found in the previous Content discovery step. Any extensions that are included in an Objective. You can find out more about each of the above types of extensions in Application - Extensions. It is possible that no extensions may need installing for a source code version - this is particularly true if you are delivering a new Version of an existing Version (i.e. Vn+1) and you tick the Same configuration as previous version in the source code delivery wizard:
Creating package from source	During these two steps the uploaded source code is compressed and transformed it into a format specific to AIP Console. The code is then transferred to the "delivery" location in "packages": %PROGRAMDATA%\CAST\AipConsole\AipNode\delivery\<application_guid>\data\<version_guid>\<package_guid>
Delivering version	The version creation process is completed and the delivery report is generated, which consists of information about any alerts raised during the version creation process and a list of all files included in the source code upload. This report can be seen in the Version details screen under Version report - see Application - Versions.