Page tree
Skip to end of metadata
Go to start of metadata

The Modern Overview panel is part of the modern onboarding workflow introduced in 2.3.1 (see 2.3.1 - Summary of changes to the onboarding process). It will only be populated when the modern onboarding workflow is enabled and used - see Administration Center - Settings - Application Onboarding.

Introduction

The panel contains the results of the fast scan phase/deep analysis for a newly onboarded application when using the modern onboarding workflow (see Administration Center - Settings - Application Onboarding for information about how to enable it). The process of delivering application source code is exactly as in the legacy onboarding workflow (either via a ZIP archive file or via a designated source code folder on disk) however, the modern onboarding process is not a "quick" one-shot onboarding (upload source code, analysis, snapshot, publishing in one go): instead, source code is delivered and then Console will perform the initial "fast-scan" phase to determine the application's technologies/languages/frameworks etc.

When the fast scan phase is complete, users are directed automatically to this Modern Overview panel where the delivered source code can be inspected (size, structure etc.) for completeness, source code filters (exclusions) can be defined and any "additional options" such as automatic extension installation, activation of Security Dataflow analysis etc. can be activated. Following that, an analysis and publishing to CAST Imaging or Dashboards (i.e. generating a snapshot) can be launched. When an analysis and upload to CAST Imaging/Dashboards has been completed, this Overview panel remains available and additional information will be presented about the deep analysis results.

The main goal of this panel is to encourage source code to be inspected before it is sent for deep analysis to ensure that the correct source code has been delivered and any unwanted code can be excluded.

Access

To access the panel manually, move to the Console screen if you are not already there:

Find the application that has been onboarded using the modern workflow and click it:

Modern Overview panel

Click to enlarge

Header

The Header section provides basic information about the source code that has been delivered:

Click to expand

Last action date & timeDate and time the application was onboarded. When new source code is uploaded, this value will update.
Status

Indicates the current state of the application:

  • Fast scan done > The fast scan process is complete - i.e. the uploaded source code has been scanned to determine the contents: technologies/languages/frameworks. 
  • Fully analyzed > The source code has been analyzed and that upload to CAST Imaging / CAST Dashboards has occurred. Note that if any missing dependencies are detected in your source code during the analysis (i.e. code that is calling another piece of code that cannot be found) a warning icon will be displayed. This should be fully investigated and corrected because it means that results may not be coherent. Clicking the warning triangle will direct you straight to the log file to see the missing dependencies alerts. See also Validate dependency configuration.

Application MapOnly visible when the status is Fully Analyzed. Links directly to the application in CAST Imaging.
Total filesTotal number of files that have been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload New (see below). In other words, files that are not considered source code (i.e. image files for example) are not included in this file count.
Application size

Total number of Line of Code (LoC) in the designated source code, as identified by Console during the fast scan process or during a Refresh (see below) - files that are not considered source code (i.e. image files for example) are not included in this value. In addition, an indicator shows the "size" of the application - clicking View Size Chart will show how Console defines the various different size categories:

New scan

Enables you to upload a new source code ZIP file or deliver new source code from a folder (see Administration Center - Settings - Source Folder Location). You can do this even if you have not yet run an analysis, i.e. when the previous fast scan has highlighted some deficiencies in the delivered code that you want to correct. 

  • if you have already uploaded a source code ZIP, the button will display the following dialog box enabling you to choose a new ZIP file. A new fast scan will then be automatically actioned on the new source code:

  • if you have already delivered source code from a source folder location, the button will display the following dialog box showing the location of the previous source folder. Ensure the source folder contains the new updated source code before clicking the Fast Scan button:

Technical information

Technically, the following things occur when a New Scan is triggered:

  • any excluded files are deleted
  • the current version is deleted and a new one is created with the same name, same title, same release date
  • the new source code is scanned and data in the Overview panel is refreshed using the scan results
  • no associated snapshots are deleted

Zip Content/Folder Content

This section will be collapsed when a deep analysis has been actioned.

The Zip Content/Folder Content section provides details of the source code that has been uploaded (either via a ZIP file or via the source folder location) with the means to filter (i.e. exclude) certain files and folders:

Click to enlarge

File Filter

This button will reveal the File Filter settings allowing you to exclude specific files and/or folders using regular expression based exclusion rulesA set of exclusion rules will be predefined via the "default" Exclusion Template which contains the most common items that should be excluded (see Administration Center - Settings - Exclusion templates for more information). If you make any changes, use the Update button to apply them:

You can add new custom filters as required: the pattern matching system uses glob patterns (see https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob for examples of how this system works). Enter an expression to match the folders/files you want to exclude and then click Add to add the expression to the list of excluded items:

For example:

  • *.txt will exclude all files with the extension .txt
  • tests/ will exclude any folders named tests and everything inside them - e.g. root_folder/tests, root_folder/another_folder/tests
  • *.Tests/ will exclude any folders whose name includes .Tests (for example C:\Support\Delivery\Sample.Tests\sample\)
  • patterns starting with / will exclude starting only from the root folder. In other words, /tests/ will exclude everything in the specific folder root_folder/tests but not root_folder/another_folder/tests
  • CAST highly recommends using the Update Data option if you configure exclusions - see below.
  • Excluding a folder or files via the UI will automatically add the specific items to the File Filter list. For example, the folder "pageart" has been manually excluded using the UI and this is reflected in the File Filter dialog:

  • If you have already actioned an analysis and you subsequently exclude files/folders, then a banner will be displayed in the Application - Config screen to prompt you that the the configuration has changed and that you should run a new analysis in order to see the changes:

Update data

The option should be run if you have added source code exclusions. It will run a scan on the existing uploaded source code to update the data in the following sections:

  • Header
  • Software Composition
  • Architecture Preview
  • Identified Frameworks

It is not mandatory to run the option, however, doing so can help you understand the impact of the source code exclusions you have added.

Left panel

The delivered source code is depicted in tree format. This is interactive and selecting an item in the tree will update the middle and right hand panels. In addition, a filter can be set to exclude an item from the subsequent analysis process by clicking the icon shown in the image below. When the icon is shown in red, the entire selected folder and all files, sub folders and files will be excluded from the analysis:

  • CAST highly recommends using the Update Data option if you configure exclusions - see above.
  • Excluding a folder or files via the UI will automatically add the specific items to the File Filter list. For example, the folder "pageart" has been manually excluded using the UI and this is reflected in the File Filter dialog:

  • If you have already actioned an analysis and you subsequently exclude files/folders, then a banner will be displayed in the Application - Config screen to prompt you that the the configuration has changed and that you should run a new analysis in order to see the changes:

Middle panel

This panel depicts the content of an item selected in the left panel and divides them into categories as follows showing the total number of files:

  • Files selected for deep analysis
  • Files not selected for deep analysis

And then:

  • Programming
  • Documentation
  • Data
  • Build

Each item depicts, per technology type, the total number of files that will be sent for deep analysis and those that will not be sent for deep analysis - i.e. have been excluded through one of the exclusion methods:

Items in the categories themselves are interactive and when clicked, will update the content in the right hand panel.

Right panel

The right panel displays the content of selections made in the left and middle panels and provides:

  • a search mechanism
  • a way to exclude individual files
  • a way to view source code

Search mechanism

The search mechanism is a simple filter on the file name itself. For example, entering "auth" shows the following files:

Exclusion mechanism

Files that have already been excluded via a specific filter, or because the parent folder has been excluded using the icons in the left panel, will be displayed with a strikethrough and a disabled unticked check box as shown below:

Click to enlarge

To exclude individual files, untick the files - the file text will use strikethrough:

  • CAST highly recommends using the Update Data option if you configure exclusions - see above.
  • Excluding a folder or files via the UI will automatically add the specific items to the File Filter list. For example, the folder "pageart" has been manually excluded using the UI and this is reflected in the File Filter dialog:

  • If you have already actioned an analysis and you subsequently exclude files/folders, then a banner will be displayed in the Application - Config screen to prompt you that the the configuration has changed and that you should run a new analysis in order to see the changes:

Code viewer

Selecting a file in the list will display its source code:

Software Composition

This section will be collapsed when a deep analysis has been actioned.

The Software Composition section provides details of the uploaded source code - note though that like the Header section, this only shows details of source code that has been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload new. In other words, files that are not considered source code (i.e. image files for example) are not included in this data:

Click to enlarge

On the left an interactive chart depicts the content of the uploaded source code that has been designated as source code (i.e. programming language types), using three different measures:

  • Lines of code: total lines of code per technology
  • File Count: total number of files per technology
  • File Size: total file size per technology, in bytes

Rolling the mouse pointer over the items will display more information:

On the right, the same information is displayed in table format. In addition, a column shows how the identified technology will be analyzed, using:

  • Product Extension > an extension provided and supported by CAST
  • Community Extension > an extension built by the CAST wider community (not supported by CAST)
  • No Known Extension > this technology will not be analyzed since there is no extension available to support it.

Architecture Preview

The Architecture Preview section is a graphical representation of the delivered source code before an analysis is run - this is determined during the fast scan process. The section's primary aim is to help check the completeness of the source code that has been delivered. Use the icon indicated with the red arrow to enlarge the preview:

All links between blocks are based on supposition only. Final architecture from an analysis may be different.

When a deep analysis has been completed, the display will automatically update to show a graphical representation of the source code as detected by the analysis process:

  • Items marked in green confirm expected elements (technologies, frameworks, and links).
  • Items marked in blue denote an additional element that has been identified during the analysis.
  • Items in dashed white show an expected element that has not been found during the analysis.
  • Numbers correspond to the number of occurrences of the item that have been found.
  • DL refers to "Direct Link".

Identified Frameworks

The Identified Frameworks section lists all the frameworks that have been detected by Console during the fast scan phase:

The icon depicts how the identified framework will be analyzed, using the same legend as in the Software Composition section:

  • Product Extension > an extension provided and supported by CAST
  • Community Extension > an extension built by the CAST wider community (not supported by CAST)
  • No Known Extension > this framework will not be taken into account since there is no extension available to support it

Analysis Reports

This section is only displayed when a deep analysis has been actioned.

This section provides a report on the files discovered/analyzed/excluded/not analyzed for the current version:

File Extensions

A list of file extensions found in the delivered source code. Extensions are grouped by technology/language - and the extension that is displayed by default (the primary extension) is the extension with the largest number of files in the delivered source code. Other related file extensions that are found will also be displayed alongside:

For the Mainframe - JCL technology, the extension .prc is not considered part of JCL language (in the vast majority of cases, these files do not contain any JCL related code), so files with this extension will be ignored in the analysis report.

Technology/LanguageTechnology or language of the file as detected by Console.
CAST Extensions

The CAST Extension Console has used to process the file. Note that some primary file extensions may be listed as processed with multiple extensions. For example the .js file extension will appear twice:

  • once with com.castsoftware.html5 
  • once with com.castsoftware.jee.
TotalThe total number of files of this type found in the source code delivery, either delivered in a ZIP file or in a source code folder. The table is sorted by default on this column.
ExcludedThe total number of files of this type that were manually excluded during the source code delivery process. This number is the difference between Total and To Analyze.
To Analyze

The total number of files of this type that were submitted for analysis.

Fully AnalyzedThe total number of files of this type that were analyzed during the most recent analysis process. This number is taken directly from the analysis schema in which the analysis results are stored, in other words this number reflects the number of files that were saved as part of the analysis process. Note that in Console ≥ 2.4, files classed as external (third party libraries etc.) can be included in this figure as well as internal files (previous releases never included external files).
Not AnalyzedThe total number of files of this type that were not analyzed during the most recent analysis process. This number is the difference between the Fully Analyzed and To Analyze (i.e files that are submitted for analysis, but not saved in the analysis schema). Note that in Console ≥ 2.4, files classed as external (third party libraries etc.) can be included in this figure as well as internal files (previous releases never included external files).
View LogsClicking this icon will direct you straight to the "Run analysis" log files.

The search option allows you to filter for specific text. The search functions on the columns File ExtensionsTechnology/Language and CAST Extensions:

Click to download the report as a .CSV file. When opened in Microsoft Excel (or equivalent), two tabs are available:

  • Source Files Analysis Summary : displays an overview, i.e. the same data available in Console UI.
  • Details: displays a list of files with their primary file extension, their technology, the list of CAST extensions that support the technology, and the status (excluded, analyzed, not analyzed).

Clicking a number in the list will open a popup with more details about the files:

To make results easy to use, some files are ignored and are not listed in this report:

  • all files with extensions that are not associated directly to a programming language (all resources or data languages for example, or project files like xml, http or json).
  • all files with patterns such as .git, .svn, node-modules, org-eclipse, CCAU\.abap, IP\.abap
  • special files like package-info.java, *CT.abap, *CP.abap, hh, h++, hpp, hcc, h, hxx, ph
  • files resulting from CAST Datbase extractions such as castextraction or uaxdirectory, because they are not currently directly associated with a language or extension.

Advanced Platform Configuration

This section is only displayed if you have configured more than one of either of the following:

This allows you to select the specific target CAST Storage Service/PostgreSQL instance (for the database schemas required for the new Application) OR the target Node (for deep analysis requirements). If you do not make a selection - i.e. you leave the options set to "ANY", Console will function in "load balancing" mode and will choose the CAST Storage Service/PostgreSQL or Node automatically:

  • If you have ALREADY run a deep analysis, the UI will prevent you from choosing a different CAST Storage Service instance or Node for any subsequent analysis related actions.
  • Load Balancing behaviour, when ANY is selected:
    • CAST Storage Service/PostgreSQL
      • For the deep analysis step (result storage), the CAST Storage Service/PostgreSQL instance with the lowest number of CAST related schemas already stored on it will be used.
    • Nodes
      • For the initial fast scan, Console will always use "load balancing" mode, which functions as follows:
        • The node running the most recent release of AIP Core will always be used before all others.
        • If there are multiple nodes running the same most recent release of AIP Core, then Console will choose the least busy node.
      • For the deep analysis step, the least busy node running the same release of AIP Core as used for the initial fast scan will be selected.
  • Node manual selection: only nodes running the same release of AIP Core as used for the initial fast scan of the onboarding process will be made available for selection - this is to prevent analysis errors. This may mean that it is not possible to choose a specific node.

Run analysis

CAST Imaging MUST be configured Administration Center - Settings - Imaging Settings otherwise the action will fail.

This section provides the following:

  • Information about the state of the source code
  • Allows you to start an analysis
  • Provides an analysis estimation time in hours and minutes. In ≥ 2.6 this estimation is valid for the analysis action and the upload to CAST Imaging (no estimation is given for CAST Dashboard actions). In previous releases, the estimation is only valid for the analysis action.
Information about the state of the source code

Information about the readiness of the delivered source code for analysis is provided based on the initial fast scan. If no "issues" are found then the "all clear" is given:

If issues are found, then a warning is given with an explanation, for example in the screenshot below Console is warning that the delivered source code has links from Hibernate/JPA/Spring Data to SQL, however, no SQL source has been delivered. In this situation, a warning does not mean that the analysis cannot proceed, however, coherent results may not be produced.

When an analysis has been run, this panel will show:

  • the previous analysis duration time
  • whether any missing dependencies were detected in your source code during the analysis (i.e. code that is calling another piece of code that cannot be found): a yellow warning icon will be displayed if this is the case. This should be fully investigated and corrected because it means that results may not be coherent. Clicking the warning triangle will direct you straight to the log file to see the missing dependencies alerts. See also Validate dependency configuration.

Start an analysis

Click the Run Analysis button to start the deep analysis process. A popup will then be displayed:

When an analysis is started, a full backup of the onboarding details (e.g. delivered source code and any exclusions that have been set) and is created (in ZIP format) and is stored in the following locations (see below). This is so that any manually or automatically (via a filter) excluded folders/files can be removed before the analysis is started. When the analysis action is complete, any excluded files/folders are put back in the original location (ZIP file unzip location or source code folder location):

  • Enterprise modeSHARED_FOLDER (common-data) location in the docker-compose.yml file - usually similar to \\shared\console\common-data\backup\source_folder_backups
  • Standalone mode >  %PROGRAMDATA%\CAST\AIP-Console-Standalone\shared\backup\source_folder_backups
Deep analysis estimation timeThe deep analysis estimation time is provided in hours and minutes and is based on anonymous statistical data that has been collected by CAST using the Allow CAST to automatically collect anonymous statistical data option in the Admin Center - see Administration Center - Settings - CAST Extend. Note that this estimation is only valid for the analysis action and does not include any other actions that may have been enabled for CAST Dashboards/Imaging.

What steps are actioned when Run Analysis is clicked?

When the Run Analysis button is clicked, the following will occur automatically:

Install, Configure, Analyze

The actions Install, Configure and Analyze are ALWAYS actioned regardless of your configuration:

The Finalizing Analysis entry will only be visible in the Analyze section when source code exclusions have been configured. This step restores the excluded files after the analysis has completed:

Upload

The Upload action differs depending on your configuration:

ConfigurationRequirementActions

Any (Standard AIP Core, AIP Core for Imaging, AIP Core for Security)

CAST Imaging MUST be configured in Administration Center - Settings - Imaging Settings, if not, the action will fail.
  • Upload to CAST Imaging will ALWAYS be actioned. 

With embedded DashboardsEmbedded CAST Dashboards MUST be configured. See Embedded CAST Dashboard deployment process.
  • A snapshot is generated (for the Engineering Dashboard)
  • Data is also uploaded to the Measure schema for the Health Dashboard
  • Upload to CAST Imaging is also actioned.

Additional analysis options

Depending on the configuration and license in use the following configuration will also be automatically applied when the Run Analysis button is clicked:

Option nameTargetAction
Security DataflowCAST Dashboards

This option focuses on user input security assessments for JEE/NET technologies. Selecting this option will:

Function PointsCAST Dashboards

This option focuses on function points measurement. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions):

If you are using a CAST global license that does not include EFP, then this option will not produce any results.

Tags for Data Access Sensitivity

CAST Imaging and CAST Dashboards

This option focuses on flow of data identification and will deliver associated resultsSelecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions):

GDPR / PCI DSS

Two additional options specifically enable a check of a set of predefined sensitive key words related to GDPR (General Data Protection Regulation) and/or PCI-DSS (Payment Card Industry Data Security Standards) data:

Each option corresponds to one .datasensitive file located in the following location on the Node:

%PROGRAMDATA%\CAST\AIP-Node\datasafetychecks

In other words, enabling the GDPR option (for example) will force the check using the keywords defined in GDPR_Keywords.datasensitive. When the analysis runs, the predefined .datasensitive file corresponding to the chosen option is sent to the LISA folder (LISA/{appGuid}/DataSafety) and any keywords defined in them will be checked. If any keywords are found in the source code a flag will be added in the analysis results on the object in question. This can be seen as below:

Click to enlarge

Click to enlarge

  • No labels