The Overview with Fast Scan panel is part of the onboarding workflow introduced in 2.3.1. It will only be populated when this onboarding workflow is enabled and used - see Administration Center - Settings - Application Onboarding. See also Workflow - Application onboarding with Fast Scan.
Introduction
The panel contains the results of the fast scan phase/deep analysis for a newly onboarded application when using the onboarding with Fast Scan workflow (see Administration Center - Settings - Application Onboarding for information about how to enable it). The process of delivering application source code is exactly as in the legacy onboarding workflow (either via a ZIP archive file or via a designated source code folder on disk) however, the onboarding with Fast Scan process is not a "quick" one-shot onboarding (upload source code, analysis, snapshot, publishing in one go): instead, source code is delivered and then Console will perform the initial "fast-scan" phase to determine the application's technologies/languages/frameworks etc.
When the fast scan phase is complete, users are directed automatically to this Overview with Fast Scan panel where the delivered source code can be inspected (size, structure etc.) for completeness, source code filters (exclusions) can be defined and any "additional options" such as automatic extension installation, activation of Security Dataflow analysis etc. can be activated. Following that, an analysis and publishing to CAST Imaging or Dashboards (i.e. generating a snapshot) can be launched. When an analysis and upload to CAST Imaging/Dashboards has been completed, this Overview panel remains available and additional information will be presented about the deep analysis results.
The main goal of this panel is to encourage source code to be inspected before it is sent for deep analysis to ensure that the correct source code has been delivered and any unwanted code can be excluded.
Access
To access the panel manually, move to the Console screen if you are not already there:
Find the application that has been onboarded using the Onboarding with Fast Scan workflow and click it:
Overview panel
Click to enlarge
Header
The Header section provides basic information about the source code that has been delivered:
Click to expand
Last action date & time | Date and time the application was onboarded. When new source code is uploaded, this value will update. |
---|---|
Status | Indicates the current state of the application:
|
Application Map | Only visible when the status is Fully Analyzed. Links directly to the application in CAST Imaging. |
Total files | Total number of files that have been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload New (see below). In other words, files that are not considered source code (i.e. image files for example) are not included in this file count. |
Application size | Total number of Line of Code (LoC) in the designated source code, as identified by Console during the fast scan process or during a Refresh (see below) - files that are not considered source code (i.e. image files for example) are not included in this value. In addition, an indicator shows the "size" of the application - clicking View Size Chart will show how Console defines the various different size categories: |
New scan | Enables you to upload a new source code ZIP file or deliver new source code from a folder (see Administration Center - Settings - Source Folder Location). You can do this even if you have not yet run an analysis, i.e. when the previous fast scan has highlighted some deficiencies in the delivered code that you want to correct.
Technical information Technically, the following things occur when a New Scan is triggered:
|
Zip Content/Folder Content
This section will be collapsed when a deep analysis has been actioned.
The Zip Content/Folder Content section provides details of the source code that has been uploaded (either via a ZIP file or via the source folder location) with the means to filter (i.e. exclude) certain files and folders:
Click to enlarge
File Filter | This button will reveal the File Filter settings allowing you to:
Expressions A set of exclusion expressions will be predefined via the "default" Exclusion Template which contains the most common items that should be excluded (see Administration Center - Settings - Exclusion templates for more information). If you make any changes, use the Update button to apply them: You can add new custom filters as required: the pattern matching system uses glob patterns (see https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob for examples of how this system works). Enter an expression to match the folders/files you want to exclude and then click Add to add the expression to the list of excluded items: For example:
Take the following hypothetical example where an application has been delivered that contains the same .SQL file in four locations - this is not correct and three of them need excluding: To exclude the *.SQL files located under the parent folder "JSP", you could manually exclude them by unticking them in the UI, however (if you have multiple files to exclude) you can also define an expression to automatically do this for you e.g.: [J]*/**/*.sql Where:
Applying this expression automatically excludes the .SQL files located under the folder "JSP":
Rules This section enables you to configure the "exclusion" rules for specific projects identified during the source code delivery. When an exclusion rule is matched, then the project in question will be ignored. The aim of these rules is to avoid a situation where multiple projects (and therefore Analysis Units) are generated for a given piece of source code when more than one is not needed. If you are unsure, you should leave the default settings as they are and review them as a post analysis action item:
|
---|---|
Update data | The option should be run if you have added source code exclusions. It will run a scan on the existing uploaded source code to update the data in the following sections:
It is not mandatory to run the option, however, doing so can help you understand the impact of the source code exclusions you have added. |
Left panel | The delivered source code is depicted in tree format. This is interactive and selecting an item in the tree will update the middle and right hand panels. In addition, a filter can be set to exclude an item from the subsequent analysis process by clicking the icon shown in the image below. When the icon is shown in red, the entire selected folder and all files, sub folders and files will be excluded from the analysis:
|
Middle panel | This panel depicts the content of an item selected in the left panel and divides them into categories as follows showing the total number of files:
And then:
Each item depicts, per technology type, the total number of files that will be sent for deep analysis and those that will not be sent for deep analysis - i.e. have been excluded through one of the exclusion methods: Items in the categories themselves are interactive and when clicked, will update the content in the right hand panel. |
Right panel | The right panel displays the content of selections made in the left and middle panels and provides:
Search mechanism The search mechanism is a simple filter on the file name itself. For example, entering "auth" shows the following files: Exclusion mechanism Files that have already been excluded via a specific filter, or because the parent folder has been excluded using the icons in the left panel, will be displayed with a strikethrough and a disabled unticked check box as shown below: Click to enlarge To exclude individual files, untick the files - the file text will use strikethrough: Files that are excluded will contain roll over tooltip information (in ≥ 2.9) to explain which pattern has excluded the file. For example, the following file was excluded in the right hand panel using the tick box and the pattern listed in the roll over tooltip will be added as a filter: This file was excluded by a filter called "bookdemo/":
Code viewer Selecting a file in the list will display its source code: |
Software Composition
This section will be collapsed when a deep analysis has been actioned.
The Software Composition section provides details of the uploaded source code - note though that like the Header section, this only shows details of source code that has been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload new. In other words, files that are not considered source code (i.e. image files for example) are not included in this data:
Click to enlarge
On the left an interactive chart depicts the content of the uploaded source code that has been designated as source code (i.e. programming language types), using three different measures:
- Lines of code: total lines of code per technology
- File Count: total number of files per technology
- File Size: total file size per technology, in bytes
Rolling the mouse pointer over the items will display more information:
On the right, the same information is displayed in table format. In addition, a column shows how the identified technology will be analyzed, using:
- Product Extension > an extension provided and supported by CAST
- Community Extension > an extension built by the CAST wider community (not supported by CAST)
- No Known Extension > this technology will not be analyzed since there is no extension available to support it.
Architecture Preview
The Architecture Preview section is a graphical representation of the delivered source code before an analysis is run - this is determined during the fast scan process. The section's primary aim is to help check the completeness of the source code that has been delivered. Use the icon indicated with the red arrow to enlarge the preview:
All links between blocks are based on supposition only. Final architecture from an analysis may be different.
When a deep analysis has been completed, the display will automatically update to show a graphical representation of the source code as detected by the analysis process:
- Items marked in green confirm expected elements (technologies, frameworks, and links).
- Items marked in blue denote an additional element that has been identified during the analysis.
- Items in dashed white show an expected element that has not been found during the analysis.
- Numbers correspond to the number of occurrences of the item that have been found.
- DL refers to "Dynamic Link" (i.e. links that have been found using search string technology) - see Validate Dynamic Links
Identified Frameworks
The Identified Frameworks section lists all the frameworks that have been detected by Console during the fast scan phase:
The icon depicts how the identified framework will be analyzed, using the same legend as in the Software Composition section:
- Product Extension > an extension provided and supported by CAST
- Community Extension > an extension built by the CAST wider community (not supported by CAST)
- No Known Extension > this framework will not be taken into account since there is no extension available to support it
Analysis Reports
This section is only displayed when a deep analysis has been actioned.
This section provides a report on the files discovered/analyzed/excluded/not analyzed for the current version: A list of file extensions found in the delivered source code. Extensions are grouped by technology/language - and the extension that is displayed by default (the primary extension) is the extension with the largest number of files in the delivered source code. Other related file extensions that are found will also be displayed alongside: For the Mainframe - JCL technology, the extension .prc is not considered part of JCL language (in the vast majority of cases, these files do not contain any JCL related code), so files with this extension will be ignored in the analysis report. The CAST Extension Console has used to process the file. Note that some primary file extensions may be listed as processed with multiple extensions. For example the .js file extension will appear twice: The total number of files of this type that were submitted for analysis. The search option allows you to filter for specific text. The search functions on the columns File Extensions, Technology/Language and CAST Extensions: Click to download the report as a .CSV file. When opened in Microsoft Excel (or equivalent), two tabs are available: Clicking a number in the list will open a popup with more details about the files: To make results easy to use, some files are ignored and are not listed in this report:File Extensions Technology/Language Technology or language of the file as detected by Console. CAST Extensions Total The total number of files of this type found in the source code delivery, either delivered in a ZIP file or in a source code folder. The table is sorted by default on this column. Excluded The total number of files of this type that were manually excluded during the source code delivery process. This number is the difference between Total and To Analyze. To Analyze Fully Analyzed The total number of files of this type that were analyzed during the most recent analysis process. This number is taken directly from the analysis schema in which the analysis results are stored, in other words this number reflects the number of files that were saved as part of the analysis process. Note that in Console ≥ 2.4, files classed as external (third party libraries etc.) can be included in this figure as well as internal files (previous releases never included external files). Not Analyzed The total number of files of this type that were not analyzed during the most recent analysis process. This number is the difference between the Fully Analyzed and To Analyze (i.e files that are submitted for analysis, but not saved in the analysis schema). Note that in Console ≥ 2.4, files classed as external (third party libraries etc.) can be included in this figure as well as internal files (previous releases never included external files). View Logs Clicking this icon will direct you straight to the "Run analysis" log files.
Analysis Results Indicators
This section is only displayed:
- in Console ≥ 2.8.
- AND when an analysis has been actioned.
Some indicators require just an analysis to be run and some require a snapshot as well. This is noted in the Available Indicators table below.
This section displays a set of indicators for a given analysis/snapshot. These indicators are designed to provide basic information quickly so that the analysis/snapshot can be validated. The indicators are generated during the analysis/snapshot in a dedicated step and can be generated on-demand manually (see below):
- The feature can be disabled if required - see Configuring Indicators.
- You can enable and disable individual indicators in Administration Center - Settings - Analysis Results Indicators - this requires the global "Admin" role.
Click to enlarge
Categories filter | This drop down filters the Indicators in the list by category. By default, all categories are displayed in the list. |
---|---|
Update banner | This banner is displayed when Console detects that a configuration change has been made and that your data should be updated. If the "Update" button is clicked, then a job will run to ensure that all Analysis Results Indicator data is correct. The triggers for this banner are identical to the triggers described in the section Update Banner in Application - Config. |
Show all indicators | By default this toggle switch is disabled, which means only Indicators that have a positive value (displayed in the Value column) will be displayed. All indicators which have "N/A" in the Value column will be hidden. Enable the option to show all Indicators regardless of their Value. |
Snapshot selector | Choose the snapshot you would like to view indicators for. By default the most recent snapshot for the Application will be displayed. |
Search | Use this to search on the Indicator name. |
Download report | This option enables you to download Microsoft Excel reports containing detailed information about the indicators in each category:
The file name used for the ZIP file and the XLSX files will contain a time stamp using the following format: Excel file report contents
|
Recompute indicators | This option enables you to recompute the indicators without needing to generate an entire new analysis/snapshot.
Note that this button is hidden if the Update Banner (described above) is displayed). |
Indicator | See list of available indicators below for more information. |
Value | The value generated for the current snapshot. Can be a ratio or a percentage. For some indicators when only one snapshot exists, the value may be N/A, for example:
|
Status | Status of the indicator - the more stars the better the results. Rolling the mouse over the stars will show the thresholds required to improve: |
Justification | Justifications can only be edited for the most recent snapshot. If the snapshot selector is changed to a historic snapshot, the field becomes read-only. A free text field enabling you to enter a justification for the result. Free text is saved and is retained for the next snapshot that is generated. For example:
≥ 2.9 Use the icon to add a new justification explanation:
And enter the justification in the pop-up: The icon changes to indicate a justification has been added: Older releases Enter the justification in the field itself. |
Remedy Action | The Remedy Action provides a suggestion for how to improve the result in the next analysis/snapshot. Any links are clickable. ≥ 2.9 Remedies are displayed in line: If they are larger than the available space, rolling the mouse over them will display the full text in a popup: Older releases Click the icon to display the full text of the remedy: |
Details | Some indicators store results in a CSV file - click this option to download the CSV file. This can help you work out why a poor result has been produced, for example. CSV files are generated and stored in the following locations: ≥ 2.2.0 \\share\aip-node-data\common-data\snapshot-indicator\{appGuid}\{snapshotGuid} ≤ 2.1.0 - on the Node %PROGRAMDATA%\CAST\AipConsole\AipNode\snapshot-indicator\{appGuid}\{snapshotGuid} |
Clickable Indicators
Some Indicators are clickable: clicking the link will take you to the relevant configuration page within Console:
Available Indicators
Technical information about Indicators
- When a snapshot is deleted all the indicators for the snapshot along with the generated CSV files are deleted. In addition, the consolidation action launched when a snapshot is deleted will only deal with Dashboard schema indicators for the next two snapshots if they exist. During consolidation, any justification text is retained for each indicator.
- Snapshot Indicators are also deleted when a version and an application are deleted.
Logging information about Indicators
Advanced Platform Configuration
This section is only displayed if you have configured more than one of either of the following:
- Multiple CAST Storage Service/PostgreSQL instances for analysis or Measurement requirements - see Administration Center - Settings - CSS and Measurement settings.
- Multiple Nodes (i.e. you are running the enterprise release of Console which allows for multiple Nodes to be configured) - see Administration Center - Nodes.
This allows you to select the specific target CAST Storage Service/PostgreSQL instance (for the database schemas required for the new Application) OR the target Node (for deep analysis requirements). If you do not make a selection - i.e. you leave the options set to "ANY", Console will function in "load balancing" mode and will choose the CAST Storage Service/PostgreSQL or Node automatically:
- If you have ALREADY run a deep analysis, the UI will prevent you from choosing a different CAST Storage Service instance or Node for any subsequent analysis related actions.
- Load Balancing behaviour, when ANY is selected:
- CAST Storage Service/PostgreSQL
- For the deep analysis step (result storage), the CAST Storage Service/PostgreSQL instance with the lowest number of CAST related schemas already stored on it will be used.
- Nodes
- For the initial fast scan, Console will always use "load balancing" mode, which functions as follows:
- The node running the most recent release of AIP Core will always be used before all others.
- If there are multiple nodes running the same most recent release of AIP Core, then Console will choose the least busy node.
- For the deep analysis step, the least busy node running the same release of AIP Core as used for the initial fast scan will be selected.
- For the initial fast scan, Console will always use "load balancing" mode, which functions as follows:
- CAST Storage Service/PostgreSQL
- Node manual selection: only nodes running the same release of AIP Core as used for the initial fast scan of the onboarding process will be made available for selection - this is to prevent analysis errors. This may mean that it is not possible to choose a specific node.
Run analysis
In ≤ 2.7, CAST Imaging MUST be configured Administration Center - Settings - Imaging Settings otherwise the action will fail. This requirement has been removed in ≥ 2.8.
This section provides the following:
- Information about the state of the source code
- Allows you to start an analysis
- Provides an analysis estimation time in hours and minutes. In ≥ 2.6 this estimation is valid for the analysis action and the upload to CAST Imaging (no estimation is given for CAST Dashboard actions). In previous releases, the estimation is only valid for the analysis action.
Information about the state of the source code | Information about the readiness of the delivered source code for analysis is provided based on the initial fast scan: All clear If no "issues" are found then the "all clear" is given: All clear but cannot access CAST Imaging/CAST Dashboards If no "issues" are found, but CAST Imaging/CAST Dashboards are either not configured or not available, the upload to CAST Imaging/CAST Dashboards (snapshot) will not run: Issues found If issues are found, then a warning is given with an explanation. In this situation, a warning does not mean that the analysis cannot proceed, however, coherent results may not be produced. For example:
Analysis complete When an analysis has been run, this panel will show:
|
---|---|
Run Analysis | Click the Run Analysis button to start the deep analysis process. A popup will then be displayed: When an analysis is started, a full backup of the onboarding details (e.g. delivered source code and any exclusions that have been set) and is created (in ZIP format) and is stored in the following locations (see below). This is so that any manually or automatically (via a filter) excluded folders/files can be removed before the analysis is started. When the analysis action is complete, any excluded files/folders are put back in the original location (ZIP file unzip location or source code folder location):
|
Deep analysis estimation time | The deep analysis estimation time is provided in hours and minutes and is based on anonymous statistical data that has been collected by CAST using the Allow CAST to automatically collect anonymous statistical data option in the Admin Center - see Administration Center - Settings - CAST Extend. Note that this estimation is only valid for the analysis action and does not include any other actions that may have been enabled for CAST Dashboards/Imaging. |
This option allows you to control what steps in the analysis process are actioned and should only be used if you know what you want to achieve: |
What steps are actioned when Run Analysis is clicked?
When the Run Analysis button is clicked, the following will occur automatically depending on the configuration:
CAST Imaging configured and available | Embedded CAST Dashboards configured and available | Analysis | Upload to CAST Imaging | Security Dataflow | Snapshot generation | Upload to CAST Dashboards |
---|---|---|---|---|---|---|
Install, Configure, Analyze
The actions Install, Configure and Analyze are ALWAYS actioned regardless of your configuration:
The Finalizing Analysis entry will only be visible in the Analyze section when source code exclusions have been configured. This step restores the excluded files after the analysis has completed:
Upload
The Upload action differs depending on your configuration:
Configuration | Requirement | Actions |
---|---|---|
Any (Standard AIP Core, AIP Core for Imaging, AIP Core for Security) | CAST Imaging MUST be configured in Administration Center - Settings - Imaging Settings and accessible. |
|
With embedded Dashboards | Embedded CAST Dashboards MUST be configured and accessible. See Embedded CAST Dashboard deployment process. |
|
Additional analysis options
Depending on the configuration and license in use the following configuration will also be automatically applied when the Run Analysis button is clicked:
Option name | Target | Action |
---|---|---|
Security Dataflow | CAST Dashboards | This option focuses on user input security assessments for JEE/NET technologies. Selecting this option will:
This configuration is applied as follows:
|
Function Points | CAST Dashboards | This option focuses on function points measurement. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): If you are using a CAST global license that does not include EFP, then this option will not produce any results. |
Tags for Data Access Sensitivity | CAST Imaging and CAST Dashboards | This option focuses on flow of data identification and will deliver associated results. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions):
GDPR / PCI DSS Two additional options specifically enable a check of a set of predefined sensitive key words related to GDPR (General Data Protection Regulation) and/or PCI-DSS (Payment Card Industry Data Security Standards) data:
In other words, enabling the GDPR option (for example) will force the check using the predefined keywords. When the analysis runs, the predefined keywords defined will be checked and if any are found in the source code a flag will be added in the analysis results on the object in question. This can be seen as below in CAST Imaging: Click to enlarge |
Resuming interrupted jobs
Should your job be interrupted for whatever reason (network issue, issue on the Node etc.), CAST Console is able to resume the job from the same point or a previous point. Take for example a job that has been interrupted in the Install step:
Returning to the Application - Overview with Fast Scan page, a Resume button will be displayed in place of Run analysis:
In addition starting CAST Console 2.9, steps that were successfully completed prior to the interruption will be displayed as follows:
Log panel | Click to enlarge |
---|---|
Job progress screen ≥ 2.9 | Click to enlarge |