- Introduction
- Access
- Overview panel
- Technical aspects
The Overview with Fast Scan panel is part of the onboarding workflow introduced in 2.3.1. It will only be populated when this onboarding workflow is enabled and used - see Administration Center - Settings - Application Onboarding. See also Workflow - Application onboarding with Fast Scan.
Introduction
The panel contains the results of the fast scan phase/deep analysis for a newly onboarded application when using the onboarding with Fast Scan workflow (see Administration Center - Settings - Application Onboarding for information about how to enable it). The process of delivering application source code is exactly as in the legacy onboarding workflow (either via a ZIP archive file or via a designated source code folder on disk) however, the onboarding with Fast Scan process is not a "quick" one-shot onboarding (upload source code, analysis, snapshot, publishing in one go): instead, source code is delivered and then Console will perform the initial "fast-scan" phase to determine the application's technologies/languages/frameworks etc.
When the fast scan phase is complete, users are directed automatically to this Overview with Fast Scan panel where the delivered source code can be inspected (size, structure etc.) for completeness, source code filters (exclusions) can be defined and any "additional options" such as automatic extension installation, activation of Security Dataflow analysis etc. can be activated. Following that, an analysis and publishing to CAST Imaging or Dashboards (i.e. generating a snapshot) can be launched. When an analysis and upload to CAST Imaging/Dashboards has been completed, this Overview panel remains available and additional information will be presented about the deep analysis results.
The main goal of this panel is to encourage source code to be inspected before it is sent for deep analysis to ensure that the correct source code has been delivered and any unwanted code can be excluded.
Access
To access the panel manually, move to the Console screen if you are not already there:
Find the application that has been onboarded using the Onboarding with Fast Scan workflow and click it:
Overview panel
Click to enlarge
Header
The Header section provides basic information about the source code that has been delivered:
Click to expand
Last action date & time | Date and time the application was onboarded. When new source code is uploaded, this value will update. |
---|---|
Status | Indicates the current state of the application:
|
Application Map | Only visible when the status is Fully Analyzed. Links directly to the application in CAST Imaging. |
Total files | Total number of files that have been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload New (see below). In other words, files that are not considered source code (i.e. image files for example) are not included in this file count. |
Application size | Total number of Line of Code (LoC) in the designated source code, as identified by Console during the fast scan process or during a Refresh (see below) - files that are not considered source code (i.e. image files for example) are not included in this value. In addition, an indicator shows the "size" of the application - clicking View Size Chart will show how Console defines the various different size categories: |
Alert | The alert icon (available in ≥ 2.10) indicates that the most recent job actioned on the current application contains one or more alerts (the number of alerts is indicated) that need to be looked at. Clicking the icon will display the alerts as a slide in panel on the right: These alerts are also displayed in the Job Progress panel. |
New scan | Enables you to upload a new source code ZIP file or deliver new source code from a folder (see Administration Center - Settings - Source Folder Location). You can do this even if you have not yet run an analysis, i.e. when the previous fast scan has highlighted some deficiencies in the delivered code that you want to correct.
Technical information Technically, the following things occur when a New Scan is triggered:
|
Zip Content/Folder Content
This section will be collapsed when a deep analysis has been actioned.
The Zip Content/Folder Content section provides details of the source code that has been uploaded (either via a ZIP file or via the source folder location) with the means to filter (i.e. exclude) certain files and folders:
Click to enlarge
This section l has different behaviour depending on the CAST Console release you are using:
- ≥ 2.10 - ALL files (regardless of whether they can be analyzed or not are displayed).
- ≤ 2.9 - files that cannot be analyzed (e.g. image files) are not displayed.
File Filter | This button will reveal the File Filter settings allowing you to:
Expressions A set of exclusion expressions will be predefined via the "default" Exclusion Template which contains the most common items that should be excluded (see Administration Center - Settings - Exclusion templates for more information). Items excluded in this way are not sent for analysis. If you make any changes, use the Update button to apply them: You can add new custom filters as required: the pattern matching system uses glob patterns (see https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob for examples of how this system works). Enter an expression to match the folders/files you want to exclude and then click Add to add the expression to the list of excluded items: For example:
Take the following hypothetical example where an application has been delivered that contains the same .SQL file in four locations - this is not correct and three of them need excluding: To exclude the *.SQL files located under the parent folder "JSP", you could manually exclude them by unticking them in the UI, however (if you have multiple files to exclude) you can also define an expression to automatically do this for you e.g.: [J]*/**/*.sql Where:
Applying this expression automatically excludes the .SQL files located under the folder "JSP":
Rules This section enables you to configure the "exclusion" rules for specific projects identified during the source code delivery. When an exclusion rule is matched, then the project in question will be ignored. The aim of these rules is to avoid a situation where multiple projects (and therefore Analysis Units) are generated for a given piece of source code when more than one is not needed. If you are unsure, you should leave the default settings as they are and review them as a post analysis action item:
Items excluded by a rule will NOT be reflected in the Zip Content/Folder Content section and will be sent for analysis. Instead, these items will only be excluded during the analysis process. For example, the Exclude Test Code rule (when ticked) will exclude all folders (and their contents) called "test", however, all folders called "test" will still be sent for analysis and will not be marked as excluded: Folders called "test" are sent for analysis: Folders called "test" are only excluded during the analysis and appear as "not analyzed" in the Analysis Report section: |
---|---|
Update data | The option should be run if you have added source code exclusions. It will run a scan on the existing uploaded source code to update the data in the following sections:
It is not mandatory to run the option, however, doing so can help you understand the impact of the source code exclusions you have added. |
Left panel | The delivered source code is depicted in tree format. This is interactive and selecting an item in the tree will update the middle and right hand panels. In addition, a filter can be set to exclude an item from the subsequent analysis process by clicking the icon shown in the image below. When the icon is shown in red, the entire selected folder and all files, sub folders and files will be excluded from the analysis (i.e. these items will NOT be sent for analysis):
|
Middle panel | This panel depicts the content of an item selected in the left panel and divides them into categories as follows showing the total number of files:
And then:
Each item depicts, per technology type, the total number of files that will be sent for analysis and those that will not be sent for analysis - i.e. have been excluded through one of the exclusion methods: Items in the categories themselves are interactive and when clicked, will update the content in the right hand panel. |
Right panel | The right panel displays the content of selections made in the left and middle panels and provides:
All files that will be sent for analysis will be ticked, files that are NOT sent for analysis will be displayed with a strikethrough: Search mechanism The search mechanism is a simple filter on the file name itself. For example, entering "auth" shows the following files: Exclusion mechanism Files that have already been excluded via a specific filter, or because the parent folder has been excluded using the icons in the left panel, will be displayed with a strikethrough and a disabled unticked check box as shown below (these items are NOT send for analysis): Click to enlarge To exclude individual files, untick the files - the file text will use strikethrough: Files that are excluded will contain roll over tooltip information (in ≥ 2.9) to explain which pattern has excluded the file. For example, the following file was excluded in the right hand panel using the tick box and the pattern listed in the roll over tooltip will be added as a filter: This file was excluded by a filter called "bookdemo/":
Code viewer Selecting a file in the list will display its source code: |
Software Composition
This section will be collapsed when a deep analysis has been actioned.
The Software Composition section provides details of the uploaded source code - note though that like the Header section, this only shows details of source code that has been designated as source code (i.e. programming language types) by Console during the fast scan process or during a Refresh/Upload new. In other words, files that are not considered source code (i.e. image files for example) are not included in this data:
Click to enlarge
On the left an interactive chart depicts the content of the uploaded source code that has been designated as source code (i.e. programming language types), using three different measures:
- Lines of code: total lines of code per technology
- File Count: total number of files per technology
- File Size: total file size per technology, in bytes
Rolling the mouse pointer over the items will display more information:
On the right, the same information is displayed in table format. In addition, a column shows how the identified technology will be analyzed, using:
- Product Extension > an extension provided and supported by CAST
- Community Extension > an extension built by the CAST wider community (not supported by CAST)
- No Known Extension > this technology will not be analyzed since there is no extension available to support it.
Complete File Inventory
Available in ≥ 2.10.
Provides a complete inventory of all the files delivered for analysis - note that this list may reflect files that were not sent for analysis (i.e. because they are unsupported, such as image files).
Technology Stack / Framework / Package Inventory
Available in ≥ 2.10.
Provides a visual representation of the technology stack in your delivered source code, including any frameworks / packages that are present.
Architecture Preview
The Architecture Preview section is a graphical representation of the delivered source code before an analysis is run - this is determined during the fast scan process. The section's primary aim is to help check the completeness of the source code that has been delivered:
≥ 2.10
≤ 2.9
Identified Frameworks
The Identified Frameworks section lists all the frameworks that have been detected by Console during the fast scan phase:
The icon depicts how the identified framework will be analyzed, using the same legend as in the Software Composition section:
- Product Extension > an extension provided and supported by CAST
- Community Extension > an extension built by the CAST wider community (not supported by CAST)
- No Known Extension > this framework will not be taken into account since there is no extension available to support it
Analysis Reports
This section is only displayed when an analysis has been actioned. It displays data from the most recent analysis, therefore if you have changed the application source code since the most recent analysis (using the New Scan button) you will need to run a new analysis in order for the data in this section to be accurate.
This section provides a report on analyzed/not analyzed files for the current version:
Click to enlarge
File Extensions | A list of file extensions found in the delivered source code. Extensions are grouped by technology/language - and the extension that is displayed by default (the primary extension) is the extension with the largest number of files in the delivered source code. Other related file extensions that are found will also be displayed alongside: For the Mainframe - JCL technology, the extension .prc is not considered part of JCL language (in the vast majority of cases, these files do not contain any JCL related code), so files with this extension will be ignored in the analysis report. |
---|---|
Technology/Language | Technology or language of the file as detected by Console. |
CAST Extensions | The CAST Extension Console has used to process the file. Note that some primary file extensions may be listed as processed with multiple extensions. For example the .js file extension will appear twice:
|
Fully Analyzed | The total number of files of this type that were analyzed during the most recent analysis process. This number is taken directly from the analysis schema in which the analysis results are stored, in other words this number reflects the number of files that were saved as part of the analysis process.
|
Not Analyzed | The total number of files of this type that were sent for analysis by CAST Console but were not analyzed during the most recent analysis process (e.g. they were not saved in the results): Click to enlarge Clicking the number will display the following dialog box, providing a list of all the unanalyzed files: Click to enlarge When the Node uses AIP Core ≥ 8.3.52, a reason (where possible) will be given in the Reason column: These reasons correspond to Project Exclusion Rules set via the File Filter button (see above) for onboarding with Fast Scan: ...and via the Project Exclusion Rules screen when using onboarding without Fast Scan.
|
View Logs | Clicking this icon will direct you straight to the "Run analysis" log files. |
The search option allows you to filter for specific text. The search functions on the columns File Extensions, Technology/Language and CAST Extensions: | |
Click to download the report as a .CSV file. When opened in Microsoft Excel (or equivalent), two tabs are available:
|
Clicking a number in the list will open a popup with more details about the files:
To make results easy to use, some files are ignored and are not listed in this report:
- all files with extensions that are not associated directly to a programming language (all resources or data languages for example, or project files like xml, http or json).
- all files with patterns such as .git, .svn, node-modules, org-eclipse, CCAU\.abap, IP\.abap
- special files like package-info.java, *CT.abap, *CP.abap, hh, h++, hpp, hcc, h, hxx, ph
- files resulting from CAST Datbase extractions such as castextraction or uaxdirectory, because they are not currently directly associated with a language or extension.
Analysis Results Indicators
This section is only displayed:
- in Console ≥ 2.8.
- AND when an analysis has been actioned.
Some indicators require just an analysis to be run and some require a snapshot as well. This is noted in the Available Indicators table below.
This section displays a set of indicators for a given analysis/snapshot. These indicators are designed to provide basic information quickly so that the analysis/snapshot can be validated. The indicators are generated during the analysis/snapshot in a dedicated step and can be generated on-demand manually (see below):
- The feature can be disabled if required - see Configuring Indicators.
- You can enable and disable individual indicators in Administration Center - Settings - Analysis Results Indicators - this requires the global "Admin" role.
Click to enlarge
Categories filter | This drop down filters the Indicators in the list by category. By default, all categories are displayed in the list. |
---|---|
Update banner | This banner is displayed when Console detects that a configuration change has been made and that your data should be updated. If the "Update" button is clicked, then a job will run to ensure that all Analysis Results Indicator data is correct. The triggers for this banner are identical to the triggers described in the section Update Banner in Application - Config. |
Show all indicators | By default this toggle switch is disabled, which means only Indicators that have a positive value (displayed in the Value column) will be displayed. All indicators which have "N/A" in the Value column will be hidden. Enable the option to show all Indicators regardless of their Value. |
Snapshot selector | Choose the snapshot you would like to view indicators for. By default the most recent snapshot for the Application will be displayed. |
Search | Use this to search on the Indicator name. |
Download report | This option enables you to download Microsoft Excel reports containing detailed information about the indicators in each category:
The file name used for the ZIP file and the XLSX files will contain a time stamp using the following format: Excel file report contents
|
Recompute indicators | This option enables you to recompute the indicators without needing to generate an entire new analysis/snapshot.
Note that this button is hidden if the Update Banner (described above) is displayed). |
Indicator | See list of available indicators below for more information. |
Value | The value generated for the current snapshot. Can be a ratio or a percentage. For some indicators when only one snapshot exists, the value may be N/A, for example:
|
Status | Status of the indicator - the more stars the better the results. Rolling the mouse over the stars will show the thresholds required to improve: |
Justification | Justifications can only be edited for the most recent snapshot. If the snapshot selector is changed to a historic snapshot, the field becomes read-only. A free text field enabling you to enter a justification for the result. Free text is saved and is retained for the next snapshot that is generated. For example:
≥ 2.9 Use the icon to add a new justification explanation:
And enter the justification in the pop-up: The icon changes to indicate a justification has been added: Older releases Enter the justification in the field itself. |
Remedy Action | The Remedy Action provides a suggestion for how to improve the result in the next analysis/snapshot. Any links are clickable. ≥ 2.9 Remedies are displayed in line: If they are larger than the available space, rolling the mouse over them will display the full text in a popup: Older releases Click the icon to display the full text of the remedy: |
Details | Some indicators store results in a CSV file - click this option to download the CSV file. This can help you work out why a poor result has been produced, for example. CSV files are generated and stored in the following locations: ≥ 2.2.0 \\share\aip-node-data\common-data\snapshot-indicator\{appGuid}\{snapshotGuid} ≤ 2.1.0 - on the Node %PROGRAMDATA%\CAST\AipConsole\AipNode\snapshot-indicator\{appGuid}\{snapshotGuid} |
Clickable Indicators
Some Indicators are clickable: clicking the link will take you to the relevant configuration page within Console:
Available Indicators
Technical information about Indicators
- When a snapshot is deleted all the indicators for the snapshot along with the generated CSV files are deleted. In addition, the consolidation action launched when a snapshot is deleted will only deal with Dashboard schema indicators for the next two snapshots if they exist. During consolidation, any justification text is retained for each indicator.
- Snapshot Indicators are also deleted when a version and an application are deleted.
Logging information about Indicators
Advanced Platform Configuration
This section is only displayed if you have configured more than one of either of the following:
- Multiple CAST Storage Service/PostgreSQL instances for analysis or Measurement requirements - see Administration Center - Settings - CSS and Measurement settings.
- Multiple Nodes (i.e. you are running the enterprise release of Console which allows for multiple Nodes to be configured) - see Administration Center - Nodes.
This allows you to select the specific target CAST Storage Service/PostgreSQL instance (for the database schemas required for the new Application) OR the target Node (for deep analysis requirements). If you do not make a selection - i.e. you leave the options set to "ANY", Console will function in "load balancing" mode and will choose the CAST Storage Service/PostgreSQL or Node automatically:
- If you have ALREADY run a deep analysis, the UI will prevent you from choosing a different CAST Storage Service instance or Node for any subsequent analysis related actions to ensure stability.
- Load Balancing behaviour, when ANY is selected:
- CAST Storage Service/PostgreSQL
- For the deep analysis step (result storage), the CAST Storage Service/PostgreSQL instance with the lowest number of CAST related schemas already stored on it will be used.
- Nodes
- For the deep analysis step, the least busy node running the same release of AIP Core as used for the initial fast scan will be selected.
- CAST Storage Service/PostgreSQL
- Node manual selection: only nodes running the same release of AIP Core as used for the initial fast scan of the onboarding process will be made available for selection - this is to prevent analysis errors. This may mean that it is not possible to choose a specific node.
Run analysis
In ≤ 2.7, CAST Imaging MUST be configured Administration Center - Settings - Imaging Settings otherwise the action will fail. This requirement has been removed in ≥ 2.8.
This section provides the following:
- Information about the state of the source code
- Allows you to start an analysis
- Provides an analysis estimation time in hours and minutes. In ≥ 2.6 this estimation is valid for the analysis action and the upload to CAST Imaging (no estimation is given for CAST Dashboard actions). In previous releases, the estimation is only valid for the analysis action.
Information about the state of the source code | Information about the readiness of the delivered source code for analysis is provided based on the initial fast scan: All clear If no "issues" are found then the "all clear" is given: All clear but cannot access CAST Imaging/CAST Dashboards If no "issues" are found, but CAST Imaging/CAST Dashboards are either not configured or not available, the upload to CAST Imaging/CAST Dashboards (snapshot) will not run: Issues found If issues are found, then a warning is given with an explanation. In this situation, a warning does not mean that the analysis cannot proceed, however, coherent results may not be produced. For example:
Analysis complete When an analysis has been run, this panel will show:
|
---|---|
Run Analysis | Click the Run Analysis button to start the deep analysis process. A popup will then be displayed: When an analysis is started, a full backup of the onboarding details (e.g. delivered source code and any exclusions that have been set) and is created (in ZIP format) and is stored in the following locations (see below). This is so that any manually or automatically (via a filter) excluded folders/files can be removed before the analysis is started. When the analysis action is complete, any excluded files/folders are put back in the original location (ZIP file unzip location or source code folder location):
|
Deep analysis estimation time | The deep analysis estimation time is provided in hours and minutes and is based on anonymous statistical data that has been collected by CAST using the Allow CAST to automatically collect anonymous statistical data option in the Admin Center - see Administration Center - Settings - CAST Extend. Note that this estimation is only valid for the analysis action and does not include any other actions that may have been enabled for CAST Dashboards/Imaging. |
Advanced configuration |
This option allows you to control what steps in the analysis process are actioned and should only be used if you know what you want to achieve: |
What steps are actioned when Run Analysis is clicked?
When the Run Analysis button is clicked, the following will occur automatically depending on the configuration:
CAST Imaging configured and available | Embedded CAST Dashboards configured and available | Analysis | Upload to CAST Imaging | Security Dataflow | Snapshot generation | Upload to CAST Dashboards |
---|---|---|---|---|---|---|
Install, Configure, Analyze
The actions Install, Configure and Analyze are ALWAYS actioned regardless of your configuration:
The Finalizing Analysis entry will only be visible in the Analyze section when source code exclusions have been configured. This step restores the excluded files after the analysis has completed:
Upload
The Upload action differs depending on your configuration:
Configuration | Requirement | Actions |
---|---|---|
Any (Standard AIP Core, AIP Core for Imaging, AIP Core for Security) | CAST Imaging MUST be configured in Administration Center - Settings - Imaging Settings and accessible. |
|
With embedded Dashboards | Embedded CAST Dashboards MUST be configured and accessible. See Embedded CAST Dashboard deployment process. |
|
Additional analysis options
Depending on the configuration and license in use the following configuration will also be automatically applied when the Run Analysis button is clicked:
Option name | Target | Action |
---|---|---|
Security Dataflow | CAST Dashboards | This option focuses on user input security assessments for JEE/NET technologies. Selecting this option will:
This configuration is applied as follows:
|
Function Points | CAST Dashboards | This option focuses on function points measurement. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions): If you are using a CAST global license that does not include EFP, then this option will not produce any results. |
Tags for Data Access Sensitivity | CAST Imaging and CAST Dashboards | This option focuses on flow of data identification and will deliver associated results. Selecting this option will currently install the following extensions (in addition to any that are discovered, set to force install or those that are automatically active / shipped extensions):
GDPR / PCI DSS Two additional options specifically enable a check of a set of predefined sensitive key words related to GDPR (General Data Protection Regulation) and/or PCI-DSS (Payment Card Industry Data Security Standards) data:
In other words, enabling the GDPR option (for example) will force the check using the predefined keywords. When the analysis runs, the predefined keywords defined will be checked and if any are found in the source code a flag will be added in the analysis results on the object in question. This can be seen as below in CAST Imaging: Click to enlarge |
Resuming interrupted jobs
Should your job be interrupted for whatever reason (network issue, issue on the Node etc.), CAST Console is able to resume the job from the same point or a previous point. Take for example a job that has been interrupted in the Install step:
Returning to the Application - Overview with Fast Scan page, a Resume button will be displayed in place of Run analysis:
In addition starting CAST Console 2.9, steps that were successfully completed prior to the interruption will be displayed as follows:
Log panel | Click to enlarge |
---|---|
Job progress screen ≥ 2.9 | Click to enlarge |
Technical details for resume functionality
Technical aspects
The information provided in this panel is generated by internal tools as follows:
- ≥ 2.10: The CAST Profiler (https://profiler.castsoftware.io/) engine is used - this aligns CAST Console with CAST Profiler and brings improved scanning performance over the Framework Detector. Note that CAST Profiler will be silently and automatically updated whenever a new release of CAST Profiler is published. This is achieved as follows: a check for updates will occur roughly 1 minute after the Node service has been (re-)started, then every 24hrs. In addition, a check is made each time the "SAVE" button is pressed in Administration Center - Settings - CAST Extend (whether a field has been changed or not).
- ≤ 2.9: The Framework Detector is used.