Summary: this page lists:
- Impacts of changes made to CAST AIP 8.2.0 on Quality Model results post upgrade
Each CAST AIP release provides new features which improve the value of the platform and justify an upgrade. However, there are a number of changes or improvements which can impact the measurement results/grades:
- New or improved Quality Rules to perform deeper analysis
- Updates to the Assessment Model, e.g. changes to rule weights, severity or thresholds. This can be mitigated by using the "Preserve assessment model" option during the upgrade.
- Improvements of the language analysis, e.g. more fine-grained detection of objects or links
- Extended automatic discovery of files included in the analysis
- Bug fixes to improve the precision of results
- And, unfortunately, a new release may also introduce new bugs which may impact the results until they are discovered and removed
Below is a list of changes made to the current release of CAST AIP that are known to cause impacts to results. You can also consult Case Study - Measurement changes after upgrade for selected customer applications which provides a more detailed analysis based on a few sample applications.
Analyzing the root causes of impacts to measurement results/grades
The following is a general description of the steps that should be taken in order to compare pre and post upgrade results:
- Step 1: Take a snapshot (including a source code analysis) with the previous release of CAST AIP before upgrading to the new release of CAST AIP
- Check the list of applications to be analyzed, the list of files per application and list of SQL objects from the Analysis Service.
- Step 2: Compare the source code in version 1 (before upgrade) with the source code in the new version 2 (after upgrade)
- Compare the list of analyzed files, list of files per application and list of SQL objects between the two Analysis Services
- Step 3: Compare the results of the application analysis and snapshot post upgrade. This can be done by comparing the snapshots available in the Dashboard Service to find the differences in:
- Quality rules
Grades at Business Criteria level
Lines of code
Step 4: Compare the data functions and transactions across the source Analysis Service and the target Analysis Service post upgrade.
Impacts of changes made in CAST AIP 8.2.0 on Quality Model results post upgrade
New "Full Application" consolidation mode in the Assessment Model for snapshot generation in the CAST Management Studio
In CAST AIP 8.1.x, CAST has introduced a new consolidation mode in the CAST Assessment Model for snapshot generation (if you are upgrading from 7.3.x or 8.0.x then this new consolidation mode is also "new" in 8.2.x). The new mode is called Full Application and calculates grades from components of an Application (as is already the case for previous consolidation modes available based on Functional Modules). The scope of components is defined as a union of components accessed from all Functional Modules:
By default in CAST AIP 8.2.x, when using the CAST AIP 8.2.x Assessment Model, the new Full Application consolidation mode will always be used unless you manually change it. In the context of an upgrade to CAST AIP 8.2.x from 7.3.x and 8.0.x, if you choose to
- If your Application is composed of multiple auto-generated Modules (i.e. you have selected the Analysis Unit Content option or you have opted to define your own User Defined Modules as shown in the screenshot below) then results WILL differ for these modules because the Quality Rule grades will no longer be calculated on the average of all the modules, but on the entire Application instead. In this situation, if you need to retain the grades from the previous release of CAST AIP, you need to manually change the Consolidation Mode in the CAST Management Studio to one of the other four existing modes to match the mode chosen in the previous release of CAST AIP.
- If you have used the default behavior in CAST AIP, which is to automatically create one single Full Content module for each Application, then results WILL NOT change because a Full Content module contains the same components as the entire Application.
Note that you can query the DSS_HISTORY table in the CAST Dashboard Service to view the status of each existing snapshot after the upgrade process has been completed by the CAST Update Tool. Messages will be entered into the table as follows for each snapshot that is handled as part of the upgrade:
- During migration, the CAST AIP version has been updated to <CAST VERSION> for the snapshot <SNAPSHOT_ID>
- During migration, the consolidation mode has been updated to <CONSOLIDATION_MODE> for the snapshot <SNAPSHOT_ID>
Calculation of Total and Failed Checks
Whatever Consolidation Mode is used in the Assessment Model in the CAST Management Studio, Total Checks and Failed Checks are now calculated in CAST AIP 8.2.x with components that can be accessed from all Functional Modules. This is also true for Sizing Measures counting critical violations. In the context of an upgrade to CAST AIP 8.2.x (from 7.3.x or 8.0.x), if you choose toresults may differ from previous releases of CAST AIP when there is an overlap between user defined modules (i.e. when a component belongs to two or more User Defined Modules).
Avoid indirect String concatenation inside loops - 7954 / Avoid direct or indirect remote calls inside a loop - 7962
In previous releases of CAST AIP, the Quality Rules listed above were not functioning correctly and were producing false negative results. The algorithm used to generate these Quality Rules has now been improved and corrected and, after an upgrade to CAST AIP 8.2.x and the generation of a new snapshot, the number of violations may increase. This increase in violations is due to the improved accuracy provided by the bug correction. In addition, although many "paths" may exist for each violation (a violation is deemed to be a "path" that starts in a loop in a method and targets an object) the Quality Rules will now only report one "path", thus keeping the Quality Rule calculation time stable and reducing the amount of data produced.
Avoid directly instantiating a Class used as a Managed bean - 7964
In previous releases of CAST AIP, the Quality Rule listed above functioned only on Spring beans. In CAST AIP 8.2.x, this behaviour has been improved:
Potentially increased time to generate a snapshot
In CAST AIP 8.2.x, the CAST Application Analytics Dashboard now provides more detailed information about Transactions in the Application. In order to do this, additional metrics are now run on Transactions where in previous releases of CAST AIP they were not:
- failed checks for Quality Rules
- number of Violations and Critical Violations by Technical and Business criterion
- number of added and removed Violations and Critical Violations by Quality Rule, Technical and Business Criteria.
These additional metrics may cause the step entitled "Configure snapshot data into dashboard service" to take more time to process than in previous releases of CAST AIP.