Summary: this page lists:
- Impacts of changes made to CAST AIP 8.2.0 on Quality Model results post upgrade
Each CAST AIP release provides new features which improve the value of the platform and justify an upgrade. However, there are a number of changes or improvements which can impact the measurement results/grades:
- New or improved Quality Rules to perform deeper analysis
- Updates to the Assessment Model, e.g. changes to rule weights, severity or thresholds. This can be mitigated by using the "Preserve assessment model" option during the upgrade.
- Improvements of the language analysis, e.g. more fine-grained detection of objects or links
- Extended automatic discovery of files included in the analysis
- Bug fixes to improve the precision of results
- And, unfortunately, a new release may also introduce new bugs which may impact the results until they are discovered and removed
Below is a list of changes made to the current release of CAST AIP that are known to cause impacts to results. You can also consult Case Study - Measurement changes after upgrade for selected customer applications which provides a more detailed analysis based on a few sample applications.
Analyzing the root causes of impacts to measurement results/grades
The following is a general description of the steps that should be taken in order to compare pre and post upgrade results:
- Step 1: Take a snapshot (including a source code analysis) with the previous release of CAST AIP before upgrading to the new release of CAST AIP
- Check the list of applications to be analyzed, the list of files per application and list of SQL objects from the Analysis Service.
- Step 2: Compare the source code in version 1 (before upgrade) with the source code in the new version 2 (after upgrade)
- Compare the list of analyzed files, list of files per application and list of SQL objects between the two Analysis Services
- Step 3: Compare the results of the application analysis and snapshot post upgrade. This can be done by comparing the snapshots available in the Dashboard Service to find the differences in:
- Quality rules
Grades at Business Criteria level
Lines of code
Step 4: Compare the data functions and transactions across the source Analysis Service and the target Analysis Service post upgrade.
Impacts of changes made in CAST AIP 8.2.0 on Quality Model results post upgrade
New "Full Application" consolidation mode in the Assessment Model for snapshot generation in the CAST Management Studio
In CAST AIP 8.1.x, CAST has introduced a new consolidation mode in the CAST Assessment Model for snapshot generation (if you are upgrading from 7.3.x or 8.0.x then this new consolidation mode is also "new" in 8.2.x). The new mode is called Full Application and calculates grades from components of an Application (as is already the case for previous consolidation modes available based on Functional Modules). The scope of components is defined as a union of components accessed from all Functional Modules:
By default in CAST AIP 8.2.x, when using the CAST AIP 8.2.x Assessment Model, the new Full Application consolidation mode will always be used unless you manually change it. In the context of an upgrade to CAST AIP 8.2.x from 7.3.x and 8.0.x, if you choose to
- If your Application is composed of multiple auto-generated Modules (i.e. you have selected the Analysis Unit Content option or you have opted to define your own User Defined Modules as shown in the screenshot below) then results WILL differ for these modules because the Quality Rule grades will no longer be calculated on the average of all the modules, but on the entire Application instead. In this situation, if you need to retain the grades from the previous release of CAST AIP, you need to manually change the Consolidation Mode in the CAST Management Studio to one of the other four existing modes to match the mode chosen in the previous release of CAST AIP.
- If you have used the default behavior in CAST AIP, which is to automatically create one single Full Content module for each Application, then results WILL NOT change because a Full Content module contains the same components as the entire Application.
Note that you can query the DSS_HISTORY table in the CAST Dashboard Service to view the status of each existing snapshot after the upgrade process has been completed by the CAST Update Tool. Messages will be entered into the table as follows for each snapshot that is handled as part of the upgrade:
- During migration, the CAST AIP version has been updated to <CAST VERSION> for the snapshot <SNAPSHOT_ID>
- During migration, the consolidation mode has been updated to <CONSOLIDATION_MODE> for the snapshot <SNAPSHOT_ID>
Calculation of Total and Failed Checks
Whatever Consolidation Mode is used in the Assessment Model in the CAST Management Studio, Total Checks and Failed Checks are now calculated in CAST AIP 8.2.x with components that can be accessed from all Functional Modules. This is also true for Sizing Measures counting critical violations. In the context of an upgrade to CAST AIP 8.2.x (from 7.3.x or 8.0.x), if you choose toresults may differ from previous releases of CAST AIP when there is an overlap between user defined modules (i.e. when a component belongs to two or more User Defined Modules).
Avoid indirect String concatenation inside loops - 7954 / Avoid direct or indirect remote calls inside a loop - 7962
In previous releases of CAST AIP, the Quality Rules listed above were not functioning correctly and were producing false negative results. The algorithm used to generate these Quality Rules has now been improved and corrected and, after an upgrade to CAST AIP 8.2.x and the generation of a new snapshot, the number of violations may increase. This increase in violations is due to the improved accuracy provided by the bug correction. In addition, although many "paths" may exist for each violation (a violation is deemed to be a "path" that starts in a loop in a method and targets an object) the Quality Rules will now only report one "path", thus keeping the Quality Rule calculation time stable and reducing the amount of data produced.
Please note however, that results for these two quality rules are not perfectly deterministic because results depend on the order in which control-flow (in particular calls in loops) is discovered. In addition, results depend on objects and links identified by the JEE analyzer. As the analyzer improves, objects and links change for the same source code. Therefore, when upgrading to forthcoming service packs, violations reported by these quality rules can vary slightly.
Avoid directly instantiating a Class used as a Managed bean - 7964
In previous releases of CAST AIP, the Quality Rule listed above functioned only on Spring beans. In CAST AIP 8.2.x, this behaviour has been improved:
Effects of changes made to J2EE Analysis Units and Execution Units in CAST AIP 8.2.x
Due to a change in the way J2EE Analysis Units containing differing analysis configurations are analyzed in CAST AIP 8.2.x - all in one Execution Unit rather than in multiple Execution Units (see below) - you may find that after an upgrade to CAST AIP 8.2.x (from 7.3.x only) a subsequent re-analysis of source code and a snapshot generation, the number of violations for the Quality Rules listed below changes (there may be more or less violations):
- Avoid using specific logging implementation - 7678
- Avoid using deprecated objects - 4574
- Declare as static all Methods not using Instance Fields - 7254
- Blocking synchronous calls should have associated timeouts - 8100
These removed and added violations are all positive improvements, in other words the list of violations is more accurate than in previous releases of CAST AIP.
Other impacts of changes made in CAST AIP 8.2.0
Analysis Units and Execution Units
In CAST AIP 8.2.x a change has been made to Analysis Unit grouping in Execution Units due to the move to a 64-bit architecture: instead of grouping Analysis Units in Execution Units according to a set of predefined rules, all J2EE Analysis Units in an Application are now grouped in one single Execution Unit. The effect of this change is that when re-analyzing the same source code following an upgrade to CAST AIP 8.2.x (from 7.3.x only), links between objects in different Analysis Units are now more accurately identified provided that Dependencies have been correctly identified by the CAST Delivery Manager Tool (see the dependencies marked as "Discovered" in the Dependencies tab in the CAST Management Studio Application editor), therefore some links that were previously incorrectly identified in earlier versions of CAST AIP may no longer be identified and/or new links that were not previously identified in earlier versions of CAST AIP may now be found. This will have a knock on effect on the number of violations produced by certain Quality Rules - seeabove - some violations that previously existed will be removed, and new ones may be found. These removed and added violations are all positive improvements, in other words the list of violations is more accurate than in previous releases of CAST AIP.
Potentially increased time to generate a snapshot
In CAST AIP 8.2.x, the CAST Application Analytics Dashboard now provides more detailed information about Transactions in the Application. In order to do this, additional metrics are now run on Transactions where in previous releases of CAST AIP they were not:
- failed checks for Quality Rules
- number of Violations and Critical Violations by Technical and Business criterion
- number of added and removed Violations and Critical Violations by Quality Rule, Technical and Business Criteria.
These additional metrics may cause the step entitled "Configure snapshot data into dashboard service" to take more time to process than in previous releases of CAST AIP.