Introduction

Each CAST AIP release provides new features which improve the value of the platform and justify an upgrade. However, there are a number of changes or improvements which can impact the measurement results/grades:

New or improved Quality Rules to perform deeper analysis
Updates to the Assessment Model, e.g. changes to rule weights, severity or thresholds. This can be mitigated by using the "Preserve assessment model" option during the upgrade.
Improvements of the language analysis, e.g. more fine-grained detection of objects or links
Extended automatic discovery of files included in the analysis

Bug fixes to improve the precision of results
And, unfortunately, a new release may also introduce new bugs which may impact the results until they are discovered and removed

Below is a list of changes made to the current release of CAST AIP that are known to cause impacts to results. You can also consult Case Study - Measurement changes after upgrade for selected customer applications which provides a more detailed analysis based on a few sample applications.

Analyzing the root causes of impacts to measurement results/grades

The following is a general description of the steps that should be taken in order to compare pre and post upgrade results:

Step 1: Take a snapshot (including a source code analysis) with the previous release of CAST AIP before upgrading to the new release of CAST AIP
- Check the list of applications to be analyzed, the list of files per application and list of SQL objects from the Analysis Service.
Step 2: Compare the source code in version 1 (before upgrade) with the source code in the new version 2 (after upgrade)
- Compare the list of analyzed files, list of files per application and list of SQL objects between the two Analysis Services
Step 3: Compare the results of the application analysis and snapshot post upgrade. This can be done by comparing the snapshots available in the Dashboard Service to find the differences in:
- Quality rules
- Violations
- Grades at Business Criteria level
- Function Points
- Transactions
- Lines of code
Step 4: Compare the data functions and transactions across the source Analysis Service and the target Analysis Service post upgrade.

Impacts of changes made in CAST AIP 8.2.0 on Quality Model results post upgrade

New "Full Application" consolidation mode in the Assessment Model for snapshot generation in the CAST Management Studio

In CAST AIP 8.1.x, CAST has introduced a new consolidation mode in the CAST Assessment Model for snapshot generation (if you are upgrading from 7.3.x or 8.0.x then this new consolidation mode is also "new" in 8.2.x). The new mode is called Full Application and calculates grades from components of an Application (as is already the case for previous consolidation modes available based on Functional Modules). The scope of components is defined as a union of components accessed from all Functional Modules:

CAST AIP 8.2.x Documentation > Changes in results post upgrade - 8.2.0 > consolidation.jpg

By default in CAST AIP 8.2.x, when using the CAST AIP 8.2.x Assessment Model, the new Full Application consolidation mode will always be used unless you manually change it. In the context of an upgrade to CAST AIP 8.2.x from 7.3.x and 8.0.x, if you choose to Use new Assessment Model from this release of CAST AIP (see Run the update in Upgrading to CAST AIP 8.2.x) then the Full Application consolidation mode will be applied when you generate the snapshot at the end of the upgrade process (and for any subsequent snapshot thereafter). Consider the following:

If your Application is composed of multiple auto-generated Modules (i.e. you have selected the Analysis Unit Content option or you have opted to define your own User Defined Modules as shown in the screenshot below) then results WILL differ for these modules because the Quality Rule grades will no longer be calculated on the average of all the modules, but on the entire Application instead. In this situation, if you need to retain the grades from the previous release of CAST AIP, you need to manually change the Consolidation Mode in the CAST Management Studio to one of the other four existing modes to match the mode chosen in the previous release of CAST AIP.

CAST AIP 8.2.x Documentation > Changes in results post upgrade - 8.2.0 > modules.jpg

If you have used the default behavior in CAST AIP, which is to automatically create one single Full Content module for each Application, then results WILL NOT change because a Full Content module contains the same components as the entire Application.

Note that you can query the DSS_HISTORY table in the CAST Dashboard Service to view the status of each existing snapshot after the upgrade process has been completed by the CAST Update Tool. Messages will be entered into the table as follows for each snapshot that is handled as part of the upgrade:

During migration, the CAST AIP version has been updated to <CAST VERSION> for the snapshot <SNAPSHOT_ID>
During migration, the consolidation mode has been updated to <CONSOLIDATION_MODE> for the snapshot <SNAPSHOT_ID>

Calculation of Total and Failed Checks

Whatever Consolidation Mode is used in the Assessment Model in the CAST Management Studio, Total Checks and Failed Checks are now calculated in CAST AIP 8.2.x with components that can be accessed from all Functional Modules. This is also true for Sizing Measures counting critical violations. In the context of an upgrade to CAST AIP 8.2.x (from 7.3.x or 8.0.x), if you choose to Use new Assessment Model from this release of CAST AIP (see Run the update in Upgrading to CAST AIP 8.2.x) then the new calculation mode for Total Checks and Failed Checks will be applied when you generate the snapshot at the end of the upgrade process (and for any subsequent snapshot thereafter). As such, results may differ from previous releases of CAST AIP when there is an overlap between user defined modules (i.e. when a component belongs to two or more User Defined Modules).

JEE

Avoid indirect String concatenation inside loops - 7954 / Avoid direct or indirect remote calls inside a loop - 7962

In previous releases of CAST AIP, the Quality Rules listed above were not functioning correctly and were producing false negative results. The algorithm used to generate these Quality Rules has now been improved and corrected and, after an upgrade to CAST AIP 8.2.x and the generation of a new snapshot, the number of violations may increase. This increase in violations is due to the improved accuracy provided by the bug correction. In addition, although many "paths" may exist for each violation (a violation is deemed to be a "path" that starts in a loop in a method and targets an object) the Quality Rules will now only report one "path", thus keeping the Quality Rule calculation time stable and reducing the amount of data produced.

Avoid directly instantiating a Class used as a Managed bean - 7964

In previous releases of CAST AIP, the Quality Rule listed above functioned only on Spring beans. In CAST AIP 8.2.x, this behaviour has been improved: it now functions with all framework beans supported by CAST AIP (Struts, CDI, Jsf, Hibernate etc.). As such, after an upgrade to CAST AIP 8.2.x and the generation of a new snapshot on existing source code, the number of violations may increase for this Quality Rule.

Provide accessors to Private Fields - 4576

In previous releases of CAST AIP this Quality Rule would be violated if injected fields of classes managed by a framework such as Struts, CDI, JSF, Hibernate etc. did not include a setter. This requirement for a setter in this particular circumstance has now been removed from the Quality Rule. As such, after an upgrade to CAST AIP 8.2.x and the generation of a new snapshot on existing source code, the number of violations may change for this Quality Rule.

Effects of changes made to J2EE Analysis Units and Execution Units in CAST AIP 8.2.x

Due to a change in the way J2EE Analysis Units containing differing analysis configurations are analyzed in CAST AIP 8.2.x - all in one Execution Unit rather than in multiple Execution Units (see Analysis Units and Execution Units below) - you may find that after an upgrade to CAST AIP 8.2.x (from 7.3.x only) a subsequent re-analysis of source code and a snapshot generation, the number of violations for the Quality Rules listed below changes (there may be more or less violations):

Avoid using specific logging implementation - 7678
Avoid using deprecated objects - 4574
Declare as static all Methods not using Instance Fields - 7254
Blocking synchronous calls should have associated timeouts - 8100

These removed and added violations are all positive improvements, in other words the list of violations is more accurate than in previous releases of CAST AIP.

Other impacts of changes made in CAST AIP 8.2.0

Analysis Units and Execution Units

In CAST AIP 8.2.x a change has been made to Analysis Unit grouping in Execution Units due to the move to a 64-bit architecture: instead of grouping Analysis Units in Execution Units according to a set of predefined rules, all J2EE Analysis Units in an Application are now grouped in one single Execution Unit. The effect of this change is that when re-analyzing the same source code following an upgrade to CAST AIP 8.2.x (from 7.3.x only), links between objects in different Analysis Units are now more accurately identified provided that Dependencies have been correctly identified by the CAST Delivery Manager Tool (see the dependencies marked as "Discovered" in the Dependencies tab in the CAST Management Studio Application editor), therefore some links that were previously incorrectly identified in earlier versions of CAST AIP may no longer be identified and/or new links that were not previously identified in earlier versions of CAST AIP may now be found. This will have a knock on effect on the number of violations produced by certain Quality Rules - see Effects of changes made to J2EE Analysis Units and Execution Units in CAST AIP 8.2.x above - some violations that previously existed will be removed, and new ones may be found. These removed and added violations are all positive improvements, in other words the list of violations is more accurate than in previous releases of CAST AIP.

Miscellaneous

Potentially increased time to generate a snapshot

In CAST AIP 8.2.x, the CAST Application Analytics Dashboard now provides more detailed information about Transactions in the Application. In order to do this, additional metrics are now run on Transactions where in previous releases of CAST AIP they were not:

failed checks for Quality Rules
number of Violations and Critical Violations by Technical and Business criterion
number of added and removed Violations and Critical Violations by Quality Rule, Technical and Business Criteria.

These additional metrics may cause the step entitled "Configure snapshot data into dashboard service" to take more time to process than in previous releases of CAST AIP.