Introduction and methology

This page provides real-life data about how measurement results are impacted by an upgrade to a newer release of CAST AIP.

The applications selected for reference are real customer applications, anonymized, for which we give only the concerned languages and frameworks. For each, we run a reference analysis in 7.3, 8.0 and 8.1, then upgrade the configuration to 8.1, analyze again and compare the results on a few key indicators. Any significant difference is investigated and fixed in case it is a bug. If the difference is nominal (change in quality model, removed false positive/false negative), the justification is given.

Notes:

• The version used in 7.3/8.0/8.1 is usually the latest service pack available at the time of the test. In case it makes a difference in the analysis of results (because of a fix introduced in a service pack), it will be mentioned in the justification.
• Changes of less than 1% are not systematically analyzed.

Application 1 (JEE, C#, C++ , Informix, WSDL, Shell)

Main indicators

Analysis of the variations

Size

No change since 8.0.

LOC change between 7.3 and 8.0 is caused by a change in the way the 8.0 .NET analyzer is taking the generated code into account.

Quality

Change in number of Critical Violations in 8.2 caused by an improvement of JEE rule "Avoid using Fields (non static final) from other Classes".

Figures in 7.3 not considered as relevant since the analysis was not done with the default AIP assessment model.

Function points

No change since 8.0

Application 2 (PL/SQL, JEE, FLEX, SHELL, TIBCO, WSDL)

Main indicators

Analysis of the variations

Size

Change in LoC count between 8.0 and 8.1 is caused by a fix in the way external JS libraries are taken into account. In previous versions, they were included in the line counts, when they should not have been.

Quality

Change in Critical Violations between 8.0 and 8.1.1 is caused by a fix in the rule "Avoid using Fields (non static final) from other Classes". This fix was introduced in version 8.0.2, after the 8.0 measurements for that app were done.

Change in Critical Violations between 8.1.1 and 8.2.0 is caused by an improvement to the rule "Avoid directly instantiating a Class used as a Managed Bean" in order to extend it to other types of managed beans outside of the Spring framework.

Function points

Change in Data Function Points before between 8.1.1 and 8.2.0 is caused the fix of a problem where in some cases some DB tables could be included in several data functions, which is incorrect according to the OMG specification. This fix will be included in 8.1.2. 

Application 3 (JEE, Oracle, Mainframe, C, Shell)

Main indicators

Analysis of the variations

Size

No change.

Quality

The small change in number of Critical Violations between 8.1 and 8.2 still has to be analyzed.

Function points

Between 8.0 and 8.1, the reduction in number of data function points is caused by a bug in a DMT extension installed on the test configuration, not by a change in the analysis procedure.

Between 8.1 and 8.2, the increase in number of transaction function points is caused by the fix of a bug which caused Struts objects to be lost, and transactions to be missed.

Application 4 (JEE, PL/SQL)

Main indicators

Analysis of the variations

Size

No change.

Quality

No significant change.

Function points

No change.

Application 5 (ABAP, SAP SQL)

Main indicators

Analysis of the variations

Size

No change.

Quality

Figures in 7.3 not considered as relevant since the analysis was not done with the default AIP assessment model.

Changes between 8.0 and 8.1 are linked to following 2 fixes:

• "Avoid using SQL queries inside a loop - 7424" : This rule is no longer enable for SAP analysis. It was redundant with ""Avoid Open SQL queries in loops - 7868". The change was done in the service packs 7.3.7, but was not present in the 7.3/8.0 versions used in this test.
• "Avoid missing WHEN OTHERS in CASE statements - 7518" is not longer critical in 8.1. The change was done to be consistent with similar rules in other languages.

Function points

No change.

Application 6 (C#, PL/SQL)

Main indicators

Analysis of the variations

Size

No change.

Quality

No significant change.

Function points

No change.

Application 7 (VB .NET/ASP.NET, C#, MicrosoftT-SQL)

Main indicators

Analysis of the variations

Size

No change.

Quality

No change.

Function points

Change in Function Points before between 8.1.1 and 8.2.0 is caused the fix of a problem where in some cases some DB tables could be included in several data functions, which is incorrect according to the OMG specification. This fix will be included in 8.1.2.

Application 8 (C++, PHP)

Main indicators

Analysis of the variations

Size

No change.

Quality

No change.

Function points

No change.