Page tree
Skip to end of metadata
Go to start of metadata
Summary: this section explains how scores/grades for your Applications (displayed in the Health Dashboard) are generated and how you can change the weighting to suit your environment. You should also read Grade and compliance score calculation.

Introduction

CAST AIP uses specific algorithms to determine a score/grade (from 1 (low) to 4 (high)) for all the source code in the Application for each Health Factor, Technical Criterion, Quality Rule, Quality Measure and Quality Distribution. These scores/grades are displayed as "Application level" scores/grades in the Health Dashboard), as highlighted below:

The algorithms that CAST AIP uses can be viewed in Expert mode in the CAST Management Studio > Assessment Model editor, under Quality Model > Consolidation Settings (these are not available in AIP Console). By default the Full Application mode is always used, unless it is manually changed to one of the other modes. A description of each mode is given below:

Consolidation modes

Full Application

This default mode calculates scores/grades based on an entire Application (i.e. all the objects within that entire Application). This mode was introduced in AIP Core 8.1.x to avoid the disadvantages that previous consolidation modes (based on the average of all functional modules (default or user-defined) in the Application) present, for example:

  • An Application level grade derived from an average of all function modules can change each time the configuration of the functional modules is changed (addition of new modules / removal or re-configuration of existing modules)
  • When the previous default "Simple Average" consolidation mode was used, an Application level grade was particularly sensitive to module scope: a small module contributes to the grade calculation as much as a large module

The Full Application mode has benefits over these previous consolidation modes:

  • Grades are more stable and less impacted by configuration changes such as the addition of User Defined Modules as long as the default Full Content module is always generated.
  • CAST AIP considers "User Defined Modules" as extracted views of Applications, in the sense that as long as a Full Content Module is selected, it is possible to add or remove additional User Defined Modules with no impact on the grade.
  • Legacy consolidation modes based on an average of all functional modules are still available as an option (see below).

How is the scope of objects defined for the Full Application mode?

In Full Application mode, the scope of objects that are included is determined as follows:

  • If the Full Content option is selected to generate a Full Content module (this is the default behaviour) then this module is always used to define the scope of objects. You can add additional User Defined Modules but these will not be used to define the scope of objects and will not, therefore, impact the score/grade, as long as the "automatic module management mode" which generates the Full Content module is not disabled:

Full Content enabledDefault behaviour with additional User Defined Modules
AIP Console - see Application - Config - Modules


CAST Management Studio - see CMS - Modules tab

  • If you decide to add one or multiple User Defined Modules and explicitly disable the "automatic module management mode" which generates the Full Content module, then the scope of objects is determined by a union of all the User Defined Modules. For this reason, it is highly recommended that you also tick the Unassigned Objects Content option to ensure that ALL the objects in the Application are included in the scope (your User Defined Modules may not include all the objects in the Application):
AIP Console

CAST Management Studio


  • If you wish to use the Analysis Unit Content option, please note the following:
    • If the Full Content and the Analysis Unit Content options are ticked, then a Full Content module will be generated and this will be used to define the scope of objects.
    • If you untick the Full Content option and the Analysis Unit Content option remains ticked, then the scope of objects is determined by a union of all the Analysis Unit Content modules, which are identical to a Full Content module.
    • If you add additional User Defined Modules, explicitly disable the "automatic module management mode" which generates the Full Content module and the Analysis Unit Content option remains ticked, then the scope of objects is determined by a union of all the Analysis Unit Content modules, which are identical to a Full Content module.

Disparities between Application grades and Module grades

When using the Full Application mode you should note that grades are not comparable at the context level (i.e between Application and Module levels). For example, you may find the following in your results:

  • Grade at Application level = Grade at Module level
  • Grade at Application level > Grade at Module level
  • Grade at Application level < Grade at Module level

Take the following example, where for each Module and for each Application (an application is a context larger than a single Module):

  • The "Grade" of each Metric is the result of a mapping between the "Metric" measure and a decimal value ranging from '1' to '4' using thresholds. Default thresholds being:
    • more than 99% compliance required to attain a grade of '4.00'
    • more than 90% compliance required to attain a grade of at least '3.00'
    • more than 70% compliance required to attain a grade of at least '2.00'
    • more than 30% compliance required to attain a grade of at least '1.00'
  • The "Grade" of each Technical criterion is simply the weighted average value of the Metric grades
  • The "Grade" of each Business criterion is simply the weighted average value of the Technical criterion.

This can be illustrated as follows:

  • Application A = (Module M1) ∪ (Module M2) = , while Module M1) ∩ (Module M2) = ∅.
  • 2 Activated QR
  • 1 Technical criterion
  • 1 Business Criterion
  • All the contributions are equal to 1x100%

Analysis results:

Application/ModuleObjects violated by QR2/ total CheckObjects violated by QR1/ total CheckGrade QR1Grade QR2
A1/21301/102032
M11/201/2033
M20/1300/100042

The Grades are as follows showing that it is possible for the BC grade at Application level to be lower then the BC grade at Module level for each Module within the Application:

Application/ModuleTechnical Criterion gradeBusiness Criterion grade
A2.52.5
M133
M233

Average of Modules (legacy mode)

This consolidation mode calculates grades based on the average score/grade of all functional modules (default or user-defined) in the Application. Take the following example:

  • Two modules - M1 and M2 are assigned to one Application
  • M1 contains 200 classes and M2 contains 50 classes
  • When the Average of Modules mode is selected, and when the snapshot is generated, the grade of M1 is calculated to 4.00 and M2 to 2.00. The Application level grade would then be calculated to 3.00 (calculated as (4+2) / (1+1) = 3) because the consolidation mode is an average of the two child modules.

In this mode it is easy to see the disadvantages of using an average of modules - module M2 with 75% less objects in it than M1 has the same "weight" in the calculation of the grade.

This mode was previously known as Simple Average.

Average of Modules (weighted by Business Value, legacy mode)

This consolidation mode is similar to the Average of Modules mode, except that instead of using a simple average of all functional modules (default or user-defined) in the Application, an average of the Business Value assigned to the modules will be used to determine the weighting of each module. The Business Value for each module is a value that must be manually assigned to each module as described in Background Facts and Business Value Metric upload.

Average of Modules (weighted by Background Facts, legacy mode)

This consolidation mode is similar to the Average of Modules mode, except that instead of using a simple average of all functional modules (default or user-defined) in the Application, an average of a specific Background Fact assigned to the modules will be used to determine the weighting of each module. In other words, the modules will retain their original weighting grades, but the parent application will have a more representative weighting. The Background Fact for each module is a value that must be manually assigned to each module as described in Background Facts and Business Value Metric upload: commonly, Cost and Effort values are used as Background Facts (although others exist).

When you choose this mode, an additional drop down list enables you to choose the Background Fact you require:

Average of Modules (weighted by Sizing Measure, legacy mode)

This consolidation mode is similar to the Average of Modules mode, except that instead of using a simple average of all functional modules (default or user-defined) in the Application, an average of a specific Sizing Measure will be used to determine the weighting of each module. Take the following example:

  • Two modules - M1 and M2 are assigned to one Application
  • M1 contains 200 classes and M2 contains 50 classes
    • When the Average of Modules mode is selected, and when the snapshot is generated, the grade of M1 is calculated to 4.00 and M2 to 2.00. The Application level grade would then be calculated to 3.00 (calculated as (4+2) / (1+1) = 3) because the consolidation mode is an average of the two child modules.
    • When the Average of Modules (weighted by Sizing Measure) > Number of classes is selected, and when the snapshot is generated, the Application level grade would then be calculated to 3.6 (calculated as ((4*200)+(2*50)) / (200+50) = 3.6)

When you choose this mode, an additional drop down list enables you to choose the Sizing Measure you require:

Sizing Measures are defined in the Sizing Model tab:


At the current time, it is NOT possible to use the following Functional Size related metrics for module weighting as these are not available in the CAST Management Studio - they are as follows:

  • Automated OMG-compliant Automated Function Points Estimation
  • Automated Enhancement Function Points

Examples of weighting using legacy consolidation modes

This section lists some further examples of how module weighting (when using the legacy consolidation modes) can affect Application level grades/scores:

Average of Modules

Same weight for each module:

Module ID

Module Grade

-

#1

3.50

1000

#2

2.10

1000000

#3

3.20

1000

Resulting grade for application:

2.93

=(3.5+2.1+3.2)/(1+1+1)

Average of Modules (by Sizing Measure)

10151 Number of LOC

Module ID

Module Grade

-

#1

3.50

1000

#2

2.10

1000000

#3

3.20

1000

Resulting grade for application:

2.10

=(3.5*1000+2.1*1000000+3.2*1000)/(1000+1000000+1000)

10152 Number of artifacts

Module ID

Module Grade

-

#1

3.50

200

#2

2.10

500

#3

3.20

300

Resulting grade for application:

2.71

=(3.5*200+2.1*500+3.2*300)/(200+500+300)

Average of Modules (by Business Value)

Module ID

Module Grade

-

#1

3.50

1

#2

2.10

1

#3

3.20

4

Resulting grade for application:

3.07

=(3.5*1+2.1*1+3.2*4)/(1+1+4)

Average of Modules (by Background Fact)

66002 Cost

Module ID

Module Grade

-

#1

3.50

100000

#2

2.10

200000

#3

3.20

110000

Resulting grade for application:

2.74

(3.5*100000+2.1*200000+3.2*110000)/(100000+200000+110000)



  • No labels