Defining the Assessment Model Upgrade Strategy

What is the purpose of this feature?

alt text

These options govern how CAST Imaging should deal with Assessment Models when upgrading an application to a new release of CAST Imaging Core, and when an existing extension is upgraded to a new release.

Why do we need to configure how CAST Imaging deals with Assessment Models?

Assessment Models are the rule definitions that are embedded in CAST Imaging Core and in extensions (also know as Assessment Model “fragments”). Take for example the HTML5/JavaScript extension: this extension (like other CAST extensions) contains an Assessment Model fragment that defines multiple rules that are specific to the extension (for example, in release 2.1.1 of the HTML5/JavaScript extension, 53 rules external link were provided). Each time a new release of an extension or CAST Imaging Core is published, changes may be made to these Assessment Models (i.e. rules), for example:

  • new rules may be added
  • existing rules may be detached or disabled
  • existing settings and parameters in existing rules may be modified

Changes to rules are an inevitable (but positive) consequence of the improvement and expansion of the platform, however, changes will also impact the results of your analyses when you compare results produced with different releases of CAST Imaging Core /extension. For example:

  • new rules and bug fixes or improvements to existing rules introduced in a new release of CAST Imaging Core or an extension can potentially uncover additional violations that were not present in the analysis results performed with the previous release of CAST Imaging Core or the extension, even though the source code may not have changed.
  • likewise, bug fixes or the deactivation of existing rules in a new release of CAST Imaging Core or an extension can potentially reduce the number of violations in comparison to the number that were recorded when using the previous release of CAST Imaging Core or the extension, even though the source code may not have changed.

As a consequence, CAST Imaging provides options that allow you to make a choice about how the Assessment Models in CAST Imaging Core or in extensions are handled during an upgrade to a new release of CAST Imaging Core or a new release of an extension. These settings are applicable at global level, i.e. to all applications managed in CAST Imaging.

Preserve Assessment Model but enable new rules (default)

This option is the default option. It provides a “minimum” impact to existing analysis results, as follows:

  • the existing Assessment Model is preserved for the most part and the following settings are retained:
    • the status of existing rules that have manually been deactivated
    • existing thresholds
    • existing weights - except rules that have been manually set to 0 weight which will be reverted to the default weight
    • existing critical contributions
    • existing parameter values
    • existing custom indicators
    • existing custom rules
  • any new rules provided in the new Assessment Model will be enabled.

Existing Assessment Models will still be subject to an upgrade process and the following settings will be overwritten by the new Assessment Model:

  • SQL and property implementation settings
  • applicable technologies
  • existing associate values
  • existing documentation
  • parameter names and types

Preserve Assessment Model and get new rules for information (>= 8.3.32)

This option is designed for those that want to “preview” any new rules added to the Assessment Model. It is identical to the Preserve Assessment Model but enable new rules option, except that:

  • any new rules provided in the new Assessment Model fragment will be enabled but their weight will be set to zero
  • any rules that are reattached will be attached but their weight will be set to zero
  • any detached or deactivated rules in the Assessment Model fragment will be ignored

A rule with zero weight is a rule that is both enabled (active), but has no impact on any parent technical criterion - this means that the rule can be “previewed” (i.e. violations can be seen) without impacting any grades.

If your target CAST Imaging Core release is ≤ 8.3.31, then the legacy behaviour will be actioned, i.e.: new rules will be disabled, instead of being enabled but set to zero weight.

Upgrade Assessment Model

This option can be thought of as the “reset” option. There will be more impact to existing analysis results when using this option, as described below:

  • the existing Assessment Model fragment will be overwritten with the new Assessment Model fragment, except for:
    • the status of existing rules that have manually been deactivated existing parameter values
  • any new rules provided in the new Assessment Model fragment will be enabled