Created by James Hurrell, last modified on Feb 25, 2021


This section describes the addition of new indicators to an existing Assessment Model

Business Criterion creation

Process overview

  • Business Criterion declaration in CAST Management Studio
    • Open the target Assessment Model
    • Click on [+] in the "Quality Model > Business Criteria" tab
    • Make it « active »
      • External ID
      • Active status
      • Aggregate Technical Criterion with the right weight and critical contribution
    • Fine-tune it
      • Documentation

Business Criterion declaration in CAST Management Studio

Adding a new Business Criterion in an Assessment Model

  • Open the target Assessment Model
    • CAST default 7.1.0 Assessment Model is already installed in Assessment Models view
  • Click on [+] in the "Quality Model > Business Criteria" tab

Make the new Business Criterion "active"

  • Ensure that the Business Criterion is correctly hooked into the Assessment Model
    • Active status
      • For temporary disabling (as it keeps the Business Criterion attached, re-activation is a single click away: no need to re-attach the Business Criterion to Technical Criteria, to re-set the Aggregate Weight and to re-set the Critical contribution option)
    • Attachment to Technical Criteria
      • Aggregate weight
      • Critical contribution
        • To cap parent grade with child grade
    • External ID
      • Customer IDs should be over 2000000
      • Wizard will propose the next available ID: you can therefore always type in 2000000

"Fine-tune" the new Business Criterion

  • Finalize the configuration
    • Documentation is key
      • Description, justification/rationale
      • To help portal end-user understand this specific quality goal
      • To teach portal end-user about this specific quality goal

Technical Criterion creation

Process overview

  • Technical Criterion declaration in CAST Management Studio
    • Open the target Assessment Model
    • Click on [+] in the "Quality Model > Technical Criteria" tab
    • Make it « active »
      • External ID
      • Active status
      • Aggregate into Technical Criterion with the right weight and critical contribution
      • Aggregate Quality Rules, Distributions, and Measures with the right weight and critical contribution
    • Fine-tune it
      • Documentation

Technical Criterion declaration in CAST Management Studio

Adding a new Technical Criterion in an Assessment Model

  • Open the target Assessment Model
    • CAST default 7.1.0 Assessment Model is already installed in Assessment Models view
  • Click on [+] in the "Quality Model > Technical Criteria" tab

Make the new Technical Criterion "active"

  • Ensure that the Technical Criterion is correctly hooked into the Assessment Model
    • Active status
      • For temporary disabling (as it keeps the Technical Criterion attached, re-activation is a single click away: no need to re-attach the Business Criterion to Technical Criteria and Quality Rules, Distributions, and Measures, to re-set the Aggregate Weight and to re-set the Critical contribution option)
    • Attachment to Technical Criteria
      • Aggregate weight
      • Critical contribution
        • To cap parent grade with child grade
    • Attachment to Quality Rules, Distributions, and Measures
      • Aggregate weight
      • Critical contribution
        • To cap parent grade with child grade
        • For Quality Rules, to highlight their violations in portal
    • External ID
      • Customer IDs should be over 2000000
      • Wizard will propose the next available ID: you can therefore always type in 2000000

"Fine-tune" the new Technical Criterion

  • Finalize the configuration
    • Documentation is key
      • Description, justification/rationale
      • To help portal end-user understand this specific quality area
      • To teach portal end-user about this specific quality area

Quality Rule creation

Please remember that the CAST Architecture Checker is designed to let you build Architectural Models that will be automatically turned into Quality Rules at Snapshot generation time.

The process below deals with non-Architectural Quality Rules.

Process overview

  • Quality Rule declaration in CAST Management Studio
    • Open the target Assessment Model
    • Click on [+] in the "Quality Model > Quality Rules" tab
      • Or duplicate an existing similar Quality Rule
    • Make it « valid »
      • I.e., fill all required configuration options (listed in Validation view)
      • Including the names of Detail and Total Procedures
    • Make it « active »
      • External ID
      • Active status
      • Attached to a Technical Criterion with the right weight and critical contribution
    • Fine-tune it
      • Compliance-to-grade thresholds
      • Documentation
  • Develop Detail and Total SQL procedures
    • In any SQL IDE
    • Based on Analysis Service content: Objects, Links, Properties
    • Including new
      • Properties generated by custom Metric Assistant configuration
      • Links generated by 7.1 Dependency management
      • Injected through the Universal Importer API…

Quality Rule declaration in CAST Management Studio

Adding a new Quality Rule in an Assessment Model

  • Open the target Assessment Model
    • CAST default 7.1.0 Assessment Model is already installed in Assessment Models view

  • Click on [+] in the "Quality Model > Quality Rules" tab

    • [New in 7.1] Or duplicate an existing Quality Rule if most of the documentation and configuration is similar

Make the new Quality Rule "valid"

  • Ensure all the required field are filled. Empty required field will cause the Quality Rule to be "invalid"
  • To make it "valid"
    • List invalid configuration areas in the Validation View
      • Double-click on a row to get the focus on the right area
    • Fill in Associated information name, number, and type
      • Depending on the implementation and the nature of the information it returns to help understand the violation
      • If Detail SQL procedure only returns IDs of Objects that violate the Quality Rule, Associated value should be "None", Number of Associated Value should be "Single Value", and you can keep Associated Values Name empty
      • If Detail SQL procedure returns IDs of Objects that violate the Quality Rule as well as IDs of Objects that are involved in the Violation pattern, Associated value should be "Object", Number of Associated Value should be "Multiple Value", and you should fill Associated Values Name with a name to help understand what these involved Objects are.

    • XXL option

      • N/A when the quality check is not available as two Quality Rules that apply to XXL/all SQL tables. This is the usual case.

      • XXL when the quality check is available as two Quality Rules, one of which must be triggered only when SQL Table size data is injected in CAST AIP (the one with the XXL value)
      • non-XXL when the quality check is available as two Quality Rules, one of which must be triggered only when SQL Table size data is not injected in CAST AIP (the one with the non-XXL value)

    • Unified option

      • N/A when the quality check is not available as X Quality Rules, one of which apply to many technologies at once and X-1 of which apply to one given technology at a time. This is the usual case.

      • Unified when the quality check is available as X Quality Rules, one of which apply to many technologies at once (the one with the Unified value)
      • non-Unified when the quality check is available as X Quality Rules, X-1 of which apply to one given technology at a time (the ones with the non-Unified value)

    • Computing configuration
      • Only "Computing based on SQL Procedures" type is open to customers
        • "Computing based on Property and Scope" type is not documented: it is a configuration based on the ID of a Scope of Objects to check and on the ID of a Property that identifies Objects with Violations to the Quality Rule
        • "Computing based on Object Sets" type is not documented and will be in use with the oncoming Quality Rule Studio: it is a configuration based on a "Scope" Object Set that contains the Objects to check and on a "Violation" Object Set that contains the Objects with Violations to the Quality Rule
      • Select applicable Technology(ies)
        • One or many
      • Provide names of Total and Detail SQL procedures

        • The Total SQL procedure counts the number of analyzed objects. For the Total Procedure you should, if possible use an existing procedure – use a procedure specified for another Quality Rule. For example, if the new Quality Rules checks an item that already has a Total procedure, you should use it. If not, then you will need to create a new one using an SQL IDE and using an existing Total Procedure as a model.

        • The Detail SQL procedure lists the number of objects with an error. For the Detail procedure, you will nearly always have to create a new procedure, unless you are creating a copy of an existing Quality Rule and merely want to add an additional parameter value. You will need to create a new one using an SQL IDE and using an existing Detail SQL procedure as a model.

        • More on these SQL procedures below

Make the new Quality Rule "active"

  • Ensure that the Quality Rule is correctly hooked into the Assessment Model
    • Active status
      • For temporary disabling (as it keeps the Quality Rule attached, re-activation is a single click away: no need to re-attach the Quality Rule to a Technical Criterion, to re-set the Aggregate Weight and to re-set the Critical contribution option)
    • Attachment to a Technical Criterion
      • Aggregate weight
        • CAST recommendation: 1-to-9 mapping « nice to have » -to- « must have »
      • Critical contribution
        • To highlight violations in portal and cap parent grade

    • External ID
      • Customer IDs should be over 2000000
      • Wizard will propose the next available ID: you can therefore always type in 2000000

"Fine-tune" the new Quality Rule

  • Finalize the configuration
    • Compliance-to-grade thresholds
      • Define tolerance level
      • CAST recommendation:
        99% / 95% / 90% / 50% to get 4.00 and more than 3.00 / 2.00 / 1.00 grade for regular Quality Rules
        99.99% / 99.5% / 99% / 98% to get 4.00 and more than 3.00 / 2.00 / 1.00 grade for critical Quality Rules

    • Documentation is key
      • Description, justification/rationale, sample, scope, …
      • To help portal end-user understand the issue
      • To teach portal end-user about quality issues


Develop Detail and Total SQL procedures

Objectives

  • Detail SQL procedures must return the IDs of Objects violating the Quality Rule
  • Detail SQL procedures can return associated information to help understand the Violation
    • Involved Object(s), value, text, …: this will drive the configuration of the Associated information (cf. "Make the new Quality Rule 'valid'" section above)
  • Total SQL procedures must return the count of Objects that were checked against the Quality Rule

How to get a Total SQL procedure

  1. Reuse an existing Total SQL procedure from a Quality Rule applicable to the same Objects. This is CAST recommendation.
  2. Or create a copy of the Total SQL procedure from similar Quality Rule
  3. (T-SQL, PL-SQL, and SQL-PSM alone, N/A for CASTStorageService) Or create a procedure based on templates from <installation root>\TPL\QualityRules directory
    • To test the code outside of AIP
      • replace /*<NoTest>*/ with /*<NoTest>
      • and /*</NoTest>*/ with </NoTest>*/
    • Then run in SQL IDE
    • More on that in dedicated section below


How to get a Detail SQL procedure


  1. Create a copy of the Detail SQL procedure from similar Quality Rule. This is CAST recommendation.

    E.g.

    • Avoid the use of is inside loops relies on DIAG_SCOPE_NETEMEM003 for property-based .NET violation identification
    • Create a copy and update the following statements to use another property:
      ... and T2.METRIC_TYPE  ='Number of XYZ‘ and T2.METRIC_VALUE  > X ...
    • Cf. "How to develop new Metric Assistant Properties" section below to create new properties to use in such Detail SQL procedure)

  2. (T-SQL, PL-SQL, and SQL-PSM alone, N/A for CASTStorageService) Or create a procedure based on templates from <installation root>\TPL\QualityRules directory

    • To test the code outside of AIP
      • replace /*<NoTest>*/ with /*<NoTest>
      • and /*</NoTest>*/ with </NoTest>*/
    • Then run in SQL IDE
    • More on that in dedicated section below


How to develop new Metric Assistant properties

Declare new Metric Assistant properties
  • This is done with <prefix>CastMetrics.xml files that contain Regular Expression-based configurations
    • <prefix> is user defined and is helpful to manage multiple such files
    • Located in
      • (Windows 2008/Vista/7) %ALLUSERSPROFILE%\CAST\CAST\7.1\Configuration\Languages
      • (Windows 2003/XP) %ALLUSERSPROFILE%\Application Data\CAST\CAST\7.1\Configuration\Languages
    • E.g.: New basic C# regular expression
Test new Metric Assistant properties
  • Run Analysis on your sample code: no need to Take a Snapshot to do so
  • Checking the "run metrics ..." log to look for <prefix>CastMetrics.xml parsing error
  • Check in Enlighten property panel
  • or in CDV_OBJECT_METRICS


Reference documentation
Best practices to develop new Metric Assistant properties
  • Use sample source code (small size, for development purposes, to take few seconds to analyze) with identified positive match
  • Do not develop "OR" between multiple Regular Expression: use the capability of the Metric Assistant configuration to handle multiple <REGEXP></REGEXP> blocks
  • Do not develop "embedded" search patterns in Regular Expression: use the capability of the Metric Assistant configuration to handle nested search <EMBEDDED><BEGIN><REGEXP></REGEXP></BEGIN><VALUE><REGEXP></REGEXP></VALUE><END><REGEXP></REGEXP></END></EMBEDDED>
  • Do not develop search in code/comment/strings in Regular Expression: use the capability of the Metric Assistant configuration to handle the processing of code/comment/stings streams with <SEARCH_IN_CODE>/<SEARCH_IN_COMMENT>/<SEARCH_IN_STRING> options
  • Using Metric Assistant capabilities, start developing the configuration skeleton
  • Then fill in the holes of the configuration with appropriate Regular Expression
  • Build Regular Expression incrementally 
  • Reference Pattern can be used to help test Regular Expression as long as previous recommendations are followed

More on Detail and Total SQL procedures

Overview
This section provides more details about the Detail and Total procedures:
  • Purposes:
    • Detail: track non-compliant items or ‘defects'
    • Total: compute the size of the group of items for which to track defects
  • Expected results:
    • Detail: list of object IDs
    • Total: number of objects
  • Storage area:
    • Detail: DSS_METRIC_SCOPES
    • Total: DSS_METRIC_RESULTS
The following diagram provides a visual explanation of how the Detail and Total procedures work:
Developing Procedures
Some tips for developing this type of procedure:
  • Constraints:
    • Must fit into the framework - i.e., procedure envelopes are mandatory
  • Development facilitators:
    • Templates delivered in: <installation folder>\TPL\QualityRules
    • For T-SQL, PL-SQL, and SQL-PSM
      • 2 Total procedure templates: Tpl[Syb|Ora|Db2]ValueProc[Generic|]Total.txt
      • 4 detail procedure templates: Tpl[Syb|Ora|Db2]TreeProc[Generic|MetricDescType|NamingParamChar|Undocumented]Total.txt
  • Development tip:
    • CAST recommends copying/pasting script from existing procedures provided by CAST when editing or creating stored procedures
Templates (T-SQL, PL-SQL, and SQL-PSM alone, N/A for CASTStorageService)
The templates provided by CAST (located in <installation folder>\TPL\QualityRules) require some explanation:
  • Mandatory code is located between the /*<NoTest>*/ and /*</NoTest>*/ tags – this must NOT be modified.
    • Customizable code is located outside these tags
  • Search for and locate each “%” location as these sections require updating:
    • E.g.: add a table/view to the query
    • E.g.: add a WHERE clause
    • E.g.: add a GROUP BY clause (for Total)
  • Get types from DSS_OBJECT_TYPES
    • OBJECT_GROUP 0 for Object Types from development languages
    • OBJECT_GROUP 2 for Technological Objects
  • Warning:
    • They do not contain Violation Exclusion clause (to be copied from regular procedures)
    • In spite of NoTest tags, it is worth reviewing the overall syntax

Quality Measure and Distribution creation

As of now, creating Quality Measures and Distributions is not documented. Please contact CAST for help in this matter.