The purpose of Transaction graph adjustment is to allow end-users to make changes to links that are used for building Transaction graphs without affecting information stored by CAST AIP analysis tools in the Analysis Service schema.
Transaction graphs are built using objects and links stored in the Analysis Service schema by CAST AIP analysis tools (such as Analyzers, Extensions, Reference Finder, KB Update Assistant, etc.) and many of the Function Points related features are based on these Transaction graphs, whose quality strongly defines the accuracy of their results.
However, end-users often report that some links are missing, or unexpected, or oriented incorrectly. Consequently, they must change or drop some of these links to get accurate Transaction graphs, but at the cost of impacts to behavior and/or results of some other features in CAST AIP. The Transaction graph adjustment feature aims to avoid these changes that are by nature affecting areas wider than just the computation of the Transaction graphs.
Note that currently, Transaction graph adjustment is only available when the CAST AIP schemas are installed on CAST Storage Service.
Features taking into account Transaction Graph Adjustment
Transaction Graph adjustment is taken into account during Function Point computation and also during the computation of Transaction Risk Index (TRI) (CAST AIP ≥ 8.3.1)
Note: The same configuration will operate on both features, which will read the same sets of adjusted links. However, computation of the Transaction Risk Index still only reads (adjusted) links of type "Call", "Use", "Fire", and "Access", whereas computation of Function Points reads (adjusted) links of any type.
Links stored in the Analysis Service schema can be adjusted to resolve the following issues (for the moment independently of any specific Application and/or Transaction):
- missing links can be added
- unexpected links can be removed
- misoriented links can be inverted
Using these basic operations, some more elaborate operations can be created: for example it is possible to change the type and/or subtype of a link by removing it and then adding a new link with the same start and end points but with a different link type and/or subtype.
As Transaction graphs are then built using these custom "adjusted" links, these graphs can also be said to have been "adjusted".
The computation of Transaction (full) graph starts by loading all objects and all links contained in the Analysis Service schema, regardless of any Application, and then restricts (full) graph contents to a smaller set according to the entry-points and end-points of interest. Therefore, Transaction graph adjustment does not have any configuration related to a specific Application (or project, with the meaning this word has regarding the Acc.IdPro column in the Analysis Service schema).
Principles of Transaction graph adjustment
Transaction graph adjustment is accomplished by using various SQL functions that are located in the Analysis Service schema. There are two categories:
- "Standard" adjustments, all located in the SQL function named "FP_Adjusted_TF_Std_Config" (located in the Analysis Service schema). Their goal is to provide general improvements on the quality of Transaction graphs. Despite the fact that such adjustments may vary from one release to another, for example because of evolutions that occur in CAST AIP analysis tools, this SQL function must not be modified.
- "Custom" adjustments, all located in the SQL function named "FP_Adjusted_TF_Custom_Config" (in the Analysis Service schema). This SQL function is the only one that can be modified by end-users for adjusting Transaction graphs. As is standard practice for other "custom" SQL functions available in CAST AIP, this function will not be refreshed during the upgrade process from one version of CAST AIP to another.
- "Add-Ons" adjustments (from CAST AIP ≥ 8.3.30), consisting of a set of SQL functions that can be delivered and installed by Extensions and/or for bug-fix purposes. These functions are located in the Analysis Service schema, and are automatically discovered at run-time of each TF graph adjustment with the following names:
- "FP_Adjusted_TF_Ext..._Config": functions delivered by Extensions
- "FP_Adjusted_TF_Fix..._Config": functions delivered for bug-fix purposes
To keep previously computed Function Point results unchanged after an upgrade has occurred, both "Standard" and "Custom" Transaction graph adjustments are initially disabled, and will remain so by default until a specific SQL command is run for enabling their execution. However, "Add-Ons" Transaction graph adjustments are initially enabled and will remain so by default until a specific SQL command is run for disabling their execution.
The processing below occurs each time Transaction graphs are about to be computed (e.g. during Snapshot computation in the CAST Management Studio, or Function Point computing in the CAST Transaction Configuration Center), and is run on the Analysis Service schema:
- execution of Transaction graph adjustment is checked to have been granted;
- the SQL function defining Standard adjustments is run;
- the SQL functions defining adjustments that have been installed by Extensions, if any, are run (executed in alphabetical order);
- the SQL functions defining adjustments that have been installed for bug-fixes, if any, are run (also executed in alphabetical order, but only after all the SQL functions installed by Extensions were run);
- the SQL function defining Custom adjustments is run;
- adjustments are checked, and execution is aborted if any operation is discovered to be badly configured, duplicated, or conflicting with another;
- adjustments are applied to links that have been produced by CAST AIP analysis tools;
- adjusted links are saved so that they are available whenever needed: during Transaction graph building, while displaying Data or Transactional Function (full) call graph in the Transaction Configuration Center, for knowing whether a Data Function is internal or external, or whether a Transactional Function is input, external, or inquiry, etc.
Note: As the scope of links adjustment is limited to Function Points related features, adjusted links will neither be displayed in Enlighten, nor taken into account by analysis tools.
Enabling or disabling Standard and Custom Transaction graph adjustments
How to enable Standard and Custom Transaction graph adjustments
Because, by default Standard and Custom Transaction graph adjustments are initially disabled, you should run the following SQL commands to enable them:
should display the messages below (the last two being issued only as of CAST AIP ≥ 8.3.30):
How to disable Standard and Custom Transaction graph adjustments
It is possible to disable Standard and Custom Transaction graph adjustments and revert to the default behaviour (that is only the adjustments installed by Extensions and/or bug-fix purposes are enabled) by running the following commands:
should display the messages below (the last two being issued only as of CAST AIP ≥ 8.3.30):
API of FP_Adjusted_TF_Enable
Enabling or disabling computation of TF graph adjustment for Standard and/or for Custom adjustments is made using a single SQL function (in the Analysis Service schema) named FP_Adjusted_TF_Enable, which on demand can also discard, or keep in their place, all links that have been currently adjusted.
I_ToEnable = String made of 0 to 2 characters, containing the adjustment type(s) to be enabled:
- 'S': enable Standard adjustments (those defined in FP_Adjusted_TF_Std_Config);
- 'C': enable Custom adjustments (those defined in FP_Adjusted_TF_Custom_Config).
Note: for convenience, you can provide a NULL value instead of an empty string for this argument.
I_KeepAdjustments = 0 or 1, whether the currently computed adjusted links should be discarded or else kept untouched.
- if I_KeepAdjustments is 0: all currently computed adjusted links will be discarded;
- if I_KeepAdjustments is 1: all currently computed adjusted links will be kept in place.
Note: If I_ToEnable is an empty string, meaning that execution of both Standard and Custom adjustments is being disabled, then adjustments will never be refreshed until FP_Adjusted_TF_Enable is called again to enable at least one of them; in other words, calling FP_Adjusted_TF_Enable('',1) "freezes" the currently computed adjusted links. Be aware that in a "frozen" configuration, adjustment actions whose calling or called object is "any object" will no longer be expanded according to the existing links, which can lead to incorrect results.
It returns 0 if all went OK, or 1 if an error occurred (the previous configuration is left unchanged).
- Select FP_Adjusted_TF_Enable( 'SC', 0 ) > Enable Standard and Custom adjustments.
- Select FP_Adjusted_TF_Enable( 'S', 0 ) > Enable Standard adjustments and disable the Custom ones.
- Select FP_Adjusted_TF_Enable( 'C', 0 ) > Disable Standard adjustments but enable the Custom ones.
- Select FP_Adjusted_TF_Enable( '', 1 ) > Disable Standard and Custom adjustments, however keeping in place all previously adjusted links, if any.
API of FP_Adjusted_TF_Enable_Internal
As of CAST AIP ≥ 8.3.30, since FP_Adjusted_TF_Enable does not allow to disable or enable the Transaction graph adjustments delivered by Extensions and/or for bug-fix purposes, the new function FP_Adjusted_TF_Enable_Internal is supplied to allow it. Its name "_Internal" denotes a usage reserved to "advanced" users because in standard situations these adjustments should always be left enabled. The arguments of this function are the same as those of FP_Adjusted_TF_Enable, except that I_ToEnable is a String made of 0 to 4 letters encoding the adjustment category(ies) to be enabled (or disabled if the letter is absent):
- 'S': enable Standard adjustments (those defined in FP_Adjusted_TF_Std_Config);
- 'C': enable Custom adjustments (those defined in FP_Adjusted_TF_Custom_Config);
- 'E': enable adjustments delivered by Extensions (those defined by all functions whose name looks like "FP_Adjusted_TF_Ext..._Config");
- 'F': enable adjustments delivered for bug-fixes (those defined by all functions whose name looks like "FP_Adjusted_TF_Fix..._Config").
The effect of each call to this function shall then carefully checked by looking at the messages outputted by the query below:
- SELECT FP_Adjusted_TF_Enable_Internal( 'SF', 0 ) would enable Standard and all bug-fixes adjustments, disabling the Custom ones and also all those delivered by Extensions since letters "C" and "E" are absent from the input string.
Customization of Transaction graph adjustments
Function to be modified for installing "custom" adjustments
Only the SQL function named "FP_Adjusted_TF_Custom_Config" (in the Analysis Service schema) should be modified for adjusting links, and consequently Transaction graphs, independently of any Application and specific Transaction.
The body of this function starts with numerous declarations of variables for all known types and subtypes of links that can then be reused in SQL queries implementing the basic operations.
Following that, as many SQL INSERT commands that are necessary for performing the required adjustments. These INSERTs feed 3 or 5 columns of a SQL table named "FP_ADJUST_TF_CONFIG" with adjustment commands. Here are the SQL queries corresponding to each operation:
Adding a link
Removing a link
Inverting a link
- <caller-ID> and <called-ID> should be replaced by object IDs from Keys.IdKey;
- NULL means "any object ID", either as calling object, or else as called object (depending on its position in the VALUES-list), and can be used only in DELETE and INVERT actions;
- <link_type> and <link_subtype> should be, respectively, replaced by link type and subtype values, and 0 should be used for links without any subtype (for example untyped "use" links).
Doing INSERTs directly into a SQL table contrary to calling an API performing the corresponding actions allows not only adjustment of several links in a single statement, but also allows users to query any part of the Analysis Service schema using all capabilities of the SQL language, as shown by examples below:
Applying adjustments out of Snapshot or Function Point computation
Adjusted links are normally produced during Snapshot or Function Point computation. However, users may also wish to run a standalone refresh of adjusted links after they have modified the function containing their "custom" adjustments, for example for testing that checks of new adjustments won't fail.
This standalone computation is run by the following commands:
should display messages looking like those below if Standard or/and Custom adjustments are enabled:
...or like those below if both Standard and Custom adjustments are disabled:
It is important to check in above messages that Standard and Custom adjustments are enabled or disabled according to the required settings, and that one of the values below was returned by FP_Adjusted_TF_Main:
- 0 : all went OK and some adjustments have been made (there will be none if both Standard and Custom functions don't contain any adjustment action);
- 2 : validation discovered badly formed adjustments (previously existing adjusted links are left unchanged, if any);
- 3 : all went OK, but no adjustments have been made because both Standard and Custom adjustments are disabled (and, as of CAST AIP ≥ 8.3.30, because also both adjustments delivered by Extensions or for bug-fix purposes are disabled); in this case, some adjusted links will exist, or none, according to the value provided for I_KeepAdjustments while calling FP_Adjusted_TF_Enable (or, as of CAST AIP ≥ 8.3.30, while calling FP_Adjusted_TF_Enable_Internal).
In case of error, the FP_Adjusted_TF_Main function would return one of the values listed below:
- 1 : an error occurred, but previously existing adjusted links could be left unchanged (if any);
- -1 : computation of adjusted links failed and the previously existing adjusted links were lost; in this case, TF graph adjustments feature is left enabled, but Transaction graphs won't be adjusted any more until errors will have been fixed.
If execution failed, it might helpful to run "Select FP_Adjusted_TF_Main( 1 );" to obtain detailed messages in the WK_FP_ADJUSTED_TF_LOG table (replacing "0" by "1" while calling FP_Adjusted_TF_Main turns the "verbose mode" ON).
Automatically discovered TF graph adjustment functions provided by Extensions or for bug-fixes
As of CAST AIP ≥ 8.3.30, in addition to Standard and Custom adjustment functions, two new categories of adjustments are available, both being enabled by default whether after a fresh AIP installation or after an upgrade:
- adjustments delivered by Extensions
- adjustments delivered for bug-fixes
The set of adjustment functions to be executed in addition to those defining Standard and/or Custom adjustments is performed by lookup of all SQL functions whose name case-insensitively complies with the following patterns where "..." can be any characters that can appear in CAST Storage Service/PostgreSQL SQL identifiers (beware not to exceed the limit of 64 characters for the whole function name):
- "FP_Adjusted_TF_Ext..._Config" => such functions will be categorized as being adjustments delivered by Extensions
- "FP_Adjusted_TF_Fix..._Config" => such functions will be categorized as being adjustments delivered for bug-fixes
Functions will always be executed in the same order to avoid fluctuating results depending on the order in which their names will be retrieved from the system catalog, and also ensure that adjustments made in bug-fixes will be run after all those provided by Extensions. Global ordering is defined as follows:
- Standard adjustment (if enabled) - execution stops here if an error happens during this step
- Adjustments delivered by Extensions (if enabled), ordered alphabetically - execution continues whichever one or some of the functions returns an error status or fails because of a SQL exception
- Adjustments delivered for bug-fixes (if enabled), ordered alphabetically - execution continues whichever one or some of the functions returns an error status or fails because of a SQL exception
- Custom adjustments (if enabled) - execution stops here if an error happens during this step
By running the query below, it is possible to view the future and past execution orders:
Example of output:
Signature and return value
Adjustment functions installed by Extensions or for bug-fixes should be defined as follows according to the way their results shall be interpreted, knowing that executions resulting in an exception will be caught (such executions will be reported as failed):
- CREATE [ OR REPLACE ] FUNCTION FP_Adjusted_TF_..._Config() ... RETURNS void AS ...
-> execution will always be considered as successful
- CREATE [ OR REPLACE ] FUNCTION FP_Adjusted_TF_..._Config() ... RETURNS integer AS ...
-> execution will be considered as successful if the returned value is >= 0 or NULL
- CREATE [ OR REPLACE ] FUNCTION FP_Adjusted_TF_..._Config() ... RETURNS varchar AS ...
-> if the returned string does not start with an integer value possibly followed by a blank space (ASCII code 32) itself possibly followed by any sequence of characters, then execution will always be considered as successful.
Typically, the string "number of removed duplicates: 874" returned by execution of "fp_adjusted_tf_fix_clean_struts_duplicates_v1_config()" in the above example of output issued by "SELECT FP_Adjusted_TF_Get_Info();" does not start by an integer value, so execution of this adjustment function will always be considered as successful.
-> otherwise, the value appearing in front of the returned string is extracted, attempted to be converted to an integer value, and if this conversion is successful then the same rules as for functions whose return type is "integer" apply.
Typically, the string "26 links could be sanitized (65 links were ignored)" returned by execution of "fp_adjusted_tf_fix_28660_config()" in the above example of output issued by "SELECT FP_Adjusted_TF_Get_Info();" not only provides detailed information of what could be performed, but also indicates that the link sanitization succeeded because the leading integer value "26" is >= 0.
So that there's no riskof execution failure of any function (even because of an exception) would prevent the others from running, failed executions will only be reported as such in the messages issued by "SELECT FP_Adjusted_TF_Get_Info();" and into the Function Points computation log for information purposes. However, failed executions require careful examination since most probably the expected TF graph adjustment configuration won't have been performed.
Adjustment checks and log
Validation of adjustment operations
Before generating the set of adjusted links, all operations that have been recorded during execution of SQL functions which install Standard and/or Custom adjustments are checked. If configuration errors or warnings are discovered, information about the problems that have been found are logged, and computation of new adjustments is aborted if at least 1 error is reported (all previously adjusted links are normally kept unchanged and could be lost only in case of an unexpected "internal error").
After computation of Transaction graph adjustments (that is after the generation of a Snapshot, or Function Point computation from the CAST Transaction Configuration Center, or manual execution of "Select FP_Adjusted_TF_Main( 0 );", it is possible to control - in addition to inspecting log files or contents of the WK_FP_ADJUSTED_TF_LOG table - to run the following SQL command "Select FP_Adjusted_TF_Get_Info();" to find out:
- whether Standard and/or Custom adjustments are currently enabled or disabled;
- what the current level of adjustments checks is;
- whether some errors or/and warnings occurred during validation of adjustment operations;
- how many adjusted links exist in the current Analysis Service schema.
If neither any error nor any warning was found, its output will consist of the following messages (in this example Standard adjustments are disabled whereas Custom adjustments are enabled, and the default level of adjustments checks hasn't been changed):
If only warnings were issued during checks of adjustment operations, its output will consist of messages and warnings as shown below:
... where warnings starting with "redundant" denote adjustments which were applied but resulted in links already existing in the Acc table, whereas warnings starting by "defective" denote adjustments which could not be applied for the explained reason (above is the case where one attempted to delete a link which calling object ID 101007 exists in the Keys table but from which no outgoing link towards object ID 101008 exists in the Acc table of the current Analysis Service schema).
If at least one error was issued during checks of adjustment operations, its output will consist of messages and errors as shown below:
... where it is indicated that indeed the new Transaction graph adjustments couldn't be applied (because of the error), but the previously existing configuration and adjusted links were both left unchanged, hence the numbers of adjusted links are identical to those previously reported so long only warnings were issued.
Checks that are performed as of version 8.3.0
Failure of any of these checks will be logged as errors, thus preventing computation of new adjustments:
- name of the action is one of 'DELETE', 'INVERT', or 'ADD' (case-insensitive);
- for DELETE and INVERT actions:
- Calling and called objects ID aren't both "any object";
- No link type/subtype are provided.
- for ADD actions:
- Neither calling nor called objects ID is "any object";
- A link type is provided, and its value is different from 0.
- for all actions: no two of the same type have the same <caller-ID> and <called-ID> (whatever these IDs are numbers of NULL for "any object") - Note: DELETE and ADD actions can have the same <caller-ID> and <called-ID> so that type and/or subtype of links can be changed.
- no two distinct INVERT and DELETE actions have the same <caller-ID> and <called-ID> (whatever these IDs are numbers of NULL for "any object"), because it is not good practice to invert a link which is also deleted;
- no ADD action has its <caller-ID> and <called-ID> equal to those, inverted, of an INVERT action (whatever these IDs are numbers of NULL for "any object") after link(s) have been inverted, because it would result in duplicated link(s).
- for DELETE and INVERT actions:
- no two actions exist where one has 2 IDs for <caller-ID> and <called-ID>, whereas the other has the same <caller-ID> but "any object" as called object;
- no two actions exist where one has 2 IDs for <caller-ID> and <called-ID>, whereas the other has "any object" as calling object but the same <called-ID
... because it means that the action having 2 IDs for <caller-ID> and <called-ID> is overlapped by the other action which calling or called object is "any object".
Checks that can be performed as of version 8.3.3
Failure of any of these checks will be logged as warnings, thus allowing computation of new adjustments even though such warnings were issued:
- for all actions: <caller-ID> and <called-ID> correspond to IDs in Keys.IdKey;
- for DELETE and INVERT actions: <caller-ID> corresponds to an ID in Acc.IdClr and <called-ID> corresponds to an ID in Acc.IdCle;
- for ADD actions: <caller-ID> and <called-ID> don't correspond to a link already existing in Acc for which no DELETE or INVERT action exists which would delete/invert this already existing link (because this former ADD action would create a duplicate link in Acc);
- for INVERT actions: <caller-ID> and <called-ID>, once they will have been inverted, won't correspond to a link already existing in Acc for which there doesn't exist a DELETE or an INVERT action which would delete/invert this already existing link (because this former INVERT action would result in a duplicate link in Acc).
However, as performing these new checks can take up to few minutes for big applications (those having several millions of objects and links) and the lack of corrections of their warnings doesn't compromise results of Transaction graph adjustment, these checks are disabled by default. The reasons are that, i), having duplicated links is not a problem provided link types/subtypes are consistent, and ii), attempting to delete or invert a link which is already non-existent doesn't change the initial set of links.
Execution of these new checks (the next time Transaction graph adjustment will be computed) can be enabled by running the following SQL commands:
should show, among some other messages, the line below:
Conversely, disabling these new checks (that is, reverting to the default configuration) is done by running the following SQL commands:
Transaction graph adjustment log
Every call to a function related to TF graph adjustment results in traces logged into a dedicated table named WK_FP_ADJUSTED_TF_LOG, located in the Analysis Service schema. This table is emptied at the beginning of each call to FP_Adjusted_TF_Main, so that when examining its contents only the most recent messages will be shown.
Running "Select DESCRIPTION From WK_FP_ADJUSTED_TF_LOG Order by HISTORY_ID;" allows the retrieval of all messages, ordered chronologically.
When the Compute action is run and you have used the Transaction Graph Adjustment option, the Log will contain a Function Computation report containing information about the Transaction Graph Adjustment actions:
First, these lines display the current statuses of whether Standard and/or Custom TF adjustments are enabled or disabled:
- Standard TF graph adjustment : enabled or disabled
- Custom TF graph adjustment : enabled or disabled
Then, if there is at least one link which has been adjusted, the counts of deleted, inverted, and added links are shown:
- Number of deleted adjusted links : 0 or # of deleted adjusted links
- Number of inverted adjusted links : 0 or # of inverted adjusted links
- Number of added adjusted links : 0 or # of added adjusted links