CMS Snapshot Analysis - Run Copy Paste Metrics Calculation - Error - The program XMLTODB has not ended correctly - error code 1

Created by user-4b453, last modified by Alex Thottunkel on Jan 16, 2024
Description

At the beginning of a snapshot (Run Copy Paste Metrics Calculation phase), CAST will save data in the databases by using the the XmlToDb transfer.

Refer to CMS Snapshot Analysis - Run Copy Paste Metrics Calculation - Error Free Logs - CAST AIP 83

for description of the step.

This page will explain how to manage the "The program XMLTODB has not ended correctly (error code "1")" error message.

When this issue occurs, the analysis is aborted  

Figure 1

To get more detailed information, you need to check the CAST-MS.log.txt file that is in %TEMP%/CAST/CAST/<VERSION> folder:

INF: 2014-06-17 23:20:09: starting Task Save results to database
INF: 2014-06-17 23:20:09: C:\Program Files (x86)\CAST\7.2/XMLTODB.exe /f:I:\IMPLEMENTATION\LargeStorageArea\LISA\2f38c0af551047668c694ba1e5a3a001\NaturalMetrics.datatransfer /c:C:\Program Files (x86)\CAST\7.2\InstallScripts\Common\APPW\table_appw.xml /scp:naturalmetrics /src:xml /st:sql /cs:LIBPQ:gaicvmcss6:5432,postgres /u:operator /p:****** /db:<Knowledge Base> /no_bcp
INF: 2014-06-17 23:20:26: Transferring XML -> DB without using bulk copy: this may take a while...
INF: 2014-06-17 23:20:26: XMLtoDB: Opening a connection to <connection_string> 'LIBPQ:gaicvmcss6:5432,postgres' with login 'operator'...successful connection.
INF: 2014-06-17 23:20:26: [23:20:11] Table # 1 'OBJECTS_SIMILARITIES': starting transfer...
INF: 2014-06-17 23:20:26: [23:20:14] table # 1 'OBJECTS_SIMILARITIES': continuing... ( 5000 rows transferred in 00h 00m 02s 516ms )
INF: 2014-06-17 23:20:26: [23:20:16] table # 1 'OBJECTS_SIMILARITIES': continuing... ( 10000 rows transferred in 00h 00m 05s 016ms )
INF: 2014-06-17 23:20:26: [23:20:19] table # 1 'OBJECTS_SIMILARITIES': continuing... ( 15000 rows transferred in 00h 00m 07s 547ms )
INF: 2014-06-17 23:20:26: [23:20:21] table # 1 'OBJECTS_SIMILARITIES': continuing... ( 20000 rows transferred in 00h 00m 10s 078ms )
INF: 2014-06-17 23:20:26: [23:20:24] table # 1 'OBJECTS_SIMILARITIES': continuing... ( 25000 rows transferred in 00h 00m 12s 610ms )
INF: 2014-06-17 23:20:26: [23:20:26] table # 1 'OBJECTS_SIMILARITIES': now transferred ( 29568 rows transferred in 00h 00m 14s 938ms ).
INF: 2014-06-17 23:20:26: [23:20:26] Whole transfer duration is 00h 00m 14s 938ms (for 29568 rows).
INF: 2014-06-17 23:20:26: [23:20:26] Table # 2 'ObjInf': starting transfer...
INF: 2014-06-17 23:20:26: [XMLtoDB] SQL Message: ERROR: duplicate key value violates unique constraint "pk_objinf"
INF: 2014-06-17 23:20:26: [XMLtoDB] SQL Error: DETAIL: Key (idobj, inftyp, infsubtyp, blkno)=(450, 9, 1509, 0) already exists. (Severity 1, Msg# 1).
INF: 2014-06-17 23:20:26: [XMLtoDB] SQL faulty command: in
INF: 2014-06-17 23:20:26: [23:20:26] table # 2 'ObjInf': now transferred ( 1 rows transferred in 00h 00m 00s 078ms ).
INF: 2014-06-17 23:20:26: [23:20:26] Whole transfer duration is 00h 00m 15s 016ms (for 29569 rows).
INF: 2014-06-17 23:20:26: XMLtoDB: Transfer failed (XML -> DB) with message "XML data or another error occurred while reading file 'I:\IMPLEMENTATION\LargeStorageArea\LISA\2f38c0af551047668c694ba1e5a3a001\NaturalMetrics.datatransfer': Failed request: 'insert into <Knowledge Base>.ObjInf (IdObj,InfTyp,InfSubTyp,BlkNo,InfVal) values (450,9,1509,0,1)'..".
ERR: 2014-06-17 23:20:26: The program XMLTODB has not ended correctly (error code "1").
Please see the CAST Management Studio log and/or the program log for further details
com.castsoftware.java.ExternalException:The program XMLTODB has not ended correctly (error code "1").

We can see here that there is the XmlToDb transfer command line :

<CAST Installation folder>\<Version>/XMLTODB.exe /f:<LISA path>\<Application ID>\NaturalMetrics.datatransfer /c:<CAST Installation folder>\<Version>\InstallScripts\Common\APPW\table_appw.xml /scp:naturalmetrics /src:xml /st:sql /cs:LIBPQ:<HOST>:<PORT>,postgres /u:operator /p:****** /db:<YOUR_KNOWLEDGE_BASE> /no_bcp

During "Run metrics calculation" step, the Metric Assistant saves data into files called dataset.castvector, located in the LISA folder. There is one such file per execution unit. Then those files are used during "Run Copy/Paste Metrics calculation" to compute some diagnostics.The issue here is due to old dataset.castvector files that are not cleaned up from the LISA folder. The fix is planned for Service Pack 8.2.10.

Applicable in CAST Version
Release
Yes/No
8.3(tick)
8.2.x, x<10(tick)
8.1.x, x<3(tick)
8.0.x, x<5(tick)
7.3.x(tick)
7.2.x(tick)
7.1.x(error)
7.0.x(error)
Applicable RDBMS
RDBMS
Yes/No
Oracle Server(tick)
Microsoft SQL Server(tick)
CSS3(tick)
CSS2(tick)
CSS1(error)
Action Plan

  1. Clean-up the LISA folder from all files named dataset.castvector and relaunch analysis + snapshot. To find out where is the LISA folder, refer to the analysis log file, as explained in the Configuration Section of CMS Snapshot Analysis - Run Analysis for Technology - Error Free Logs - CAST AIP 83

  2. For the current version of the application:

    1. Make sure the active AU's are the ones you want, and only the ones you want. Deactivate the unwanted ones. The delivery process can create on occasion duplicated analysis units or analysis units for sections of the code which are not desired to be analyzed based on the application scope. In general, the delivery should be validated and the analysis configuration confirmed as described in the documentation here: Set-up the analysis
    2. Click on "clean up analysis units" (for documentation on cleaning up analysis units, see:  CMS - Clean up Analysis Units). The unwanted AU's should disappear.

  3. Clean-up the knowledge base If there is only one application (execute the queries below) and re-run analysis.

    Clean up the KB
    Set search_path = <YOUR_KB>;
Truncate table Objects_similarities;
Delete from Objinf where InfTyp = 9 and InfSubTyp = 1509;

    If this did not solve the issue, or if the knowledge base has multiple applications, then contact CAST Technical Support with the following Relevant Input for assistance. 

NB for the next deliveries :

  • The unwanted analysis units will appear again in the new delivery. This is normal as they are automatically generated
  • DO NOT LAUNCH ANY ANALYSIS BEFORE YOU DEACTIVATE THE UNWANTED ANALYSIS UNITS
  • Deactivate again the unwanted AU, clean up analysis units, and then launch analysis.

 

Relevant Input

  • CAST Support Tool (CST) - alias Sherlock with options Export Log and Export Bases (MNGT+ KB). Make sure that Sherlock is taken with the same Windows user that ran the analysis, in order to fetch file CAST-MS.log.txt properly.
  • LISA folder from customer


Reference

 Ticket # 6938, 8216, 10670