User Input Security - Advanced configuration to improve performance

Introduction

Enabling User Input Security checks will impact overall analysis performance - i.e. the analysis of a given Application will take longer to complete if you enable User Input Security checks. This is due to the additional checks and metrics that need to be processed during this type of analysis. However, the impact on performance will vary according to various different aspects related to the Applications you are analyzing:

• Number of lines of code (LOC)
• Number of objects after analysis
• Number of violations after analysis

In other words, the impact will be less noticeable on small/simple Applications, whereas the impact will be more significant for larger/complex Applications. If you need to use the User Input Security with a large or complex Application and you need to try and limit the performance hit that this will have, then there are some advanced configuration settings that can be set to do this. This page explains these settings.

Availability of configuration options in AIP Core

The configuration options described in this page are only valid for specific releases of AIP Core as described below:

Configuration option	AIP Core release	Availability	Notes
Blank execution mode	≥ 8.3.15
Disable intensive searches	≥ 8.3.3		Some User Input Security related rules are not available in older releases of AIP Core, however, the mechanism to disable rules is available since 8.3.3.
Configure maximum number of steps	≥ 8.3.7
Search abort	≥ 8.3.15

Blank execution mode

When the User Input Security feature computes the rules to generate results, searches are performed to attempt to find all violations of a specific rule. The duration of this search process is largely determined by the number of entrypoints for a given flaw. For large and complex Applications, the number of entrypoints can be significant, therefore, finding out how many entrypoints exist BEFORE you run an analysis/snapshot can help you decide whether you might need to disable intensive searches. "Blank" execution mode has therefore been designed with this in mind, i.e. to help you discover the number of entrypoints in a given Application without having to actually run an analysis (including User Input Security).

Enabling blank execution mode

Locate the DataflowRunner.exe.config file at the root of your AIP Core installation folder:

DataflowRunner.exe.config

Find the following section:

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<!--<add key="MainCounterValue" value="15000" />-->
	<!--<add key="AnalyzeTargets" value="true"/>-->
</appSettings>

Add a new line as follows (the order in which the <add key> tags are added is not important):

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
	<add key="MainCounterValue" value="5000"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<!--<add key="MainCounterValue" value="15000" />-->
	<!--<add key="AnalyzeTargets" value="true"/>-->
	<add key="AnalyzeTargets" value="false"/>
</appSettings>

Save the file and then run a new User Input Security check (if you have already run an analysis, you can choose to skip the analysis in this new User Input Security check since it is not required).

Viewing the number of entrypoints in blank execution mode

Once a new analysis that includes a User Input Security check has been completed, you can check the SecurityAnalyzer.log located in the LISA folder - this log file will contain the required information about the number of entrypoints. For example:

2019-07-15 11:05:46,348 [1] INFO    !!!Process flaw 1 of 22: Path manipulation
2019-07-15 11:05:46,348 [1] INFO    !!! 276 targets not analyzed
2019-07-15 11:05:46,380 [1] INFO  Total time = 00:00:00.0299736 seconds
 
2019-07-15 11:06:07,458 [1] INFO  Summary: Total Found= 0 flaws
2019-07-15 11:06:07,474 [1] INFO  Summary= 0 distinct flaws found.

In this example, the flaw search for the File Path manipulation rule (7752) has 276 targets to process ("not analyzed" due to blank mode). Usually, flaw searches with less than 5000 entrypoints can be kept safely because these will generally complete in a reasonable time period. Searches with more than 10000 entrypoints could take more than two hours, and are therefore candidates for disabling (via Disable intensive searches) if you cannot afford the cost of search.

Disable intensive searches

As already discussed, the number of entrypoints largely determines the speed with which the flaw searches will complete. If you have determined that there are a significant number of entrypoints (see Blank execution mode), CAST recommends disabling certain rules to prevent the searches being performed. Remember that disabling a rule in this way will mean that flaws will not be found (but performance will be improved) and you should therefore proceed with caution and evaluate which rules you do not need to compute. This process involves some trial and error to see which rules you may not need and which rules provide the best performance improvement when disabled.

Disable Log Forging search

The Log Forging (see Avoid log forging vulnerabilities - 8044) flaw search is usually the search that has the highest number of entrypoints: for example, an example application of 2 million LOC had up to 100,000 entrypoints. Given the fact that each entrypoint is the start of a search and that the User Input Security can perform an average of 2 to 10 searches per second, for an "average" search (some searches are not comparable to others - e.g. searches for hard-coded passwords is fast, whereas searches for XSS and SQLi will take longer) it is easy to see how Log Forging searches can take a long time to complete for large or complex application. However, Log Forging is considered to be one of the less critical flaws compared to others and for this reason it is a good candidate to be disabled in order to save a significant amount of time.

To disable Log Forging locate the DataflowRunner.exe.config file at the root of your AIP Core installation folder:

DataflowRunner.exe.config

Find the following section:

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<!--<add key="MainCounterValue" value="15000" />-->
	<!--<add key="AnalyzeTargets" value="true"/>-->
</appSettings>

Add a new line for the rule ID 8044 and set the value to false, as follows:

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
	<add key="MetricID-8044" value="false"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<add key="MetricID-8044" value="false"/>
	<!--<add key="MainCounterValue" value="15000" />-->
	<!--<add key="AnalyzeTargets" value="true"/>-->
</appSettings>

Save the file and then run a new User Input Security check (if you have already run an analysis, you can choose to skip the analysis in this new User Input Security check since it is not required).

Disable other intensive searches

Similarly to Log Forging (see above), other search intensive rules can be disabled in the DataflowRunner.exe.config file. Add a new <add key> line in the file with the corresponding rule ID and set it to false. The following table lists the rules that can be disabled to improve performance:

Configure maximum number of steps

For each entrypoint, User Input Security performs a series of graph explorations using a depth-first search (DFS) algorithm. At each node, several sub-graphs are explored, and so on, until the end of the branch is reached. For most searches, like injection flaws, the end of the branch is reached when the parameter searched for is resolved or a sanitization method is encountered:

• parameter is made of static strings > safe, no flaw.
• part of the parameter string is a tainted input, keyed in by the user, or crafted by an attacker > flaw found, violation reported.

Some searches have so many branches and sub-branches that a cutoff mechanism is required in order that the search does not take forever to complete. This cutoff is the maximum number of nodes traversed, known as the maximum number of steps (within CAST this is known as the  MainCounterValue parameter). When the value of MainCounterValue is reached, the engine stops the search for the current entrypoint, and moves to the next entrypoint. This limit is hard-coded at 15,000 in AIP Core 8.3.0 - 8.3.28 and 1,000 in ≥ 8.3.29. While most flaws found are found after a few steps (1 to 100), a significant number of flaws are also found after 500, 1,000 or more steps.

The MainCounterValue can be modified to improve performance, but because the number of steps required to reach a flaw depends on the size, the architecture and code style of the application, the average depth of searches required to find flaws cannot easily be predicted using external parameters like the the number of projects, number for classes, or LOC etc. As such, choosing the right value for MainCounterValue is a process of trial and error. In general, CAST recommends reducing this value in increments and then performing analyses including User Input Security so that you can easily see whether there is any improvement in performance and whether the quality of the results that are generated is sufficient (this is a trade off).

Changing the MainCounterValue parameter

Locate the DataflowRunner.exe.config file at the root of your AIP Core installation folder:

DataflowRunner.exe.config

Find the following section:

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<!--<add key="MainCounterValue" value="15000" />-->
	<!--<add key="AnalyzeTargets" value="true"/>-->
</appSettings>

Add a new line as follows and enter the required MainCounterValue parameter (the order in which the <add key> tags are added is not important) - in the following example the value has been changed from the default 15000 to 5000:

<appSettings>
	<add key="MetricID-8414" value="true"/>
	<add key="MetricID-8416" value="true"/>
	<add key="MetricID-8418" value="true"/>
	<add key="MetricID-8424" value="true"/>
	<add key="MainCounterValue" value="5000"/>
</appSettings>

<appSettings>
	<!--<add key="MetricID-8044" value="true"/>-->
	<!--<add key="MainCounterValue" value="15000" />-->
	<add key="MainCounterValue" value="5000"/>
	<!--<add key="AnalyzeTargets" value="true"/>-->
</appSettings>

Save the file and then run a new User Input Security check (if you have already run an analysis, you can choose to skip the analysis in this new User Input Security check since it is not required).

Choosing a value to improve performance

Choosing the right value for MainCounterValue is a process of trial and error. In general, CAST recommends reducing this value in increments and then performing analyses including User Input Security so that you can easily see whether there is any improvement in performance and whether the quality of the results that are generated is sufficient (this is a trade off). For example:

• If you want high quality results, CAST recommends using a value of 15,000 - this value should ensure that all flaws are found but the analysis may take a long time to complete.
• If you need results very quickly for a very large application, you can use a value of 1,000 (this is the default value in AIP Core ≥ 8.3.29).

Example performance increase

The examples below shows the results of using different values for MainCounterValue. However, one size does not fit all and you should treat these results with caution:

• Not all applications can improve runtime performance by 83% and only lose 4% of flaws (Example 1)
• On certain applications, you may lose more flaws if you use an "aggressive" value for MainCounterValue.

Example 1

On a large application, approx 10,000 java classes, 700,000 LOC and 9,448 entrypoints, analyzed using AIP Core 8.3.18, two analyses were performed with different values for MainCounterValue. The table below shows the difference in performance and results using the different values:

Example 2

On a large application, approx 951 java classes, 459,000 LOC and 51,000 entrypoint,s analyses were performed with different values for MainCounterValue. The table below shows the difference in performance and results using the different values:

Choosing a value to improve results

While this aspect is the opposite goal of this documentation, it is worth mentioning that the MainCounterValue parameter can be set to a value that is higher than 15,000 or 1000 when you have the time and want to find more flaws. This approach can be used when the analysis triggers the default cutoff. When this occurs, it will be logged as "Steps=15001 (>Limit)" for the default value of 15,000. You can therefore increase the value (to 30,000, 50,000 or even 75,000) to help find even more flaws. However, doing so will greatly reduce performance.

Search abort

The search abort option provides a means to stop a specific rule flaw search "on the go". In other words, you can proactively monitor the analysis process (using the log files) and force the User Input Security check to abort a specific rule flaw search and move on to the next rule flaw search if that specific check is taking too long to complete.

Configuring search abort

To stop a specific rule flaw search, you first of all need to know the ID of the rule that is conducting the flaw search. For example, if the flaw search conducted for file path manipulation is taking too long, you can use the table above to find the rule ID (7752). Once you have the ID, you need to create a blank .TXT file named as follows:

KillID-<rule_ID>.txt

For example:

KillID-7752.txt

Place the .TXT file in the following folder - this folder will contain the BuildAgent.flaw file and the CASTIL sub folder:

LISA\<ID>\Src<ID>

Click to enlarge

As soon as the User Input Security "engine" detects the presence of the .TXT file, the search for flaws related to the rule specified in the .TXT file will abort and the "engine will move on to the next rule flaw search.

Example log file after search abort

In the following example, it is clear that the flaw search for File path manipulation (7752) will take too long (67,642 entrypoints is significant), and the search has been aborted after 300 searches have been performed via the present of the KillID-7752.txt file:

2019-07-19 11:34:40,140 [7] DEBUG Analyzed target: 300/67642. Found=0. Steps=5. Arg 0 of line 04: v8= callstatic v118<v6.+ctor(v7) // [classpath]java.io.File.+ctor(ref [classpath]java.lang.String) // C:\Temp\C_8314_2028\Deploy\JulietB1\Sources\src\testcases\CWE89_SQL_Injection\s02\CWE89_SQL_Injection__File_executeQuery_03.java (470,28)->(470,52)
2019-07-19 11:34:43,447 [7] WARN !!!ABORT analysis for diag 7752
2019-07-19 11:34:43,447 [7] INFO !!!Tested= 300/67642 entrypoints, Found= 0 flaws
2019-07-19 11:34:43,504 [7] INFO Total time = 00:00:04.1019647 seconds