NuGet Resources Extractor - 1.1

Extension ID

com.castsoftware.dmtdotnetnugetresourcesextractor

What’s new?

Please see NuGet Resources Extractor - 1.1 - Release Notes.

Description

The NuGet Resources Extractor provides the means to configure an automatic extraction of NuGet package dependencies from a NuGet repository for use during a .NET analysis. In other words, NuGet package based source code that resides in a simple local or nuget.org location. This extractor should be used when you want to extract packages (.nupkg) based source code that is stored in a NuGet repository. For example, when your .NET application contains .csproj files which have package references defined, you can use this extractor to extract those NuGet packages from the NuGet repository. Example package references in a .csproj are shown below:

<ItemGroup>
    <!-- ... -->
    <PackageReference Include="Contoso.Utility.UsefulStuff" Version="3.6.0" />
    <!-- ... -->
</ItemGroup>

packages.config

The content of the packages.config file (if this file is delivered and referenced via the .csproj file) is handled differently depending on the release of CAST Imaging Core you are using on your node to perform the analysis:

≥ 8.3.37

The Nuget Resources Extractor will identify the packages.config file and any packages defined in it will be identified and extracted. For example:

<?xml version="1.0" encoding="utf-8"?>
<packages>
    <package id="jQuery" version="3.1.1" targetFramework="net46" />
    <package id="NLog" version="4.3.10" targetFramework="net46" />
</packages>

≤ 8.3.36

The NuGet Resources Extractor will identify the packages.config file but any packages defined in it will be ignored.

Compatibility

Microsoft Windows Win32 long paths requirement

CAST highly recommends that Win32 long paths is enabled on the Node on which the .NET analysis will be run: the extracted package dependencies are stored inside a folder called “nugetPck” which is located in the Deployment folder. Depending on where the Deployment folder is located, the path to the extracted package dependencies may exceed the default 260 character length, which would cause the analyzer to fail to access the dependencies during an analysis. To enable long paths, use the Local Group Policy Editor (gpedit) on the node and drill down to:

Computer Configuration > Administrative Templates > System > Filesystem

Then double click the Enable Win32 long paths option to enable the setting:

Using the extractor

Download and installation

The extractor will be automatically downloaded and installed when at least one .csproj file is delivered for analysis.

Usage

This extractor is driven by options located in the following file on each node:

Imaging v3
%PROGRAMDATA%\CAST\Imaging\CAST-Imaging-Analysis-Node\application.yml

Console v2:
%PROGRAMDATA%\CAST\AIP-Node\application.yml

Console v1:
%PROGRAMDATA%\CAST\AipConsole\AipNode\aip-node-app.properties

V2/V3:

## HTTP V3 Nuget repository to download package dependencies https://api.nuget.org/v3/index.json or file system like file://C:/Users/johndoe/.nuget/packages nuget:
  repository: https://api.nuget.org/v3/index.json
  # depth level parameter for manual extraction By default value is 1, if set to -1 then automatic extraction happens
  depthLevel: 1

V1:

## HTTP V3 Nuget repository to download package dependencies https://api.nuget.org/v3/index.json or file system like file://C:/Users/johndoe/.nuget/packages
scanner.nuget.repository=https://api.nuget.org/v3/index.json
## depth level parameter for manual extraction By default value is 1, if set to -1 then automatic extraction happens
scanner.nuget.depthLevel=1

Where the following options are available:

repository / scanner.nuget.repository

By default this attribute is present and defines the location of a NuGet repository. It will accept two values:

• a nuget.org URL such as https://api.nuget.org/v3/index.json - this value is present by default
• or a file system path - the location input should start with protocol file:// for example: file://C:/Users/johndoe/.nuget/packages

If a .csproj file is detected in the delivered source code, the extension will be downloaded and installed as part of the analysis process. If the .csproj file contains NuGet package dependency references, the defined NuGet repository will be automatically accessed and resources will be extracted to a dedicated folder called nugetPck in the Deploy folder:

These references will then be used during the .NET analysis.

depthLevel

This option controls the extraction depth for dependencies referenced in the .csproj file:

• By default this option is set to a value of 1, which means that any referenced dependencies will only be resolved and extracted to a depth of one dependency (i.e. dependencies of referenced dependencies will not be resolved and extracted).
• Setting the value to -1 will ensure that ALL dependencies (i.e. dependencies of dependencies etc. etc.), regardless of depth, will be resolved and extracted. This can cause analysis performance issues since potentially a significant amount of dependencies will need to be resolved and extracted. In addition the size of the nugetPck folder on the node will increase significantly in size.
• Setting the value to any other number will limit the dependency extraction to the specified number.

Extraction messages

The following messages emitted by the extractor may appear during the source code extraction process

Troubleshooting

.nuget folder

In some situations, a folder called .nuget is created in the following location on the node, where <username> is the name of the user that is running the node Windows Service:

C:\Users\<username>\.nuget

This folder, if it exists, contains expanded data downloaded by this extension as part of the analysis process. The folder may grow quite large over time and over successive analyses.

There are two methods for managing the size of this folder:

• delete the folder periodically: this will free up space but will impact the performance of any future analysis because the resources will need to be re-downloaded and expanded.
• change the location of the folder to another local drive: this option involves creating a system level environment variable called NUGET_PACKAGES and setting the target to the desired folder location and then restarting the node Windows Service so that the variable is taken into account, e.g.: