Source pulling is a catch all when other options are not viable. It can be used to collect source code from many different places, or as in a recent real-world example, it can be used to filter out extraneous content (non-source code) from a repository before submitting a scan.
This article will provide an overview of using source pulling and a pre-scan action to circumvent challenges with scanning a repository with a lot of extraneous content in it.
What You'll Need
There are several things you'll need in order to implement a pre-scan action to accomplish source pulling:
- Administrative access to Checkmarx in order to define a pre-scan action
- File system access to the Executables folder on the Checkmarx Manager server
- Scripts that will do the gathering of pre-processing of source code
- Access to a file share where Checkmarx will ultimately pick up the source code to submit for scanning (after the pre-processing has been completed)
1. Access the file system of the Checkmarx manager and navigate to [Checkmarx Installation Directory]\Executables. This is the directory where you will place any scripts or files needed by your scripts to collect or pre-process source code.
2. Create (or place) your scripts in this directory. It is important to note that defining pre-scan actions in Checkmarx requires a Windows batch file. It is alright to have other scripts that actually do the pre-processing (e.g., exe or .ps1); however, a batch file (.bat or .cmd) must be used to execute those other scripts.
The screenshot below shows filter.bat which executes a PowerShell script called CxStage.ps1. This PowerShell script (CxStage.ps1) uses files in CxZip and CxExt.txt, which is why these additional files also reside in the Executables directory.
3. If we take a closer look at the CxStage.ps1 (PowerShell) file, we'll see that it does some pre-processing on files contained within a github repository for demonstration purposes. Once the pre-processing has been completed, the source code is dropped onto a file share \\WIN-04MAT5MP9H1\demo. This is the directory that Checkmarx will pick up the files to be submitted for the scan. Note: the Checkmarx user configuring the scan in Checkmarx must have access to this file share.
4. Now that we have created our pre-processing scripts, we must configure a pre-scan action in Checkmarx. To do this, navigate to the Checkmarx Portal, log in with administrative privileges, and browse to Management / Scan Settings / Pre & Post Scan Actions.
At the top, click, Create New Action. The Create Action dialog will be displayed. For Action Type, select Pre Scan Action, provide a name for the action, and select the Command from the drop down. In our example above, the batch file we create was called filter.bat, so we would select filter.bat.
If our script required arguments to be passed to it, we could configure that as well in the Arguments field. For this example, no arguments are required.
Once the action has been configured, click Create.
5. Now that we have defined a pre-scan action, we can use it when configuring a project. To use a defined pre-scan action, the user does not need to have administrative permissions.
Navigate to the new project screen (you can also edit an existing project). For the Location settings, choose Source Pulling, and then click the Select button.
When you press the Select button, you will be prompted to provide network credentials to authenticate your access to the file share specified in your source pulling script.
After authenticating, you'll be presented with a folder selection dialog. Specify the Root Path (the path of the file share specified in your pre-processing script, and press Go. You'll then be able to select the folder and finalize the pre-scan action by clicking OK.
6. You can now configure the remaining parameters of the project, and when a scan is run for the project, your pre-processing script will be executed and the files that are placed on the file share will be submitted for the scan.
If source pulling fails, it is possible there is some issue with your pre-processing script. To look for insight into why a source pulling job may have failed, browse to [Checkmarx Installation Directory]\Logs\JobsManager\Trace\CxJobsManagerAll.log.