Skip to main content

Local Scanner and Hook deployment

If you are integrating a new scanner or hook and want to test from a local build, this document will guide you through it. This guide assumes that you have your Kubernetes cluster already up and running and that we can work in your default namespace. If not, check out the installation for more information. We also assume that you are or have followed the steps in either the Integrating A Scanner or Integrating A Hook guide.

Makefile-based build & deploy (recommended)#

To make local testing easier, the secureCodeBox team has provided a Makefile based solution. The specific Make targets and examples to customize them are given in these documents:

This document explains how to use these targets to deploy your scanner locally.

  1. Scanner only: Inspect your scanner's Makefile. If your scanner uses a remote Docker image (such as one hosted on Dockerhub), you can comment out or remove the custom_scanner line. If you have defined your own Dockerfile in the scanner/ directory, you should leave the line as-is.

  2. In the root of the secureCodeBox git repository, under hook-sdk/nodejs/ and scanner-sdk/nodejs/, you need to build the Dockerfiles. This ensures that you have the latest version of the SDK available locally. You need this in order to build secureCodeBox parsers and hooks. To build the image:

    1. Minikube: run eval $(minikube docker-env) && make docker-build.
    2. Kind: run make docker-build.
  3. In your scanner or hook directory, build the Dockerfiles:

    1. Minikube: run eval $(minikube docker-env) && make docker-build.
    2. Kind: run make docker-build docker-export kind-import.
  4. Run make deploy to install your Helm chart in your active Kubernetes cluster into the integration-tests namespace. The make target ensures that the image name and tag matches that built in the previous step.

  5. Now run an example scan and inspect whether the images are correctly used.

Example shell#

securecodebox$ eval $(minikube docker-env)securecodebox$ cd parser-sdk/nodejs/securecodebox/parser-sdk/nodejs$ make docker-build.: โš™๏ธ Build 'parser-sdk'.docker build -t securecodebox/parser-sdk-nodejs:"sha-$(git rev-parse --short HEAD)" .[...]Successfully built af5faaf0be6eSuccessfully tagged securecodebox/parser-sdk-nodejs:sha-a4490167securecodebox/parser-sdk/nodejs$ cd ../../scanners/nmap/securecodebox/scanners/nmap$ make docker-build.: โš™๏ธ Build 'nmap' parser with BASE_IMG_TAG: 'sha-a4490167'.[...][Warning] One or more build-args [scannerVersion] were not consumedSuccessfully built 931ac83a3e42Successfully tagged securecodebox/parser-nmap:sha-a4490167.: โš™๏ธ Build 'nmap' scanner with BASE_IMG_TAG: 'sha-a4490167'.[...][Warning] One or more build-args [baseImageTag namespace] were not consumedSuccessfully built 721c154357ebSuccessfully tagged securecodebox/scanner-nmap:sha-a4490167securecodebox/scanners/nmap$ make deploy.: ๐Ÿ’พ Deploying 'nmap' scanner HelmChart with the docker tag 'sha-a4490167' into kind namespace 'integration-tests'.[...]Release "nmap" does not exist. Installing it now.NAME: nmapLAST DEPLOYED: Thu Nov 18 15:00:14 2021NAMESPACE: integration-testsSTATUS: deployedREVISION: 1TEST SUITE: Nonesecurecodebox/scanners/nmap$ kubectl get scantypes.execution.securecodebox.io -n integration-tests NAME   IMAGEnmap   docker.io/securecodebox/scanner-nmap:sha-a4490167securecodebox/scanners/nmap$ kubectl get parsedefinitions.execution.securecodebox.io -n integration-tests NAME       IMAGEnmap-xml   docker.io/securecodebox/parser-nmap:sha-a4490167
note

Notice that the Make target automatically feeds the chart's AppVersion into the scannerVersion build arg.

Manual build & deploy#

  1. Minikube only:
$ eval $(minikube docker-env).

Scanner & Parser#

  1. Build your scanner image
securecodebox/scanners/your-custom-scanner/scanner$ docker build \        --build-arg=scannerVersion="7.91-r0" \        -t your-custom-scanner:local-dev \        .
  1. Kind only: import your scanner image
securecodebox$ kind load docker-image your-custom-scanner:local-dev
  1. Build Parser SDK
securecodebox/parser-sdk/nodejs$ docker build -t securecodebox/parser-sdk-nodejs:local-dev .
  1. Build your parser image
securecodebox/scanners/your-custom-scanner/parser$ docker build \        --build-arg=baseImageTag="local-dev" \        --build-arg=namespace=securecodebox \        -t your-custom-parser:local-dev \        .
  1. Kind only: import your parser image
securecodebox$ kind load docker-image your-custom-parser:local-dev
  1. Update deployment image references. Change the fields in values.yaml file like this:
parser:  image:    repository: your-custom-parser    pullPolicy: Never    tag: local-devscanner:  image:    repository: your-custom-scanner    pullPolicy: Never    tag: local-dev

Hook#

  1. Build Hook SDK
securecodebox/hook-sdk/nodejs$ docker build -t securecodebox/hook-sdk-nodejs:local-dev .
  1. Build hook
securecodebox/hooks/your-custom-hook/hook/$ docker build \        --build-arg=baseImageTag="local-dev" \        --build-arg=namespace=securecodebox \        -t your-custom-hook:local-dev \        .
  1. Kind only: import your hook image
securecodebox$ kind load docker-image your-custom-hook:local-dev
  1. Update deployment image references. Change the fields in values.yaml file like this:
hook:  image:    repository: your-custom-hook    tag: local-dev    pullPolicy: Never
  1. Deploy the scanner.
securecodebox/hooks/your-custom-hook/$ helm upgrade --install your-custom-hook .

Debugging#

ImagePullBackOff#

Kubernetes is reporting that it isn't able to find the specified image. Check with kubectl describe pod [name of pod] which image your scanner wants to use and whether you have made that image available. Check your Docker build logs to verify that the image has been correctly tagged. You can also check if the image is actually available:

  • Minikube: minikube ssh docker images
  • Kind: docker exec kind-control-plane crictl images

Don't forget that all images you want to use in your Minikube Kubernetes cluster must be either remotely available or made available in your Kubernetes cluster.

  • Minikube: built using eval $(minikube docker-env).
  • Kind: imported after building
    • Using Makefile: make docker-export kind-import.
    • Manually: kind load docker-image parser-nmap:[tag].

Namespace#

In some cases, you might accidentally install secureCodeBox in an unexpected namespace. Check that you don't have any left-over releases installed.

$ helm list --all-namespacesNAME                    NAMESPACE               REVISION    UPDATED                                 STATUS      CHART                                   APP VERSIONnmap                    integration-tests       1           2021-11-18 15:00:14.712583292 +0100 CET deployed    nmap-v3.1.0-alpha1                      7.91-r0    securecodebox-operator  securecodebox-system    1           2021-11-18 10:09:24.804277463 +0100 CET deployed    operator-v3.1.0-alpha1                             update-category         integration-tests       1           2021-11-18 11:18:45.104860436 +0100 CET deployed    update-field-hook-v3.1.0-alpha1                    update-severity         integration-tests       1           2021-11-18 11:18:45.267164462 +0100 CET deployed    update-field-hook-v3.1.0-alpha1                    

You can install secureCodeBox components in any namespace, however verify that you are starting your scans in the same namespace as where you deployed your scanner or hook.

$ kubectl apply -f ./nmap-scan.yaml -n integration-testsscan.execution.securecodebox.io/nmap created$ kubectl get scans -ANAMESPACE           NAME   TYPE   STATE   FINDINGSintegration-tests   nmap   nmap   Done    1