Deploying DTP in Kubernetes with a Helm Chart

Parasoft has published an official Helm chart to Docker Hub for your convenience. Full installation instructions are included in the readme. See https://hub.docker.com/r/parasoft/dtp-helm.

Deploying DTP in Kubernetes Manually

Prerequisites

Before you start, establish an external database for DTP. You may create a new, empty database or provide an existing one. In either case, a database user must be set up with any necessary permissions required to access the database. Supported databases are MySQL, Oracle, and PostgreSQL (for supported versions, see Requirements and Support). You can use one of the MySQL, Oracle, or PostgreSQL Docker images available on Docker Hub.

Then create a namespace for DTP to run in. For example:

kubectl create namespace parasoft-dtp-namespace

Note: The namespace name "parasoft-dtp-namespace" is used throughout this documentation in command and resource examples. If you use a different name for your namespace, be sure to change any instances of "parasoft-dtp-namespace" in those examples to your namespace name.

Once DTP has been licensed, deleting the namespace will invalidate machine-locked licenses, even if you recreate the same namespace.

Next, you will need a Kubernetes cluster. After starting the cluster, create the service account and permissions required by the DTP pod and related resources. An example of a yaml file that might be used to for this purpose is shown below.

parasoft-permissions.yaml

# Stable access for clients to license server
apiVersion: v1
kind: ServiceAccount
metadata:
  name: parasoft-account
  namespace: parasoft-dtp-namespace
automountServiceAccountToken: true
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: parasoft-read
  namespace: parasoft-dtp-namespace
rules:
- apiGroups:
  - ""
  resources:
  - "namespaces"
  - "pods"
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: parasoft-read-bind
  namespace: parasoft-dtp-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: parasoft-read
subjects:
- kind: ServiceAccount
  name: parasoft-account
  namespace: parasoft-dtp-namespace

Use your yaml file to create the required service account and permissions before creating the DTP environment:

kubectl create -f parasoft-permissions.yaml

You should see something similar to the output below in your console:

serviceaccount/parasoft-account created
role.rbac.authorization.k8s.io/parasoft-read created
rolebinding.rbac.authorization.k8s.io/parasoft-read-bind created

Custom Keystore

If you want to set up a custom keystore, you will need to create a configuration map for the .keystore and server.xml files. The command below creates a configuration map called "keystore-cfgmap" with file mappings for the custom .keystore and server.xml files. In this example, each file mapping is given a key: "keystore" for the .keystore file and "server-config" for the server.xml file. While giving each file mapping a key is not necessary, it is useful when you don't want the key to be the file name.

~$ kubectl create configmap keystore-cfgmap --from-file=keystore=/path/to/.keystore --from-file=server-config=/path/to/server.xml
configmap/keystore-cfgmap created

DTP Setup

To set up DTP, create a yaml file that defines a Secret (optional), volume, Pod or StatefulSet, internal-access Service, and external-access Service (optional). The Secret is used to pull the DTP image from the repository. The Pod or StatefulSet creates a pod set up to run DTP, Data Collector, and Extension Designer in separate containers. Each container is configured with a volume to persist data and a liveness probe, which is the Kubernetes equivalent of a Docker Healthcheck. The internal-access Service exposes the DTP pod to other pods, allowing them to communicate via the Service name instead of an explicit IP address. The external-access Service makes DTP, Data Collector, and Extension Designer accessible via external clients by allocating ports in the node and mapping them to ports in the pod. Example yaml files for a Pod or a StatefulSet (both called "parasoft-dtp.yaml") are shown below. These examples use an NFS volume, but this is not required; use whatever volume type fits your needs.

Note: kind: Deployment is not supported. Use either kind: Pod or kind: StatefulSet, which are supported.

Once DTP has been deployed using Pod or StatefulSet, switching the Kind will invalidate machine-locked licenses.

Example yaml using 'kind: Pod'

parasoft-dtp.yaml

apiVersion: v1
kind: Pod
metadata:
  name: dtp
  namespace: parasoft-dtp-namespace
  labels:
    app: DTP
spec:
  volumes:
    - name: dtp-data
      nfs:
        server: NFS_SERVER_HOST
        path: /dtp/
# Uncomment section below if you are setting up a custom keystore; you will also need to uncomment out the associated volumeMounts below
#    - name: keystore-cfgmap-volume
#      configMap:
#        name: keystore-cfgmap
  securityContext:
    runAsNonRoot: true
  containers:
    - name: dtp-server
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
        seccompProfile:
          type: RuntimeDefault    
      image: DTP_DOCKER_IMAGE
      imagePullPolicy: Always
      env:
        - name: PARASOFT_POD_NAME				#REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: PARASOFT_POD_NAMESPACE		    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace 
# To configure DTP to connect to your database on startup, please provide your database type, user, user password, and connection URL to the container environment by injecting the values as the DB_TYPE, DB_USER, DB_PASSWORD, and DB_URL environment variables.
# To prevent exposing sensitive data, please create a user password secret to use for the DB_PASSWORD environment variable.
# Note that the database type must be one of the following: mysql | oracle | postgresql
# Note that the environment variable values will override the equivalent persisted values in the PSTRootConfig.xml each time the container or pod is restarted.
# The following are example settings for a MySQL container called "mysql-container", a database called "DTP", a user called "dtp_user", and a user password secret.
#        - name: DB_TYPE
#          value: "mysql"
#        - name: DB_USER
#          value: "dtp_user"
#        - name: DB_PASSWORD
#          valueFrom: 
#            secretKeyRef: 
#              name: YOUR_DB_USER_PASSWORD_SECRET
#              key: YOUR_DB_USER_PASSWORD_SECRET_KEY
#        - name: DB_URL
#          value: "jdbc:mysql://mysql-container:3306/DTP"
# To configure DTP to automatically download the driver for your database on startup, please provide the JDBC driver URL to the container environment by injecting the value as the JDBC_DRIVER_URL environment variable.
# The following is an example URL to download the JDBC driver for MySQL 8.0.30.
#        - name: JDBC_DRIVER_URL
#          value: "https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.30/mysql-connector-java-8.0.30.jar"
# To configure DTP to use JVM arguments, please provide the arguments to the container environment by injecting the value as the JAVA_CONFIG_ARGS environment variable.
# The following is an example JVM argument "com.parasoft.sdm.storage.managers.admin.enable.delete.project.data=true"
#        - name: JAVA_CONFIG_ARGS
#          value: "-Dcom.parasoft.sdm.storage.managers.admin.enable.delete.project.data=true"
      args: ["--run", "dtp"]
      ports:
        - name: "http-server"
          containerPort: 8080
        - name: "https-server"
          containerPort: 8443
      volumeMounts:
        - mountPath: "/usr/local/parasoft/data"
          name: dtp-data
# Uncomment section below if you are setting up a custom keystore. Note that updates made to these files will not be reflected inside the container once it's been deployed; you will need to restart the container for it to contain any updates.
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/dtp/tomcat/conf/.keystore"
#          subPath: keystore
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/dtp/tomcat/conf/server.xml"
#          subPath: server-config
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
      livenessProbe:
        exec:
          command:
          - healthcheck.sh
          - --verify
          - dtp
        initialDelaySeconds: 120
        periodSeconds: 60
        timeoutSeconds: 30
        failureThreshold: 5
    - name: data-collector
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
        seccompProfile:
          type: RuntimeDefault    
      image: DTP_DOCKER_IMAGE
      imagePullPolicy: Always
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects JAVA_DC_CONFIG_ARGS
#      env:
#        - name: JAVA_DC_CONFIG_ARGS
#          value: "-Dcom.parasoft.sdm.dc.traffic.max.length=1000000"
      args: ["--run", "datacollector", "--no-copy-data"]
      ports:
        - containerPort: 8082
      volumeMounts:
        - mountPath: "/usr/local/parasoft/data"
          name: dtp-data
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
      livenessProbe:
        exec:
          command:
          - healthcheck.sh
          - --verify
          - datacollector
        initialDelaySeconds: 120
        periodSeconds: 60
        timeoutSeconds: 30
        failureThreshold: 5
# Uncomment section below if using DTP with Extension Designer
#    - name: extension-designer
#      securityContext:
#        allowPrivilegeEscalation: false
#        capabilities:
#          drop: ["ALL"]
#        seccompProfile:
#          type: RuntimeDefault
#      image: DTP_DOCKER_IMAGE
#      imagePullPolicy: Always
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects JAVA_CONFIG_ARGS
#      args: ["--run", "dtpservices"]
#      ports:
#        - containerPort: 8314
#      volumeMounts:
#        - mountPath: "/usr/local/parasoft/data"
#          name: dtp-data
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
#      livenessProbe:
#        exec:
#          command:
#          - healthcheck.sh
#          - --verify
#          - dtpservices
#        initialDelaySeconds: 120
#        periodSeconds: 60
#        timeoutSeconds: 30
#        failureThreshold: 5
# Uncomment section below if using Extension Designer with an external MongoDB
#      env:
#       - name: DEP_USE_REMOTE_DB
#         value: "true"
#       - name: DEP_DB_HOSTNAME
#         value: "mongodb-hostname" # Put your mongodb hostname here
#       - name: DEP_DB_PORT
#         value: "27017"
  restartPolicy: Always
  serviceAccountName: parasoft-account
  automountServiceAccountToken: true
  imagePullSecrets:
    - name: YOUR_SECRET
---
apiVersion: v1
kind: Service
metadata:
  name: dtp
  namespace: parasoft-dtp-namespace
spec:
  selector:
    app: DTP
  ports:
    - name: "http-server"
      protocol: TCP
      port: 8080
      targetPort: 8080
    - name: "data-collector"
      protocol: TCP
      port: 8082
      targetPort: 8082
    - name: "https-server"
      protocol: TCP
      port: 8443
      targetPort: 8443
# Uncomment section below if using DTP with Extension Designer
#    - name: "extension-designer"
#      protocol: TCP
#      port: 8314
#      targetPort: 8314
---
apiVersion: v1
kind: Service
metadata:
  name: dtp-external
  namespace: parasoft-dtp-namespace
spec:
  type: NodePort
  selector:
    app: DTP
  ports:
    - port: 8080
      name: HTTP_PORT_NAME
      nodePort: XXXXX
    - port: 8082
      name: DC_PORT_NAME
      nodePort: XXXXX
    - port: 8443
      name: HTTPS_PORT_NAME
      nodePort: XXXXX
# Uncomment section below if using DTP with Extension Designer
#    - port: 8314
#      name: EXTENSION_DESIGNER_PORT_NAME
#      nodePort: XXXXX
   
# SERVICE CONFIG NOTES:
# 'name' can be whatever you want
# 'nodePort' must be between 30000-32768
# 'spec.selector' must match 'metadata.labels' in pod config

Example yaml using 'kind: StatefulSet'

parasoft-dtp.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: dtp
  namespace: parasoft-dtp-namespace
  labels:
    app: DTP
spec:
  selector:
    matchLabels:
      app: DTP
  serviceName: dtp-service
  replicas: 1
  template:
    metadata:
      labels:
        app: DTP
    spec:
      volumes:
      - name: dtp-data
        nfs:
          server: NFS_SERVER_HOST
          path: /dtp/
#        persistentVolumeClaim:
#          claimName: dtp-pvc
# Uncomment section below if you are setting up a custom keystore; you will also need to uncomment out the associated volumeMounts below
#      - name: keystore-cfgmap-volume
#        configMap:
#          name: keystore-cfgmap
      securityContext:
        runAsNonRoot: true
      containers:
      - name: dtp-server
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
          seccompProfile:
            type: RuntimeDefault   
        image: DTP_DOCKER_IMAGE
        imagePullPolicy: Always
        env:
        - name: PARASOFT_POD_NAME			    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: PARASOFT_POD_NAMESPACE		    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
# To configure DTP to connect to your database on startup, please provide your database type, user, user password, and connection URL to the container environment by injecting the values as the DB_TYPE, DB_USER, DB_PASSWORD, and DB_URL environment variables.
# To prevent exposing sensitive data, please create a user password secret to use for the DB_PASSWORD environment variable.
# Note that the database type must be one of the following: mysql | oracle | postgresql
# Note that the environment variable values will override the equivalent persisted values in the PSTRootConfig.xml each time the container or pod is restarted.
# The following are example settings for a MySQL container called "mysql-container", a database called "DTP", a user called "dtp_user", and a user password secret.
#        - name: DB_TYPE
#          value: "mysql"
#        - name: DB_USER
#          value: "dtp_user"
#        - name: DB_PASSWORD
#          valueFrom:
#            secretKeyRef:
#              name: YOUR_DB_USER_PASSWORD_SECRET
#              key: YOUR_DB_USER_PASSWORD_SECRET_KEY
#        - name: DB_URL
#          value: "jdbc:mysql://mysql-container:3306/DTP"
# To configure DTP to automatically download the driver for your database on startup, please provide the JDBC driver URL to the container environment by injecting the value as the JDBC_DRIVER_URL environment variable.
# The following is an example URL to download the JDBC driver for MySQL 8.0.30.
#        - name: JDBC_DRIVER_URL
#          value: "https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.30/mysql-connector-java-8.0.30.jar"
# To configure DTP to use JVM arguments, please provide the arguments to the container environment by injecting the value as the JAVA_CONFIG_ARGS environment variable.
# The following is an example JVM argument "com.parasoft.sdm.storage.managers.admin.enable.delete.project.data=true"
#        - name: JAVA_CONFIG_ARGS
#          value: "-Dcom.parasoft.sdm.storage.managers.admin.enable.delete.project.data=true"
        args: ["--run", "dtp"]
        ports:
        - name: "http-server"
          containerPort: 8080
        - name: "https-server"
          containerPort: 8443
        volumeMounts:
        - mountPath: "/usr/local/parasoft/data"
          name: dtp-data
# Uncomment section below if you are setting up a custom keystore. Note that updates made to these files will not be reflected inside the container once it's been deployed; you will need to restart the container for it to contain any updates.
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/dtp/tomcat/conf/.keystore"
#          subPath: keystore
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/dtp/tomcat/conf/server.xml"
#          subPath: server-config
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
        livenessProbe:
          exec:
            command:
            - healthcheck.sh
            - --verify
            - dtp
          initialDelaySeconds: 120
          periodSeconds: 60
          timeoutSeconds: 30
          failureThreshold: 5
      - name: data-collector
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
          seccompProfile:
            type: RuntimeDefault   
        image: DTP_DOCKER_IMAGE
        imagePullPolicy: Always
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects JAVA_DC_CONFIG_ARGS
#        env:
#        - name: JAVA_DC_CONFIG_ARGS
#          value: "-Dcom.parasoft.sdm.dc.traffic.max.length=1000000"
        args: ["--run", "datacollector", "--no-copy-data"]
        ports:
        - containerPort: 8082
        volumeMounts:
        - mountPath: "/usr/local/parasoft/data"
          name: dtp-data
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
        livenessProbe:
          exec:
            command:
            - healthcheck.sh
            - --verify
            - datacollector
          initialDelaySeconds: 120
          periodSeconds: 60
          timeoutSeconds: 30
          failureThreshold: 5
# Uncomment section below if using DTP with Extension Designer
#      - name: extension-designer
#        securityContext:
#          allowPrivilegeEscalation: false
#          capabilities:
#            drop: ["ALL"]
#          seccompProfile:
#            type: RuntimeDefault
#        image: DTP_DOCKER_IMAGE
#        imagePullPolicy: Always
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects JAVA_CONFIG_ARGS
#        args: ["--run", "dtpservices"]
#        ports:
#        - containerPort: 8314
#        volumeMounts:
#        - mountPath: "/usr/local/parasoft/data"
#          name: dtp-data
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
#        livenessProbe:
#          exec:
#            command:
#            - healthcheck.sh
#            - --verify
#            - dtpservices
#          initialDelaySeconds: 120
#          periodSeconds: 60
#          timeoutSeconds: 30
#          failureThreshold: 5
# Uncomment section below if using Extension Designer with an external MongoDB
#        env:
#        - name: DEP_USE_REMOTE_DB
#          value: "true"
#        - name: DEP_DB_HOSTNAME
#          value: "mongodb-hostname" # Put your mongodb hostname here
#        - name: DEP_DB_PORT
#          value: "27017"
      restartPolicy: Always
      serviceAccountName: parasoft-account
      automountServiceAccountToken: true
      imagePullSecrets:
      - name: YOUR_SECRET
---
apiVersion: v1
kind: Service
metadata:
  name: dtp
  namespace: parasoft-dtp-namespace
spec:
  selector:
    app: DTP
  ports:
    - name: "http-server"
      protocol: TCP
      port: 8080
      targetPort: 8080
    - name: "data-collector"
      protocol: TCP
      port: 8082
      targetPort: 8082
    - name: "https-server"
      protocol: TCP
      port: 8443
      targetPort: 8443
# Uncomment section below if using DTP with Extension Designer
#    - name: "extension-designer"
#      protocol: TCP
#      port: 8314
#      targetPort: 8314
---
apiVersion: v1
kind: Service
metadata:
  name: dtp-external
  namespace: parasoft-dtp-namespace
spec:
  type: NodePort
  selector:
    app: DTP
  ports:
    - port: 8080
      name: HTTP_PORT_NAME
      nodePort: XXXXX
    - port: 8082
      name: DC_PORT_NAME
      nodePort: XXXXX
    - port: 8443
      name: HTTPS_PORT_NAME
      nodePort: XXXXX
# Uncomment section below if using DTP with Extension Designer
#    - port: 8314
#      name: EXTENSION_DESIGNER_PORT_NAME
#      nodePort: XXXXX
    
# SERVICE CONFIG NOTES:
# 'name' can be whatever you want
# 'nodePort' must be between 30000-32768
# 'spec.selector' must match 'metadata.labels' in pod config

Required Settings for a Stable Machine ID

As you modify either of the parasoft-dtp.yaml samples shown above or craft your own yaml, be aware that the following fields need to be consistent across upgrades and redeployments in order to assure a stable machine ID:

metadata: name
metadata: namespace
containers: name

In addition, the following environment variables are required:

env: name: PARASOFT_POD_NAME
env: name: PARASOFT_POD_NAMESPACE

Create the DTP Environment

DTP startup can take up to two minutes, assuming that minimum hardware requirements are met. During the startup time, DTP will not be accessible from the browser.

Prepare the volume mount location on your cluster. By default, the image runs as the "parasoft" user with a UID of 1000 and GID of 1000. Prepare the volume such that this user has read and write access to it.

Then create the DTP environment defined in the DTP setup yaml file created previously:

kubectl create -f parasoft-dtp.yaml

This will initialize the contents of the persistent volume.

If you injected any environment variables such as JVM arguments or database configuration settings, you can use the following command to verify their status:

kubectl exec <POD_NAME> -c <CONTAINER_NAME> -- printenv

If you are using DTP with Extension Designer, you will need to update the Reverse Proxy settings in Extension Designer to reflect the expected hostname and the exposed ports for accessing DTP and Extension Designer.

Configuring the Database JDBC URL

The structure of your database URL depends on how your external database is accessed. For example:

Database is hosted on a remote server. In this case (assuming a MySQL database), the URL might look like this: jdbc:mysql://<DB_HOST>:3306/DTP.
Database is running within your K8 cluster and accessible via configured service name or IP address. In this case, (assuming a MySQL database), the URL might look like this: jdbc:mysql://<CONFIGURED-SERVICE-NAME>.<NAMESPACE>:3306/DTP.

Custom Truststore

Using a custom truststore in Kubernetes environments is similar to using a custom keystore as described above. Adjust the directions for using a custom keystore as appropriate. Note that the truststore location is /usr/local/parasoft/dtp/jre/lib/security/cacerts.

Troubleshooting

Accessing Additional Logging

DTP captures debugging logs automatically. You can access them in the tests.log file found in the <DTP_DATA_DIR>/logs/ directory.

machineId is LINUX2-0

This issue can occur when there is an underlying permission issue. To resolve it, try the following options:

Search the tests.log file found in the <DTP_DATA_DIR>/logs/ directory for the error: "Kubernetes API call fails with status=403 error".
Verify that you have created permissions required by DTP using parasoft-permissions.yaml.
- Note: if you are upgrading, make sure to use the parasoft-permissions.yaml for the version to which you are upgrading.
Confirm that all Parasoft-required resources are using the same namespace.

machineId changes when pod restarts

This issue can occur if you are using an unsupported Kubernetes object. Make sure you are not using kind: Deployment. Only kind: Pod and kind: StatefulSet are supported.

Page tree

Deploying DTP in Kubernetes