Deploying License Server in Kubernetes

In this section:

Docker Image

Parasoft has published official Docker images to Docker Hub for your convenience. 

• https://hub.docker.com/repository/docker/parasoft/lss

Deploying License Server in Kubernetes with a Helm Chart

Parasoft has published an official Helm chart to Docker Hub for your convenience. Full installation instructions are included in the README there. See https://hub.docker.com/r/parasoft/lss-helm.

Deploying License Server in Kubernetes Manually

Prerequisites

Namespace for License Server

First, create a namespace for License Server to run in. For example:

kubectl create namespace parasoft-lss-namespace

Note: The namespace name "parasoft-lss-namespace" is used throughout this documentation in command and resource examples. If you use a different name for your namespace, be sure to change any instances of "parasoft-lss-namespace" in those examples to your namespace name.

Service Account for License Server

Next, you will need a Kubernetes cluster. After starting the cluster, create the service account and permissions required by the License Server pod and related resources.

# Stable access for clients to license server
apiVersion: v1
kind: ServiceAccount
metadata:
  name: parasoft-account
  namespace: parasoft-lss-namespace
automountServiceAccountToken: true
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: parasoft-read
  namespace: parasoft-lss-namespace
rules:
- apiGroups:
  - ""
  resources:
  - "namespaces"
  - "pods"
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: parasoft-read-bind
  namespace: parasoft-lss-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: parasoft-read
subjects:
- kind: ServiceAccount
  name: parasoft-account
  namespace: parasoft-lss-namespace

Use your yaml file to create the required service account and permissions before creating the License Server environment:

kubectl create -f parasoft-permissions.yaml

You should see something similar to the output below in your console:

serviceaccount/parasoft-account created
role.rbac.authorization.k8s.io/parasoft-read created
rolebinding.rbac.authorization.k8s.io/parasoft-read-bind created

License Server Setup

To set up License Server, create a yaml file that defines the following:

• Secret (optional)
  • To pull the License Server image from the repository.
• Volume
  • To persist data.
• Pod or StatefulSet
  • Creates a pod set up to run a License Server container configured with a volume to persist data and a liveness probe for the container health.
• Service (optional).
  • Makes License Server accessible via external clients by allocating ports in the node and mapping them to ports in the pod.

Example yaml files for a Pod or StatefulSet (both called "parasoft-lss.yaml") are shown below. These examples use an NFS volume, but that is not required; use the volume type that fits your needs best.

Required Settings for a Stable Machine ID

As you modify either of the parasoft-lss.yaml samples shown below or craft your own yaml, be aware that the following fields need to be consistent across upgrades and redeployments in order to assure a stable machine ID:

• metadata: name
• metadata: namespace
• containers: name

In addition, the following environment variables are required:

• env: name: PARASOFT_POD_NAME
• env: name: PARASOFT_POD_NAMESPACE

Example yaml using 'kind: Pod'

apiVersion: v1
kind: Pod
metadata:
  name: lss
  namespace: parasoft-lss-namespace
  labels:
    app: LSS
spec:
  volumes:
    - name: lss-data
      nfs:
        server: NFS_SERVER_HOST
        path: /lss/
# Uncomment section below if you are setting up a custom keystore; you will also need to uncomment out the associated volumeMounts below
#    - name: keystore-cfgmap-volume
#      configMap:
#        name: keystore-cfgmap
  securityContext:
    runAsNonRoot: true
  containers:
    - name: lss
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
        seccompProfile:
          type: RuntimeDefault    
      image: LSS_DOCKER_IMAGE
      imagePullPolicy: Always
      env:
        - name: PARASOFT_POD_NAME			    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: PARASOFT_POD_NAMESPACE		    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects LSS_JAVA_OPTS
#        - name: LSS_JAVA_OPTS
#          value: "-Dparasoft.use.license.v2=true"
      ports:
         - name: "lss-http-port"
           containerPort: 8080
         - name: "lss-https-port"
           containerPort: 8443 
      volumeMounts:
        - mountPath: "/usr/local/parasoft/license-server/data"
          name: lss-data
# Uncomment section below if you are setting up a custom keystore. Note that updates made to these files will not be reflected inside the container once it's been deployed; you will need to restart the container for it to contain any updates.
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/license-server/app/tomcat/conf/.keystore"
#          subPath: keystore
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/license-server/app/tomcat/conf/server.xml"
#          subPath: server-config
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
      livenessProbe:
        exec:
          command:
          - healthcheck.sh
        initialDelaySeconds: 120
        periodSeconds: 60
        timeoutSeconds: 30
        failureThreshold: 5
  restartPolicy: Always
  serviceAccountName: parasoft-account
  automountServiceAccountToken: true
  imagePullSecrets:
    - name: YOUR_SECRET

Example yaml using 'kind: StatefulSet'

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: lss
  namespace: parasoft-lss-namespace
  labels:
    app: LSS
spec:
  selector:
    matchLabels:
      app: LSS
  serviceName: lss-service
  replicas: 1			#Only 1 is supported
  template:
    metadata:
      labels:
        app: LSS
    spec:
      volumes:
      - name: lss-data
        nfs:
          server: NFS_SERVER_HOST
          path: /lss/
#        persistentVolumeClaim:
#          claimName: lss-pvc
# Uncomment section below if you are setting up a custom keystore; you will also need to uncomment out the associated volumeMounts below
#      - name: keystore-cfgmap-volume
#        configMap:
#          name: keystore-cfgmap
      securityContext:
        runAsNonRoot: true
      containers:
      - name: lss
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ "ALL" ]
          seccompProfile:
            type: RuntimeDefault
        image: LSS_DOCKER_IMAGE
        imagePullPolicy: Always
        env:
        - name: PARASOFT_POD_NAME			    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: PARASOFT_POD_NAMESPACE		    #REQUIRED, DO NOT CHANGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
# To inject JVM arguments into the container, specify the "env" property as in the example below, which injects LSS_JAVA_OPTS
#        - name: LSS_JAVA_OPTS
#          value: "-Dparasoft.use.license.v2=true"
        ports:
           - name: "lss-http-port"
             containerPort: 8080
           - name: "lss-https-port"
             containerPort: 8443 
        volumeMounts:
        - name: lss-data
          mountPath: "/usr/local/parasoft/license-server/data"
# Uncomment section below if you are setting up a custom keystore. Note that updates made to these files will not be reflected inside the container once it's been deployed; you will need to restart the container for it to contain any updates.
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/license-server/app/tomcat/conf/.keystore"
#          subPath: keystore
#        - name: keystore-cfgmap-volume
#          mountPath: "/usr/local/parasoft/license-server/app/tomcat/conf/server.xml"
#          subPath: server-config
# To prevent liveness probe failures on environments with low or overly taxed RAM/CPU, we recommend increasing the timeout seconds
        livenessProbe:
          exec:
            command:
            - healthcheck.sh
          initialDelaySeconds: 120
          periodSeconds: 60
          timeoutSeconds: 30
          failureThreshold: 5
      restartPolicy: Always
      serviceAccountName: parasoft-account
      automountServiceAccountToken: true
      imagePullSecrets:
        - name: YOUR_SECRET

Service Definition

Create the service that can be used to access the License Server in Kubernetes. The example shown below exposes it using a node port, which provides a stable endpoint for applications to access it.

apiVersion: v1
kind: Service
metadata:
  name: lss-svc
  namespace: parasoft-lss-namespace
spec:
  type: NodePort
  selector:
    app: DTP
  ports:
    - name: "lss-http-port"
      protocol: TCP
      port: 8080
      nodePort: 30080
    - name: "lss-https-port"
      protocol: TCP
      port: 8443
      nodePort: 30443
# SERVICE CONFIG NOTES:
# 'name' can be whatever you want, but must follow the DNS label naming conventions
# 'nodePort' must be between 30000-32768
# 'spec.selector' must match 'metadata.labels' in pod config 

Create the License Server Environment

Use the yaml file to create the License Server environment:

kubectl create -f parasoft-lss.yaml

To access the UI on a web browser, use the node ports allocated in the service definition as the address (for example, NODE_HOST:NODE_PORT).

If you injected JVM arguments into the container and want to verify their status, run the following command:

kubectl exec <POD_NAME> -c <CONTAINER_NAME> -- printenv

Custom Keystore

If you want to set up a custom keystore, you will need to create a configuration map for the .keystore and server.xml files. The command below creates a configuration map called "keystore-cfgmap" with file mappings for the custom .keystore and server.xml files. In this example, each file mapping is given a key: "keystore" for the .keystore file and "server-config" for the server.xml file. While giving each file mapping a key is not necessary, it is useful when you don't want the key to be the file name. 

~$ kubectl create configmap keystore-cfgmap --from-file=keystore=/path/to/.keystore --from-file=server-config=/path/to/server.xml
configmap/keystore-cfgmap created

Custom Truststore

Using a custom truststore in Kubernetes environments is similar to using a custom keystore as described above. Adjust the directions for using a custom keystore as appropriate. Note that the truststore location is /usr/local/parasoft/license-server/app/jre/lib/security/cacerts.

Volume Mount Security Policies (Optional)

If your security policy requires applications to only write to mounted volumes, then in addition to the lss-data directory (which is already set in the example pod) you will need to mount the following locations:

/usr/local/parasoft/license-server/app/tomcat/logs
/usr/local/parasoft/license-server/app/tomcat/temp
/usr/local/parasoft/license-server/app/tomcat/work

Note: Do not mount /usr/local/parasoft/data-tmp.  You can ignore errors related to data-tmp like "rm: cannot remove '/usr/local/parasoft/data-tmp/LicenseServer/conf/licserv.enc': Read-only file system" in the Kubernetes logs.

Working with OIDC Secret (Optional)

You can configure an OIDC secret for License Server. To do so:

1. Create a secret for your OIDC configuration.
   Example OIDC secret yaml
   
   lss-oidc-secret.yaml

   apiVersion: v1
   kind: Secret
   metadata:
     name: lss-oidc-secret
     namespace: parasoft-lss-namespace
   type: Opaque
   stringData:
     oidc_json: |
       {
           "enabled": true,
           "issuerUri": "your issuer uri",
           "clientId": "your client id",
           "clientSecret": "your client secret",
           "scopes": ["openid", "profile", "email"],
           "claimMappings": {
               "username": "preferred_username",
               "firstName": "given_name",
               "lastName": "family_name",
               "email": "email"
           },
           "adminUsers": ["your admin user"]
       }

2. Update your lss-pod.yaml as follows:
   • Add an OIDC secret volume under spec.volumes:
     

     # Adding the OIDC secret as a volume
         - name: lss-oidc-secret-volume
           secret:
             secretName: lss-oidc-secret
             optional: true
             items:
             - key: oidc_json
               path: oidc.json

   • Add an initContainers section that will populate the data directory before the other containers are started under spec and before containers. This should use the License Server image and the License Server data directory /usr/local/parasoft/license-server/data:
     

     # Populating the data directory before main container starts
       initContainers:
         - name: init-lss-data
           image: LSS_DOCKER_IMAGE
           command: ["/bin/sh", "-c"]
           args:
             - |
               set -e
               if [ ! -f "$LSS_DATADIR/.lssrc" ]; then
                 touch "$LSS_DATADIR/.lssrc"
                 cp --preserve=timestamps -rnT "$TMP_DATADIR" "$LSS_DATADIR"
               fi
           volumeMounts:
             - mountPath: "/usr/local/parasoft/license-server/data"
               name: lss-data

   • Add an OIDC secret volume under spec.containers.name (=lss).volumeMounts:
     

     # Mounting the OIDC secret volume to the container
             - name: lss-oidc-secret-volume
               mountPath: "/usr/local/parasoft/license-server/data/conf/oidc.json"
               subPath: oidc.json
               readOnly: true

Troubleshooting

Enabling Additional Logging

1. Copy log4j.xml from the <INSTALL_DIR>/app/ directory to <INSTALL_DIR>/data/.

2. Open the log4j.xml file in <INSTALL_DIR>/data/ and add the following logger in Loggers element:

   <Logger name="com.parasoft.xtest" level="ALL">
     <AppenderRef ref="CONSOLE" />
   </Logger>

3. Find commented-out section for LSS_JAVA_OPTS in the yaml file, uncomment it, then add the following as the value for LSS_JAVA_OPTS:

   -Dparasoft.cloudvm.verbose=true -Dparasoft.logging.config.file=/usr/local/parasoft/license-server/data/log4j.xml

4. Restart the application.

5. Additional logging will go to catalina log file (stdout). You can run this command to get the log file to local file system (replace "lss-pod1-nfs" with your pod name and "parasoft-lss-namespace" with the namespace you used):

   kubectl logs lss-pod1-nfs -n parasoft-lss-namespace > lss-debug.log

machineId is LINUX2-0

This issue can occur when there is an underlying permission issue. To resolve it, try the following options:

1. Search the ls.log file found in the <LS_DATA_DIR>/logs/ directory for a "fails to get UUID" message.
2. Verify that you have created permissions required by License Server using parasoft-permissions.yaml.
   • Note: if you are upgrading, make sure to use the parasoft-permissions.yaml for the version to which you are upgrading.
3. Confirm that all Parasoft-required resources are using the same namespace.

machineId changes when pod restarts

1. This issue can occur if you are using an unsupported Kubernetes object. Make sure you are not using kind: Deployment. Only kind: Pod and kind: StatefulSet are supported.
2. Make sure you are following all the directions under Required Settings for a Stable Machine ID.
3. Make sure that your environment is not setting automountServiceAccountToken to false.