Importing untrusted TLS certificates to Che

Che components communications with external services are encrypted with TLS. They require TLS certificates signed by trusted Certificate Authorities (CA). Therefore, you must import into Che all untrusted CA chains in use by an external service such as:

  • A proxy

  • An identity provider (OIDC)

  • A source code repositories provider (Git)

Che uses labeled config maps in Che namespace as sources for TLS certificates. The config maps can have an arbitrary amount of keys with a random amount of certificates each.

When an OpenShift cluster contains cluster-wide trusted CA certificates added through the cluster-wide-proxy configuration, Che Operator detects them and automatically injects them into a config map with the config.openshift.io/inject-trusted-cabundle="true" label. Based on this annotation, OpenShift automatically injects the cluster-wide trusted CA certificates inside the ca-bundle.crt key of the config map.
Prerequisites

  • An active kubectl session with administrative permissions to the destination Kubernetes cluster. See Overview of kubectl.

  • The eclipse-che namespace exists.

  • For each CA chain to import: the root CA and intermediate certificates, in PEM format, in a ca-cert-for-che-<count>.pem file.

Procedure

  1. Concatenate all CA chains PEM files to import, into the custom-ca-certificates.pem file, and remove the return character that is incompatible with the Java truststore.

    $ cat ca-cert-for-che-*.pem | tr -d '\r' > custom-ca-certificates.pem

  2. Create the custom-ca-certificates config map with the required TLS certificates:

    $ kubectl create configmap custom-ca-certificates \
    --from-file=custom-ca-certificates.pem \
    --namespace=eclipse-che

  3. Label the custom-ca-certificates config map:

    $ kubectl label configmap custom-ca-certificates \
    app.kubernetes.io/component=ca-bundle \
    app.kubernetes.io/part-of=che.eclipse.org \
    --namespace=eclipse-che

  4. Deploy Che if it hasn’t been deployed before. Otherwise, wait until the rollout of Che components finishes.

  5. Restart running workspaces for the changes to take effect.

Verification steps

  1. Verify that the config map contains your custom CA certificates. This command returns your custom CA certificates in PEM format:

    $ kubectl get configmap \
    --namespace=eclipse-che \
    --output='jsonpath={.items[0:].data.custom-ca-certificates\.pem}' \
    --selector=app.kubernetes.io/component=ca-bundle,app.kubernetes.io/part-of=che.eclipse.org

  2. Verify Che pod contains a volume mounting the ca-certs-merged config map:

    $ kubectl get pod \
    --selector=app.kubernetes.io/component=che \
    --output='jsonpath={.items[0].spec.volumes[0:].configMap.name}' \
    --namespace=eclipse-che \
    | grep ca-certs-merged

  3. Verify the Che server container has your custom CA certificates. This command returns your custom CA certificates in PEM format:

    $ kubectl exec -t deploy/che \
    --namespace=eclipse-che \
    -- cat /public-certs/custom-ca-certificates.pem

  4. Verify in the Che server logs that the imported certificates count is not null:

    $ kubectl logs deploy/che --namespace=eclipse-che \
    | grep custom-ca-certificates.pem

  5. List the SHA256 fingerprints of your certificates:

    $ for certificate in ca-cert*.pem ;
  do openssl x509 -in $certificate -digest -sha256 -fingerprint -noout | cut -d= -f2;
  done

  6. Verify that Che server Java truststore contains certificates with the same fingerprint:

    $ kubectl exec -t deploy/che --namespace=eclipse-che -- \
    keytool -list -keystore /home/user/cacerts \
    | grep --after-context=1 custom-ca-certificates.pem

  7. Start a workspace, get the namespace name in which it has been created: <workspace_namespace>, and wait for the workspace to be started.

  8. Verify that the che-trusted-ca-certs config map contains your custom CA certificates. This command returns your custom CA certificates in PEM format:

    $ kubectl get configmap che-trusted-ca-certs \
    --namespace=<workspace_namespace> \
    --output='jsonpath={.data.custom-ca-certificates\.custom-ca-certificates\.pem}'

  9. Verify that the workspace pod mounts the che-trusted-ca-certs config map:

    $ kubectl get pod \
    --namespace=<workspace_namespace> \
    --selector='controller.devfile.io/devworkspace_name=<workspace_name>' \
    --output='jsonpath={.items[0:].spec.volumes[0:].configMap.name}' \
    | grep che-trusted-ca-certs

  10. Verify that the universal-developer-image container (or the container defined in the workspace devfile) mounts the che-trusted-ca-certs volume:

    $ kubectl get pod \
    --namespace=<workspace_namespace> \
    --selector='controller.devfile.io/devworkspace_name=<workspace_name>' \
    --output='jsonpath={.items[0:].spec.containers[0:]}' \
    | jq 'select (.volumeMounts[].name == "che-trusted-ca-certs") | .name'

  11. Get the workspace pod name <workspace_pod_name>:

    $ kubectl get pod \
    --namespace=<workspace_namespace> \
    --selector='controller.devfile.io/devworkspace_name=<workspace_name>' \
    --output='jsonpath={.items[0:].metadata.name}' \

  12. Verify that the workspace container has your custom CA certificates. This command returns your custom CA certificates in PEM format:

    $ kubectl exec <workspace_pod_name> \
    --namespace=<workspace_namespace> \
    -- cat /public-certs/custom-ca-certificates.custom-ca-certificates.pem
Additional resources