Skip to content

feat: Update CoCo pattern docs for v4 GA (Trustee 1.0, OSC 1.11) #645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
butler54 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: Update CoCo pattern docs for v4 GA (Trustee 1.0, OSC 1.11) #645

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 25 additions & 5 deletions .wordlist.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

aaaee
aab
aap
Expand All @@ -14,7 +15,6 @@ additionalimages
addon
addons
addr
addr
adoc
ae
aeg
Expand All @@ -39,9 +39,9 @@ anattama
anonymized
anonymizer
ansible
api's
apicast
apicurito
api's
apis
apiversion
appdev
Expand Down Expand Up @@ -107,7 +107,9 @@ byo
cacert
cakephp
canarypausestep
cas
cdd
cdh
cdn
centos
centric
Expand Down Expand Up @@ -155,11 +157,13 @@ cmzwn
cncf
cnv
cockroachdb
coco
codepath
coffeeshop
colocated
compliancetype
conf
confidentialcontainers
config
configmanagement
configmap
Expand All @@ -171,6 +175,7 @@ containerimage
controlplane
controlplaneendpoint
coreos
cosign
cp
crd
crds
Expand All @@ -192,6 +197,7 @@ ctrl
cuda
customerloyalty
customermocker
customise
customizable
customizations
cves
Expand Down Expand Up @@ -413,13 +419,15 @@ hybridcloudpatterns
hyperconverged
hyperscaler
hypershift
hyperthreading
iaa
iam
ib
ibmcloud
idempotence
idms
idp
ietf
iframe
ignoredifferences
iio
Expand Down Expand Up @@ -504,6 +512,9 @@ kam
kamelet
kasten
kastendr
kata
katacontainers
kbs
keycloak
keypair
keypairs
Expand Down Expand Up @@ -563,6 +574,7 @@ lsv
lvm
lvms
machineapi
machineconfig
machineconfigpool
machineconfigs
machineset
Expand Down Expand Up @@ -618,6 +630,7 @@ mq
mqtt
multicloud
multicluster
multiclusterhub
multisource
multisourceconfig
musthave
Expand Down Expand Up @@ -686,8 +699,8 @@ opendatahub
openid
openjdk
openshift
openshiftpullsecret
openshift's
openshiftpullsecret
openshiftsdn
openshiftversion
openssl
Expand All @@ -700,6 +713,7 @@ operatorgroups
operatorhub
operatorsource
opr
osc
osspa
osx
ouput
Expand All @@ -722,6 +736,7 @@ patternsh
patternsoperator
pbivukilnpoe
pci
pcr
pem
performant
persistentvolumeclaim
Expand Down Expand Up @@ -787,6 +802,7 @@ querier
quickassist
quickstart
rabbitmq
raci
rbac
rbklxs
rdma
Expand All @@ -803,8 +819,8 @@ renderers
replicaset
replicasets
repo
repolist
repo's
repolist
repos
repourl
reranked
Expand Down Expand Up @@ -847,6 +863,7 @@ runtimes
rxpm
saas
saml
sandboxed
sas
sbom
scada
Expand Down Expand Up @@ -880,6 +897,7 @@ signin
sigstore
siteadmin
skipdryrunonmissingresource
skopeo
sla
slas
sme
Expand Down Expand Up @@ -939,6 +957,7 @@ targetport
tbd
tcp
techpreview
tee
tei
tekron
tekton
Expand Down Expand Up @@ -984,6 +1003,7 @@ tradeoff
tradeoffs
transactional
travelops
trustee
trvlops
tsa
tst
Expand All @@ -1002,8 +1022,8 @@ unsealvault
untrusted
updatingconfig
updatingversion
upstreaming
upstream's
upstreaming
ure
uri
usecsv
Expand Down
41 changes: 26 additions & 15 deletions content/patterns/coco-pattern/_index.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,48 +25,59 @@ include::modules/comm-attributes.adoc[]

= About the Confidential Containers pattern

Confidential computing is a technology for securing data in use. It uses a https://en.wikipedia.org/wiki/Trusted_execution_environment[Trusted Execution Environment] provided within the hardware of the processor to prevent access from others who have access to the system.
https://confidentialcontainers.org/[Confidential containers] is a project to standardize the consumption of confidential computing by making the security boundary for confidential computing to be a Kubernetes pod. https://katacontainers.io/[Kata containers] is used to establish the boundary via a shim VM.
Confidential computing is a technology for securing data in use. It uses a https://en.wikipedia.org/wiki/Trusted_execution_environment[Trusted Execution Environment] (TEE) provided within the hardware of the processor to prevent access from others who have access to the system, including cluster administrators and hypervisor operators.
https://confidentialcontainers.org/[Confidential containers] is a project to standardize the consumption of confidential computing by making the security boundary for confidential computing a Kubernetes pod. https://katacontainers.io/[Kata containers] is used to establish the boundary via a shim VM.

A core goal of confidential computing is to use this technology to isolate the workload from both Kubernetes and hypervisor administrators.
A core goal of confidential computing is to use this technology to isolate the workload from both Kubernetes and hypervisor administrators. In practice this means that even a `kubeadmin` user cannot `exec` into a running confidential container or inspect its memory.

image::coco-pattern/isolation.png[Schematic describing the isolation of confidential contains from the hosting system]
image::coco-pattern/isolation.png[Schematic describing the isolation of confidential containers from the hosting system]


This pattern uses https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/deploying-on-azure#deploying-cc_azure-cc[Red Hat OpenShift sandbox containers] to deploy and configure confidential containers on Microsoft Azure.
This pattern uses https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.11/html/deploying_confidential_containers/cc-overview[Red Hat OpenShift sandbox containers] to deploy and configure confidential containers on Microsoft Azure. On Azure, confidential containers run as "peer pods" — VMs from the `Standard_DCas_v5` family provisioned directly on the Azure hypervisor rather than nested inside OpenShift worker nodes.

It deploys three copies of 'Hello OpenShift' to demonstrate some of the security boundaries that enforced with confidential containers.
It deploys three copies of 'Hello OpenShift' to demonstrate some of the security boundaries enforced with confidential containers, along with a `kbs-access` application that verifies end-to-end secret delivery from the Key Broker Service.

== Requirements

- An an azure account with the link:./coco-pattern-azure-requirements/[required access rights]
- An OpenShift cluster, within the Azure environment updated beyond 4.16.10
- An Azure account with the link:./coco-pattern-azure-requirements/[required access rights], including quota for `Standard_DCas_v5` confidential VMs
- An OpenShift 4.17+ cluster within the Azure environment
- Tools: `podman`, `yq`, `jq`, `skopeo`
- An OpenShift pull secret at `~/pull-secret.json`


== Security considerations

**This pattern is a demonstration only and contains configuration that is not best practice**

- The default configuration deploys everything in a single cluster for testing purposes. The https://www.ietf.org/archive/id/draft-ietf-rats-architecture-22.html[RATS] architecture mandates that the Key Broker Service (e.g. https://github.com/confidential-containers/trustee[Trustee]) is in a trusted security zone.
- The https://github.com/confidential-containers/trustee/tree/main/attestation-service[Attestation Service] has wide open security policies.
- The pattern supports both single-cluster (`simple` clusterGroup) and multi-cluster (`trusted-hub` + `spoke`) topologies. The default is single-cluster, which breaks the RACI separation expected in a remote attestation architecture. In the single-cluster topology, the Key Broker Service and the workloads it protects run on the same cluster, meaning a compromised cluster could affect both. The multi-cluster topology addresses this by separating the trusted zone (Trustee, Vault, ACM on the hub) from the untrusted workload zone (spoke). The https://www.ietf.org/archive/id/draft-ietf-rats-architecture-22.html[RATS] architecture mandates that the Key Broker Service (e.g. https://github.com/confidential-containers/trustee[Trustee]) is in a trusted security zone.
- The https://github.com/confidential-containers/trustee/tree/main/attestation-service[Attestation Service] has wide open security policies. The default `insecure` policy accepts all images without signature verification. For production use, configure the `signed` policy in `values-secret.yaml.template` and provide cosign public keys.

== Future work

* Deploying the environment the 'Trusted' environment including the KBS on a separate cluster to the secured workloads
* Deploying to alternative environments supporting confidential computing including bare metal x86 clusters; IBM Cloud; IBM Z
* Finishing the sample AI application
* Finishing the sample AI application for confidential inference with protected GPUs
* Supporting air-gapped deployments

== Architecture

Confidential Containers typically has two environments. A trusted zone, and an untrusted zone. In these zones, Trustee, and the sandbox container operator are deployed, respectively.
Confidential Containers typically has two environments. A trusted zone, and an untrusted zone. In the trusted zone, the Key Broker Service (Trustee), attestation service, and secrets management (Vault) are deployed. In the untrusted zone, the sandboxed containers operator provisions confidential VMs and runs workloads.

** For demonstration purposes the pattern currently is converged on one cluster**
The pattern supports both single-cluster and multi-cluster topologies. In the single-cluster topology (`simple` clusterGroup), all components are converged on one cluster. In the multi-cluster topology, the `trusted-hub` clusterGroup runs on the hub cluster and the `spoke` clusterGroup runs on one or more managed clusters imported via ACM. See link:./coco-pattern-getting-started/[Getting started] for deployment options.

image::coco-pattern/overview-schematic.png[Schematic describing the high level architecture of confidential containers]

=== Key components

- **Red Hat Build of Trustee 1.0**: The Key Broker Service (KBS) and attestation service. Trustee verifies that workloads are running in a genuine TEE before releasing secrets. Certificates for Trustee are managed by cert-manager using self-signed CAs.
- **HashiCorp Vault**: Secrets backend for the Validated Patterns framework. Stores KBS keys, attestation policies, and PCR measurements.
- **OpenShift Sandboxed Containers 1.11**: Deploys and manages peer-pod VMs for confidential workloads. Operator subscriptions are pinned to specific CSV versions with manual install plan approval to ensure version consistency.
- **Red Hat Advanced Cluster Management (ACM)**: Manages the spoke cluster in multi-cluster deployments. Policies and applications are deployed to the spoke via ACM's application lifecycle management.


== References
- https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/about-osc#about-confidential-containers_about-osc[OpenShift sandboxed containers documentation]
- https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.11[OpenShift Sandboxed Containers 1.11 documentation]
- https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.11/html/deploying_confidential_containers/cc-overview[Deploying confidential containers on OpenShift]
- https://docs.redhat.com/en/documentation/red_hat_build_of_trustee/1.0[Red Hat Build of Trustee 1.0 documentation]
- https://www.redhat.com/en/blog/red-hat-openshift-sandboxed-containers-111-and-red-hat-build-trustee-10-accelerate-confidential-computing-across-hybrid-cloud[OSC 1.11 and Trustee 1.0 announcement blog]
- https://www.redhat.com/en/blog/exploring-openshift-confidential-containers-solution[OpenShift confidential containers solution blog]
- https://www.redhat.com/en/blog/introducing-confidential-containers-trustee-attestation-services-solution-overview-and-use-cases[Trustee attestation services overview]
28 changes: 15 additions & 13 deletions content/patterns/coco-pattern/coco-pattern-azure-requirements.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,15 @@ include::modules/comm-attributes.adoc[]
:imagesdir: ../../../images
= Azure requirements

This demo currently has been tested only on azure.
The configuration tested used the `openshift-install`.
https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-default.html[OpenShift documentation] contains details on how to do this.
This pattern has been tested on Microsoft Azure using self-managed OpenShift clusters provisioned with `openshift-install`.
https://docs.openshift.com/container-platform/4.17/installing/installing_azure/installing-azure-default.html[OpenShift documentation] contains details on how to install a cluster on Azure.

The documentation outlines https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-account.html[minimum required configuration] for an azure account.
The documentation outlines the https://docs.openshift.com/container-platform/4.17/installing/installing_azure/installing-azure-account.html[minimum required configuration] for an Azure account.

== Changes required

Do not accept default sizes for OpenShift install. It is recommended to up the workers to at least `Standard_D8s_v5`.
This can be done by using `openshift-install create install-config` first and adjusting the workers under platform e.g.:
Do not accept default sizes for OpenShift install. It is recommended to increase the worker node size to at least `Standard_D8s_v5`.
This can be done by using `openshift-install create install-config` first and adjusting the workers under platform, for example:

[source,yaml]
----
Expand All @@ -35,29 +34,31 @@ This can be done by using `openshift-install create install-config` first and ad

On a cloud provider the virtual machines for the kata containers use "peer pods" which are running directly on the cloud provider's hypervisor (see the diagram below).
This means that access is required to the "confidential computing" virtual machine class. On Azure the `Standard_DCas_v5` class of virtual machines are used.
These virtual machines are *NOT* available in all regions. Users will also need to up the specific limits for `Standard_DC2as_v5` virtual machines.
These virtual machines are *NOT* available in all regions. Check https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/[Azure products by region] to confirm availability in your target region.

Users will also need to request quota increases for the `Standard_DC2as_v5` (and optionally `Standard_DC4as_v5`, `Standard_DC8as_v5`, `Standard_DC16as_v5`) virtual machine families in their target region. By default, Azure subscriptions may have zero quota for confidential computing VM sizes.

image::coco-pattern/peer_pods.png[Schematic diagram of peer pods vs standard kata containers]

DNS for the openshift cluster also *MUST* be provided by azure DNS.
DNS for the OpenShift cluster *MUST* be provided by Azure DNS. The pattern uses Azure DNS for both the cluster's ingress and for cert-manager DNS01 challenge validation when issuing certificates.

== Azure configuration required for the validated pattern

The validated pattern requires access to azure apis to provision peer-pod VMs and to obtain certificates from let's encrypt.
The validated pattern requires access to Azure APIs to provision peer-pod VMs.

Azure configuration information must be provided in two places:

- The a secret must be loaded using a ../../../learn/secrets-management-in-the-validated-patterns-framework/[values-secret] file.
The https://github.com/validatedpatterns/coco-pattern/blob/main/values-secret.yaml.template[`values-secret.yaml.template`] file provides the appropriate structure
- A secret must be loaded using a ../../../learn/secrets-management-in-the-validated-patterns-framework/[values-secret] file.
The https://github.com/validatedpatterns/coco-pattern/blob/main/values-secret.yaml.template[`values-secret.yaml.template`] file provides the appropriate structure. The Azure client secret (service principal password) is stored here and loaded into Vault.

- A broader set of information about the cluster is required in https://github.com/validatedpatterns/coco-pattern/blob/main/values-global.yaml[`values-global.yaml`] (see below).
- A broader set of information about the cluster is required in https://github.com/validatedpatterns/coco-pattern/blob/main/values-global.yaml[`values-global.yaml`] (see below). These values are used by the sandboxed containers operator to provision peer-pod VMs in the correct Azure subscription, resource group, and virtual network.


[source,yaml]
----
global:
azure:
clientID: '' # Service principle ID
clientID: '' # Service principal ID
subscriptionID: ''
tenantID: '' # Tenant ID
DNSResGroup: '' # Resource group for the azure DNS hosted zone
Expand All @@ -68,3 +69,4 @@ global:
clusterRegion: ''
----

The `clusterResGroup`, `clusterSubnet`, and `clusterNSG` values can be found in the Azure portal after the cluster has been provisioned, or via `openshift-install` metadata. The `DNSResGroup` and `hostedZoneName` correspond to the Azure DNS zone used for the cluster's base domain.
Loading