Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions dev-docs/frozen/eks-bare-metal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Contrast on EKS

This is based on the setup presented in https://github.com/aws-samples/howto-runtime-attestation-on-aws.

## Resources

Replace the placeholder in eks-managed-sev-snp-metal-ubuntu-template.yaml with your SSH key name.

Create cluster and node group with

```
eksctl create cluster --without-nodegroup -f eks-cluster-template.yaml
eksctl create nodegroup -f eks-managed-sev-snp-metal-ubuntu-template.yaml
```

## Prevent autoscaling of the node

This is needed as the node needs to be rebooted.

```sh
aws autoscaling suspend-processes \
--auto-scaling-group-name eksctl-raas-nodegroup-selfmanaged-NodeGroup-c2opXigZr6N9 \
--scaling-processes ReplaceUnhealthy
```

## Setup the node

Get ssh access, then

```sh
sudo apt update
sudo apt upgrade -y
```

```bash
sudo apt install dracut -y
sudo tee -a /etc/dracut.conf.d/20-omit-ccp.conf <<EOF
omit_drivers+=" ccp "
EOF
sudo dracut --force
sudo tee -a /etc/modprobe.d/60-ccp.conf <<EOF
options ccp init_ex_path=/SEV_metadata
EOF
```

```bash
sudo sed 's/\(GRUB_CMD.*\)"/\1 mem_encrypt=on kvm_amd.sev=1 iommu=nopt"/' -i /etc/default/grub.d/50-cloudimg-settings.cfg
grep GRUB_CMD /etc/default/grub.d/50-cloudimg-settings.cfg # To validate
sudo update-grub
sudo grep sev /boot/grub/grub.cfg # Check
```

Reboot the node

```sh
sudo reboot
```
Comment on lines +53 to +57
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I tried but cannot remember the complication with it.


## Get the reference values

Access the node again via ssh, then

```sh
$ sh <(curl --proto '=https' --tlsv1.2 -L https://nixos.org/nix/install) --daemon
```

As root

```sh
nix-shell -p snphost
```

Reference values and check of host configuration via

```sh
modprobe msr
snphost ok
```

```sh
lscpu | grep "Model name\|CPU family"
```

CPU family = 25 → Milan
CPU family = 26 → Genoa

## Get the kubeconfig of the cluster

```sh
aws eks update-kubeconfig \
--region eu-central-1 \
--name raas \
--kubeconfig kubeconf
export KUBECONFIG=$(realpath kubeconf)
```

## Make `ebs` default storage class

```sh
kubectl patch storageclass gp2 \
-p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
```

## Configure the Contrast Manifest

- Remove the non-matching SNP processor family
- Update the TCB values with the output of `snphost ok`
- Set `TSMEEnabled` to `true`

## Update settings.json

Set the correct pause image

```json
"cluster_config": {
"pause_container_image": "public.ecr.aws/eks-distro/kubernetes/pause:3.5",
```

## Getting the coordinator service IP

Notice EKS isn't giving a LoadBalancer IP by default but a DNS name.
The instructions from the docs won't work to get it, describe the service and use the ingress field.
21 changes: 21 additions & 0 deletions dev-docs/frozen/eks-cluster-template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
version: "1.31"
region: eu-central-1
name: contrast-testing

iam:
withOIDC: true
serviceAccounts:
- metadata:
name: aws-load-balancer-controller
namespace: kube-system
wellKnownPolicies:
awsLoadBalancerController: true

addons:
- name: aws-ebs-csi-driver
wellKnownPolicies: # Adds an IAM service account
ebsCSIController: true
33 changes: 33 additions & 0 deletions dev-docs/frozen/eks-managed-sev-snp-metal-ubuntu-template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: contrast-testing
region: eu-central-1
version: "1.31"

iam:
withOIDC: true

nodeGroups:
- name: selfmanaged
instanceType: "m6a.metal"
overrideBootstrapCommand: |
#!/bin/bash
source /var/lib/cloud/scripts/eksctl/bootstrap.helper.sh
# Note "--node-labels=${NODE_LABELS}" needs the above helper sourced to work, otherwise will have to be defined manually.
/etc/eks/bootstrap.sh ${CLUSTER_NAME} --container-runtime containerd --kubelet-extra-args "--node-labels=${NODE_LABELS}"
desiredCapacity: 1
maxSize: 1
amiFamily: Ubuntu2404
volumeType: gp3
volumeSize: 300
iam:
attachPolicyARNs:
- arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
- arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
- arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
- arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
ssh:
allow: true
publicKeyName: ssh-key-name # replace with your actual key name