Kubernetes

Kubernetes Methodology

TODO:

• https://securitylabs.datadoghq.com/articles/kubernetes-security-fundamentals-part-3/
• https://securitycafe.ro/2023/02/27/a-complete-kubernetes-config-review-methodology/
• https://tldrsec.com/p/kubernetes-security-threat-informed-defense
• Kubernetes Threat Matrix

Basic Defense Idea

Attackers will come through three potential entry points:

• end user traffic: application traffic
• management traffic: traffic to the API servers of our cluster
• container image registry traffic: containers with malware

To protect against this, we need to scan our containers and minimizer the identities that have authorization to talk with the management API endpoints.

You can block the metadata service from workloads:

On Amazon EKS and Azure AKS, block access to the IMDS with a NetworkPolicy On EKS, you should also enforce IMDSv2 with response hop limit to 1 on all worker nodes. Note that _enforcing IMDSv2 alone is not sufficient_. You also need to set the "response hop limit" to 1.

Limit allowed pods:

This can typically be achieved through the use of an admission controller, such as Kyverno or OPA Gatekeeper. Since Kubernetes 1.25, the built-in Pod Security Admission feature also allows you to control workload admission, including using custom rules since 1.28. All of these support an audit mode that will only generate warning events, and an enforcement mode that will reject non-compliant workloads.

Image and Container inspection

If, during the assessment, you get access to container images, you can analyze them for leaked credentials, misconfigurations, known vulnerabilities, etc.

Tools that automate (parts) of this:

• https://github.com/aquasecurity/trivy
• https://github.com/anchore/grype

If you have access to a running pod, there are additional tools that automate escaping from an owned container:

• https://github.com/inguardians/peirates
• https://github.com/quarkslab/kdigger

Configuration Review

Typically you will check your cluster against a well-known benchmark such as the CIS Kubernetes Benchmark.

A simple way of doing this is KubeBench. Also make sure that you check for common files that might contain sensitive data (check https://github.com/random-robbie/bruteforce-lists/blob/master/k8s.txt for a list of potential files).

Permissions/RBAC (Privilege Escalation vectors)

Mostly this is searching for overly permissive rights to users and service accounts. Kubernetes roles are based on Roles (namespace-level), ClusterRoles (for all namespaces in the cluster). Applying those roles happens through bindings, there are RoleBindings and ClusterRoleBindings. We have a typical <user/service> can <verb> on <resource> framework.

Potential privilege escalation opportunities are:

• creating new pods (https://bishopfox.com/blog/kubernetes-pod-privilege-escalation)
• list/get/watch secrets, e.g. kubectl get secret --token $TOKEN -o json | jq -r '.items[] | select(.metadata.name=="secret-name")'
• any resource or verb wildcards (*)
• Create/Update/Delete Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs
• Get/Patch/Create Rolebindings
• Get/Create Node/Proxy
  • curl -k -H “Authorization: Bearer $TOKEN” -XPOST https://kube-apiserver:10250/run/{namespace}/{pod}/{container} -d “cmd=whoami”
  • https://blog.aquasec.com/privilege-escalation-kubernetes-rbac
• Impersonate user, group, service account
  • https://www.cyberark.com/resources/threat-research-blog/kubernetes-pentest-methodology-part-1
  • some automated tools:
    • https://github.com/cyberark/KubiScan
    • https://github.com/alcideio/rbac-tool

Publicly Exposed Services and Ingresses

#get all services and ingresses, from all namespaces
kubectl get services --all-namespaces
kubectl get ingresses --all-namespaces
 
#view details about a service or ingress
kubectl get service <SERVICENAME> -n <NAMESPACE> -o json
kubectl get ingress <INGRESSNAME> -n <NAMESPACE> -o json
 
#get network policies from all namespaces
kubectl get networkpolicies -A

Vulnerability/Service Scanning

Check for common vulnerable Kubernetes Components. You can do this manually:

kubectl get svc --all-namespaces -o go-template='{{range .items}}{{ $save := . }}{{range.spec.ports}}{{if .nodePort}}{{$save.metadata.namespace}}{{"/"}}{{$save.metadata.name}}{{" - "}}{{.name}}{{": "}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}'

Or try to use automated tools such as Kube-Hunter

#view the Active and Passive mode actions
kube-hunter --list
kube-hunter --list --active
 
#scan specific worker/master node
kube-hunter --remote <node-ip>
#scan all IP addresses in a network segment
kube-hunter --cidr 192.168.0.0/24
#use the kubeconfig file to connect to the Kubernetes API and detect nodes
kube-hunter --k8s-auto-discover-nodes
 
#run the tool in active mode
kube-hunter --remote <node-ip> --active
 
#manually specify the service account token
kube-hunter --active --service-account-token <token>

Common Kubernetes Add-ons

Those are typically running with privileged pods, so check:

• prometheus
• new relic
• cAdvisor
• Skooner
• Fluentbit
• Calico
• CoreDNS
• ArgoCD

Tooling

• Break out the Box
• KubeHound