First major security flaw in popular cloud container orchestrator Kubernetes discovered – and it may be impossible to tell if you have been compromised
Find any firm at the forefront of digital transformation and there’s one thing you can bet on: it’s leveraging Kubernetes to deploy sophisticated applications that push the boundaries of modern-day application development.
Or, to be technically precise, it’s using Kubernetes to orchestrate containerised applications – enabling incredibly complex composite services comprising simpler microservices.
By solving the problem of container orchestration, a Kubernetes craze has taken hold of the cloud community, and its rapid adoption has left its technical precursor, the container platform Docker, eating dust.
But since Kubernetes won the container war (sometime in 2017 after it allied with the Cloud Native Computing Foundation (CNFC)), the security stakes have been high. Having all your eggs in one basket meant that any major security breach could be messy.
It’s a problem that Kubernetes has largely managed to avoid – in part thanks to its loyal following in the open source community. Indeed, the new security flaw was made prominent thanks to a post by a Google staff engineer on a public Kubernetes Google Group, was publicly disclosed on GitHub a week ago, and has been outlined in detail in a series of posts on Open Source patron Redhat’s website.
Kubernetes privilege escalation flaw
As outlined on Redhat’s website, the security hole or “privilege escalation flaw” is a nasty piece of work. In a nutshell, it makes it possible for any user to gain full administrator privileges on any compute node being run in a Kubernetes cluster.
Yes, you heard it right: that means any sensitive data can be stolen, malicious code injected, heck, if a hacker fancied it, they could just terminate an application altogether – all from within the firm’s firewall. Yikes.
The vulnerability itself is located in the Kubernetes API server. Using a specially crafted connection request, the hacker can connect through the Kubernetes API server direct to the backend. Once in the network, they can then send arbitrary requests over the same connection to the backend server.
Perhaps most alarmingly, the Kubernetes API server connections to the backend are all authenticated with Kubernetes Transport Layer Security (TLS) credentials – meaning all the nefarious connections appear above board and applications functioning as normal.
The stark reality of the flaw – as outlined in the original Github post – makes for some heavy reading for if you’re a Kubernetes user:
“There is no simple way to detect whether this vulnerability has been used. Because the unauthorized requests are made over an established connection, they do not appear in the Kubernetes API server audit logs or server log. The requests do appear in the kubelet or aggregated API server logs, but are indistinguishable from correctly authorized and proxied requests via the Kubernetes API server,” reads the post.
It doesn’t take a whole lot of hacking-nous or access privileges to take advantage of the flaw, either: “In default configurations, all users (authenticated and unauthenticated) are allowed to perform discovery API calls that allow this escalation,” continues the post.
Am I affected?
Don’t be fooled into thinking your Kubernetes-based service or product is immune from the flaw. Just read this from Redhat:
“It’s important to note that all Kubernetes-based services and products – including Red Hat OpenShift Container Platform, Red Hat OpenShift Online, and Red Hat OpenShift Dedicated – are affected.”
Thankfully, as quickly as the flaw was detected it was resolved. Kubernetes has issued patched versions of Kubernetes v1.10.11, v1.11.5, v1.12.3, and v1.13.0-rc.1. All prior versions remain exposed and users should stop using them immediately. Redhat has also issued patches and service updates to all affected Openshift users.
It remains to be seen whether the security flaw has been used to attack any Kubernetes user.