Role-based-access control (RBAC) is an important tool for restricting the privileges of users. In this article, we'll discuss some of the best practices for leveraging RBAC within an organization, as well as some overarching security guidelines.
Principle of Least Privilege
A fundamental guiding security principle is the principle of least privilege. This states that a user (or team, application, etc) should have only the minimum set of privileges that allows them to perform their job function. For example, an application developer probably does not require privileges to create and delete Kubernetes clusters. In fact, in most cases a developer probably only needs access to a particular Kubernetes namespace.
In practice, following this principle means starting with zero privileges and adding only what is needed from there. This is in contrast with starting with all privileges and removing access from there, since it's much easier to maintain strict security using the former approach.
Kubernetes Namespaces as a Security Boundary
It's typical that a user will require both read and write access in order to be able to perform their job function properly. However, it's often the case that this read/write access is only required for a subset of resources available on a Kubernetes cluster. But where does it make sense to draw the line? In most cases, the easiest answer is to draw the line along Kubernetes namespaces.
Let's consider a typical application developer. This developer is working on an application that will run on Kubernetes. This developer should be able to easily test and iterate changes on a development cluster. This means that they should have both read and write access in some capacity. However, in most cases they shouldn't have write access to the entire development cluster. It's easy to imagine scenarios in which there are many developers all relying on the development cluster to be properly functioning. If each developer has access to system resources and other resources that surpass their minimum privilege requirements, then the potential is there for things to go wrong. This is why only giving access to specific namespaces is a good practice.
System and Critical Namespaces
There are two namespaces in particular that an administrator should pay special attention to. These are
kube-system (for all Kubernetes clusters) and
containership-core (for clusters managed by Containership).
kube-system namespace contains core Kubernetes components such as the scheduler and API server. It also contains other resources that are critical for proper cluster operation, e.g. the Container Network Interface (CNI) plugin and DNS pods. Suffice to say that erroneously modifying these resources can cause bad things to happen, so it's good to ensure that only those users that absolutely need access have access to
For Containership clusters, the
containership-core namespace is also of specific concern. This namespaces holds resources that are critical to a cluster's Containership integration, such as the cluster management plugin. Additionally, it holds custom resources which are synced from Containership cloud in order to implement various Containership features. For example, SSH keys (for users with SSH access) are synced to nodes by first syncing them from Containership cloud to a
User custom resource. In practice, this means that a user with write/edit access to
containership-core should be assumed to also have SSH access.
In summary, users should be locked down to a specific namespace whenever possible, and special care must be taken to avoid granting users access to some critical system namespaces.
If a user is given SSH access, it should be assumed that they also have full cluster admin access even if they have not been explicitly granted any Kubernetes permissions. This is because the master nodes contain an admin kubeconfig file that is generated during (and required for) the cluster bootstrapping process and cluster management thereafter. All users that have SSH access have root access to the nodes, so the full system is accessible.
It's important to keep in mind that there is no way to automatically remove resources that a user has created after their privilege to do so has been revoked. This is particularly important if a user is granted privileges to create Kubernetes roles and role bindings and then those privileges are revoked. Any roles and role bindings they may have created will remain on the cluster until they are manually deleted. As such, administrators must audit all Kubernetes roles and role bindings - especially after revoking privileges for a user. Hopefully many of these concerns can be avoided in the first place by following the principle of least privilege.