Docker Desktop Kubernetes

External

• https://docs.docker.com/get-started/kube-deploy/

Internal

• Docker Desktop

Overview

Docker Desktop Kubernetes creates a virtual machine on your local machine and starting a single-node Kubernetes cluster inside that VM. It also configures the kubectl installed on the local machine with a context that allows it to talk to the cluster.

Installation

https://docs.docker.com/docker-for-windows/#kubernetes

Operations

Connecting into the Kubernetes VM

docker run -it --rm --privileged --pid=host justincormack/nsenter1

This is where kubelet, kube-apiserver, etc. run.

ingress-nginx Installation

ingress-nginx Installation on Docker Desktop Kubernetes

Troubleshooting

Access to Kubelet Logs

/Users/<...>/Library/Containers/com.docker.docker/Data/log/vm/kubelet.log

Kubelet pods (the directory is relative to the Kubernetes VM):

/var/lib/kubelet/pods/<pod-id>/volumes/....

Turning on kubelet verbosity.

Containers

Kubernetes pod containers are available on the Mac instances and can be listed with docker ps.

Other Resources

https://docs.docker.com/docker-for-mac/troubleshoot/

Idiosyncrasies

All Service Accounts have cluster-admin Permissions

Docker Desktop Kubernetes automatically adds a cluster role binding giving cluster-admin to all service accounts.. More details in https://stackoverflow.com/questions/62892972/kubernetes-service-account-default-permissions. The offending cluster role is "docker-for-desktop-binding":

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: docker-for-desktop-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts
  namespace: kube-system

Apparently, "namespace:" in the "system:serviceaccounts" group does not work as intended.

To fix, overwrite the biding with this:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: docker-for-desktop-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:kube-system

⚠️ The setting gets restored with each Kubernetes upgrade.

Directories for to /tmp hostPath volumes are not created in the system's /tmp

Docker Desktop 2.5.0.0, engine 19.03.13. If a hostPath volume is mounted as such:

spec:
  volumes:
    - name: 'test'
      hostPath:
        path: /tmp/something

the corresponding localhost "/tmp/something" does not show up in the system's /tmp. A corresponding directory is created in /containers/services/docker/rootfs/tmp inside the docker VM, which is unexpected.