How to get exclusive core for Pod in Kubernetes?

Question

I have a cluster and set up kubelet on a node (name is myNode) with the static CPU Manager Policy. So I've started kubelet with --cpu-manager-policy=static (to set the static policy) and --reserved-cpus=1 (to make sure kubelet has one core to run on exclusively) as explained here.

Checking /var/lib/kubelet/cpu_manager_state it gives me

cat /var/lib/kubelet/cpu_manager_state
{"policyName":"static","defaultCpuSet":"0-3","checksum":611748604}

which should be fine. I then start a pod with the following pod spec

apiVersion: v1
kind: Pod
metadata:
  name: wl
  labels:
    app: wl
spec:
  containers:
  - name: wl
    image: docker.io/polinux/stress:latest
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh","-c"]
    args: ["echo 'workload' && stress --cpu 4"]
    resources:
      requests:
        cpu: 1
      limits:
        cpu: 1
  nodeName: myNode

and start it. It get's scheduled on the desired node "myNode". I then check for the processes with

ps aux | grep stress
root     2966141  0.2  0.0    780     4 ?        Ss   10:54   0:00 stress --cpu 4
root     2966154 27.1  0.0    780    36 ?        R    10:54   0:02 stress --cpu 4
root     2966155 26.7  0.0    780    36 ?        R    10:54   0:02 stress --cpu 4
root     2966156 28.6  0.0    780    36 ?        R    10:54   0:02 stress --cpu 4
root     2966157 27.3  0.0    780    36 ?        R    10:54   0:02 stress --cpu 4

and then which CPUs they are running on with

ps -o pid,psr,comm -p 2966154 2966155 2966156 2966157
    PID PSR COMMAND
2966154   0 stress
2966155   1 stress
2966156   2 stress
2966157   3 stress

It looks like there are 4 processes running, but all of them on different CPUs. I would have expected that the Pod fails to run since it's allowed to only run on one core while the stress --cpu 4 wants to start 4 threads on 4 CPUs. With the default CPU Manager Policy, this would be the expected behavior, but I've configured the static one.

Any hint what the problem could be?

Answer 1

You also need to provide memory request and limit in order to qualify for the Guaranteed tier and exclusive cores:

apiVersion: v1
kind: Pod
metadata:
  name: wl
  labels:
    app: wl
spec:
  containers:
  - name: wl
    image: docker.io/polinux/stress:latest
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh","-c"]
    args: ["echo 'workload' && stress --cpu 4"]
    resources:
      requests:
        cpu: "1"
        memory: "200Mi"
      limits:
        cpu: "1"
        memory: "200Mi"
  nodeName: myNode

Verify the Pod by kubectl describe pod wl