Manual Scheduling

Labels & Selectors

Taints and Toleration

Node Selectors

Node Affinity

Resources Requirement & Limit

DaemonSets

Static Pods

Multiple Scheduler

ref: https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/

Deploy Additional Scheduler

Each scheduler uses a separate configuration file and with each file having its own scheduler name. This is not how you would deploy a cusstom scheduler 99% of the time today because with kubeadm deployment, all the control plane component run as a pod or a deployment within the kubernetes cluster.

Deploy Additional Scheduler As a Pod

pod definition for custom scheduler

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

apiVersion: v1
kind: Pod
metadata:
  name: annotation-second-scheduler
  labels:
    name: multischeduler-example
spec:
  schedulerName: my-scheduler
  containers:
  - name: pod-with-second-annotation-container
    image: registry.k8s.io/pause:3.8

Exercise

Questions

1. What is the name of the POD that deploys the default kubernetes scheduler in this environment?
2. What is the image used to deploy the kubernetes scheduler?
3. Let’s create a configmap that the new scheduler will employ using the concept of ConfigMap as a volume. We have already given a configMap definition file called my-scheduler-configmap.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apiVersion: v1
data:
  my-scheduler-config.yaml: |
    apiVersion: kubescheduler.config.k8s.io/v1
    kind: KubeSchedulerConfiguration
    profiles:
      - schedulerName: my-scheduler
    leaderElection:
      leaderElect: false    
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: my-scheduler-config
  namespace: kube-system

at /root/ path that will create a configmap with name my-scheduler-config using the content of file /root/my-scheduler-config.yaml. 4. Deploy an additional scheduler to the cluster following the given specification Use the manifest file provided at /root/my-scheduler.yaml.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: my-scheduler
  name: my-scheduler
  namespace: kube-system
spec:
  serviceAccountName: my-scheduler
  containers:
  - command:
    - /usr/local/bin/kube-scheduler
    - --config=/etc/kubernetes/my-scheduler/my-scheduler-config.yaml
    image: <use-correct-image>
    livenessProbe:
      httpGet:
        path: /healthz
        port: 10259
        scheme: HTTPS
      initialDelaySeconds: 15
    name: kube-second-scheduler
    readinessProbe:
      httpGet:
        path: /healthz
        port: 10259
        scheme: HTTPS
    resources:
      requests:
        cpu: '0.1'
    securityContext:
      privileged: false
    volumeMounts:
      - name: config-volume
        mountPath: /etc/kubernetes/my-scheduler
  hostNetwork: false
  hostPID: false
  volumes:
    - name: config-volume
      configMap:
        name: my-scheduler-config

Use the same image as used by the default kubernetes scheduler. 5. a POD definition file is given. Use it to create a POD with the new custom scheduler.

1
2
3
4
5
6
7
8

apiVersion: v1 
kind: Pod 
metadata:
  name: nginx 
spec:
  containers:
  - image: nginx
    name: nginx

Solution

1. run k get pods -n kube-system -> kube-scheduler-controlplane
2. run k describe pod kube-scheduler-controlplane
3. run kubectl create configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml -n kube-system, then check the result by running k get configmap my-scheduler-config -n kube-system
4. steps

• check the image from pod using k describe pod kube-scheduler-controlplane -n kube-system | grep Image
• copy the image to the yaml file
• run k create -f my-scheduler.yaml

5. edit the yaml file

1
2
3
4
5
6
7
8
9

apiVersion: v1 
kind: Pod 
metadata:
  name: nginx 
spec:
  schedulerName: my-scheduler
  containers:
  - image: nginx
    name: nginx

and run k apply -f nginx-pod.yaml

Admission Controllers

Validating & Mutating Admission Controllers