Update per 1.24 changes

Signed-off-by: Paul Bastide <[email protected]>
prb112 · Sep 28, 2022 · adc4f92 · adc4f92
1 parent 8204849
commit adc4f92
Show file tree

Hide file tree

Showing 4 changed files with 42 additions and 45 deletions.
diff --git a/...r_policy/2_TopologyAndDuplicates_RemovePodsViolatingTopologySpreadConstraint.md b/...r_policy/2_TopologyAndDuplicates_RemovePodsViolatingTopologySpreadConstraint.md
@@ -10,6 +10,8 @@ Do not run this in a LIVE cluster, this should be dedicated to the specific test
 
 Since this is run as part of the TopologyAndDuplicates profile there is an implication of `RemoveDuplicates` which makes it harder to exercise without three nodes and at most a 1 Pod in a Kubernetes (Deployment, ReplicaSet, DaemonSet et cetra) as when the labels change in the environment the `RemoveDuplicates` will be executed first.
 
+Note, in Kubernetes 1.24, they introduced Built-in default constraints. [link](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#internal-default-constraints) as such I have used `custom` instead of the node label `topology.kubernetes.io/zone`.
+
 ## Steps
 
 0. Check you have Three Nodes
@@ -32,7 +34,7 @@ kubedescheduler.operator.openshift.io/cluster created
 2. Check the configmap to see the Descheduler Policy. 
 
 ```
-oc -n openshift-kube-descheduler-operator get cm cluster -o=yaml
+$ oc -n openshift-kube-descheduler-operator get cm cluster -o=yaml
 ```
 
 This ConfigMap should show the excluded namespaces and `strategies.RemovePodsViolatingTopologySpreadConstraint.includeSoftConstraints: false` is configured.
@@ -60,15 +62,22 @@ Note, you'll see `Error from server (NotFound): namespaces "test" not found` if
 a. `worker-0`
 
 ```
-$ oc label node 'worker-0.xip.io' topology.kubernetes.io/zone=a
+$ oc label node 'worker-0.xip.io' custom=a
 node/worker-0.xip.io labeled
 ```
 
 b. `worker-1`
 
 ```
-$ oc label node 'worker-1.xip.io' topology.kubernetes.io/zone=b
-node/worker-1.xip.io not labeled
+$ oc label node 'worker-1.xip.io' custom=a
+node/worker-1.xip.io labeled
+```
+
+c. `worker-2`
+
+```
+$ oc label node 'worker-2.xip.io' custom=a
+node/worker-2.xip.io labeled
 ```
 
 5. Create the ReplicaSet
@@ -83,77 +92,65 @@ replicaset.apps/ua created
 b. second replicaset
 
 ```
-$ oc -n test apply -f files/2_TopologyAndDuplicates_rs_b.yml                                          
+$ oc -n test apply -f files/2_TopologyAndDuplicates_rs_b.yml
 replicaset.apps/ub created
 ```
 
 c. third replicaset
 
 ```
-$ oc -n test apply -f files/2_TopologyAndDuplicates_rs_c.yml                                          
+$ oc -n test apply -f files/2_TopologyAndDuplicates_rs_c.yml
 replicaset.apps/uc created
 ```
 
-6. Verify the pods are scheduled on `worker-0`.
+6. Verify the pods are split and worker-2 has most of the load.
 
 ```
 $ oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName'
 Name               NodeName
-Name       NodeName
 ua-dwwwh   worker-0.xip.io
-ua-gql9r   worker-0.xip.io
-ub-7j6bw   worker-1.xip.io
-ub-rhhx8   worker-0.xip.io
-uc-4f52z   worker-2.xip.io
-uc-kkffv   worker-1.xip.io
-```
-
-7. Remove a label to worker-1
-
-```
-$ oc label node 'worker-1.xip.io' topology.kubernetes.io/zone-
-node/worker-1.xip.io unlabeled
+ua-gql9r   worker-2.xip.io
+ub-7j6bw   worker-0.xip.io
+ub-rhhx8   worker-2.xip.io
+uc-4f52z   worker-1.xip.io
+uc-kkffv   worker-2.xip.io
 ```
 
+Note, you can use `oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName' | grep -v NodeName | awk '{print $NF}' | sort | uniq -c` to see the node / pod distribution.
 
-8. Reassign the label for worker-2
+7. Reassign the label for worker-1
 
 ```
-oc label node 'worker-1.xip.io' topology.kubernetes.io/zone=b
+$ oc label node worker-1.xip.io custom=b --overwrite  
 node/worker-1.xip.io labeled
 ```
 
-9. Check the Pod distribution to see the rebalancing.
+8. Check the Pod distribution to see the rebalancing.
 
 ```
 $ oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName'
 Name       NodeName
-ua-482lg   worker-0.xip.io
-ua-hqvm2   worker-1.xip.io
-ua-pnlx8   worker-2.xip.io
-ub-k7g2m   worker-2.xip.io
-ub-ndmqp   worker-1.xip.io
-ub-vsnpt   worker-0.xip.io
-uc-l5pkk   worker-0.xip.io
-uc-rc9bp   worker-2.xip.io
-uc-tkpn8   worker-1.xip.io
+ua-7z859   worker-0.xip.io
+ua-fs8xl   worker-2.xip.io
+ub-42qlf   worker-0.xip.io
+ub-d2bgz   worker-2.xip.io
+uc-cxlcn   worker-1.xip.io
+uc-g22db   worker-0.xip.io
 ```
 
-10. Verify the logs show an eviction based on the deschedulerPodTopologySpread
+Note, you can use `oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName' | grep -v NodeName | awk '{print $NF}' | sort | uniq -c` to see the node / pod distribution.
+
+9. Verify the logs show an eviction based on the deschedulerPodTopologySpread
 
 ```
-$ oc -n openshift-kube-descheduler-operator logs -l app=descheduler  --tail=20000
-I0517 15:39:38.704420       1 event.go:294] "Event occurred" object="test/ua-hqvm2" kind="Pod" apiVersion="v1" type="Normal" reason="Descheduled" message="pod evicted by sigs.k8s.io/deschedulerPodTopologySpread"
-I0517 15:39:38.709303       1 round_trippers.go:553] POST https://172.30.0.1:443/api/v1/namespaces/test/events 201 Created in 4 milliseconds
-I0517 15:39:38.791390       1 round_trippers.go:553] POST https://172.30.0.1:443/api/v1/namespaces/test/pods/ub-vsnpt/eviction 201 Created in 87 milliseconds
-I0517 15:39:38.791471       1 evictions.go:160] "Evicted pod" pod="test/ub-vsnpt" reason="PodTopologySpread"
-I0517 15:39:38.791560       1 descheduler.go:287] "Number of evicted pods" totalEvicted=3
+$ oc -n openshift-kube-descheduler-operator logs -l app=descheduler  --tail=20000 | | grep deschedulerPodTop
+I0928 19:43:09.447104       1 event.go:294] "Event occurred" object="test/uc-rfdkq" fieldPath="" kind="Pod" apiVersion="v1" type="Normal" reason="Descheduled" message="pod evicted by sigs.k8s.io/deschedulerPodTopologySpread"
 ```
 
-11. Clean up replicaset
+10. Clean up replicaset
 
 ```
-$ oc -n test delete replicaset/ua replicaset/ub replicaset/uc                                                   
+$ oc -n test delete rs ua ub uc                                                   
 replicaset.apps "ua" deleted
 replicaset.apps "ub" deleted
 replicaset.apps "uc" deleted
@@ -168,7 +165,7 @@ This is a simple realignment of pods using a topology spread constraint.
 1. Check the Node-Pod Distribution
 
 ```
-$ oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName' | awk '{print $NF}'  | sort | uniq -c
+$ oc -n test get pods -o=custom-columns='Name:metadata.name,NodeName:spec.nodeName' | grep -v NodeName | awk '{print $NF}'  | sort | uniq -c
    1 NodeName
    4 worker-1.xip.io
    2 worker-2.xip.io

diff --git a/descheduler_policy/files/2_TopologyAndDuplicates_rs_a.yml b/descheduler_policy/files/2_TopologyAndDuplicates_rs_a.yml
@@ -15,7 +15,7 @@ spec:
     spec:
       topologySpreadConstraints:
         - maxSkew: 1
-          topologyKey: topology.kubernetes.io/zone
+          topologyKey: custom
           whenUnsatisfiable: DoNotSchedule
           labelSelector:
             matchLabels:

diff --git a/descheduler_policy/files/2_TopologyAndDuplicates_rs_b.yml b/descheduler_policy/files/2_TopologyAndDuplicates_rs_b.yml
@@ -15,7 +15,7 @@ spec:
     spec:
       topologySpreadConstraints:
         - maxSkew: 1
-          topologyKey: topology.kubernetes.io/zone
+          topologyKey: custom
           whenUnsatisfiable: DoNotSchedule
           labelSelector:
             matchLabels:

diff --git a/descheduler_policy/files/2_TopologyAndDuplicates_rs_c.yml b/descheduler_policy/files/2_TopologyAndDuplicates_rs_c.yml
@@ -15,7 +15,7 @@ spec:
     spec:
       topologySpreadConstraints:
         - maxSkew: 1
-          topologyKey: topology.kubernetes.io/zone
+          topologyKey: custom
           whenUnsatisfiable: DoNotSchedule
           labelSelector:
             matchLabels: