I deploy a kubernetes cluster with conjure-up using juju and maas , but when I reboot a worker node , the kubelet always stay "Waiting to start".

juju run --application kubernetes-worker 'service snap.kubelet.daemon status '

     Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.671081   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "heapster-token-9srk4" (UniqueName: "kubernetes.io/secret/911ca2db-f23a-11e8-a65b-b4b52fc68a95-heapster-token-9srk4") pod "heapster-v1.6.0-beta.1-7854d488cd-bf69l" (UID: "911ca2db-f23a-11e8-a65b-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.671242   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "heapster-config-volume" (UniqueName: "kubernetes.io/configmap/911ca2db-f23a-11e8-a65b-b4b52fc68a95-heapster-config-volume") pod "heapster-v1.6.0-beta.1-7854d488cd-bf69l" (UID: "911ca2db-f23a-11e8-a65b-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.671276   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "eventer-config-volume" (UniqueName: "kubernetes.io/configmap/911ca2db-f23a-11e8-a65b-b4b52fc68a95-eventer-config-volume") pod "heapster-v1.6.0-beta.1-7854d488cd-bf69l" (UID: "911ca2db-f23a-11e8-a65b-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.671310   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-7bbx7" (UniqueName: "kubernetes.io/secret/665b2a28-f239-11e8-a65b-b4b52fc68a95-default-token-7bbx7") pod "default-http-backend-d4p7b" (UID: "665b2a28-f239-11e8-a65b-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.671350   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "nginx-ingress-kubernetes-worker-serviceaccount-token-zn2rj" (UniqueName: "kubernetes.io/secret/6e795d96-f239-11e8-a65b-b4b52fc68a95-nginx-ingress-kubernetes-worker-serviceaccount-token-zn2rj") pod "nginx-ingress-kubernetes-worker-controller-b2xk4" (UID: "6e795d96-f239-11e8-a65b-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.771972   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "metrics-server-token-9gnf7" (UniqueName: "kubernetes.io/secret/6d1daf56-f242-11e8-98bb-b4b52fc68a95-metrics-server-token-9gnf7") pod "metrics-server-v0.3.1-67bb5c8d7-kc6mm" (UID: "6d1daf56-f242-11e8-98bb-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.772249   27126 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "metrics-server-config-volume" (UniqueName: "kubernetes.io/configmap/6d1daf56-f242-11e8-98bb-b4b52fc68a95-metrics-server-config-volume") pod "metrics-server-v0.3.1-67bb5c8d7-kc6mm" (UID: "6d1daf56-f242-11e8-98bb-b4b52fc68a95")
    Nov 27 15:45:54 node01 kubelet.daemon[27126]: I1127 15:45:54.872746   27126 reconciler.go:154] Reconciler: start to sync state
    Nov 27 15:45:55 node01 kubelet.daemon[27126]: E1127 15:45:55.873021   27126 configmap.go:195] Couldn't get configMap kube-system/metrics-server-config: couldn't propagate object cache: timed out waiting for the condition
    Nov 27 15:45:55 node01 kubelet.daemon[27126]: E1127 15:45:55.873200   27126 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/configmap/6d1daf56-f242-11e8-98bb-b4b52fc68a95-metrics-server-config-volume\" (\"6d1daf56-f242-11e8-98bb-b4b52fc68a95\")" failed. No retries permitted until 2018-11-27 15:45:56.373151686 +0000 UTC m=+23.511345715 (durationBeforeRetry 500ms). Error: "MountVolume.SetUp failed for volume \"metrics-server-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d1daf56-f242-11e8-98bb-b4b52fc68a95-metrics-server-config-volume\") pod \"metrics-server-v0.3.1-67bb5c8d7-kc6mm\" (UID: \"6d1daf56-f242-11e8-98bb-b4b52fc68a95\") : couldn't propagate object cache: timed out waiting for the condition"
  UnitId: kubernetes-worker/0
- ReturnCode: 3
  Stdout: |
    ● snap.kubelet.daemon.service - Service for snap application kubelet.daemon
       Loaded: loaded (/etc/systemd/system/snap.kubelet.daemon.service; enabled; vendor preset: enabled)
       Active: failed (Result: exit-code) since Tue 2018-11-27 15:45:35 UTC; 2min 8s ago
      Process: 24799 ExecStart=/usr/bin/snap run kubelet.daemon (code=exited, status=255)
     Main PID: 24799 (code=exited, status=255)

    Nov 27 15:45:35 node02 systemd[1]: snap.kubelet.daemon.service: Service hold-off time over, scheduling restart.
    Nov 27 15:45:35 node02 systemd[1]: snap.kubelet.daemon.service: Scheduled restart job, restart counter is at 5.
    Nov 27 15:45:35 node02 systemd[1]: Stopped Service for snap application kubelet.daemon.
    Nov 27 15:45:35 node02 systemd[1]: snap.kubelet.daemon.service: Start request repeated too quickly.
    Nov 27 15:45:35 node02 systemd[1]: snap.kubelet.daemon.service: Failed with result 'exit-code'.
    Nov 27 15:45:35 node02 systemd[1]: Failed to start Service for snap application kubelet.daemon.

snap list conjure-up

Name        Version              Rev   Tracking  Publisher   Notes
conjure-up  2.6.1-20181018.1610  1031  stable    canonical✓  classic

lsb_release -a

No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 16.04.5 LTS
Release:    16.04
Codename:    xenial
  • Which k8s from conjure-up is that please, core or cdk?, What is the output of juju run --application kubernetes-worker 'snap services ' Thanks! – bogdan Dec 8 at 19:24

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.