❌pod节点启动失败，nginx服务无法正常访问，服务状态显示为imagepullbackoff。
[root@m1 ~]# kubectl get podsname ready status restarts agenginx-f89759699-cgjgp 0/1 imagepullbackoff 0 103m
查看nginx服务的pod节点详细信息。
[root@m1 ~]# kubectl describe pod nginx-f89759699-cgjgpname: nginx-f89759699-cgjgpnamespace: defaultpriority: 0service account: defaultnode: n1/192.168.200.84start time: fri, 10 mar 2023 08:40:33 +0800labels: app=nginx pod-template-hash=f89759699annotations: <none>status: pendingip: 10.244.3.20ips: ip: 10.244.3.20controlled by: replicaset/nginx-f89759699containers: nginx: container id: image: nginx image id: port: <none> host port: <none> state: waiting reason: imagepullbackoff ready: false restart count: 0 environment: <none> mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-zk8sj (ro)conditions: type status initialized true ready false containersready false podscheduled true volumes: default-token-zk8sj: type: secret (a volume populated by a secret) secretname: default-token-zk8sj optional: falseqos class: besteffortnode-selectors: <none>tolerations: node.kubernetes.io/not-ready:noexecute op=exists for 300s node.kubernetes.io/unreachable:noexecute op=exists for 300sevents: type reason age from message ---- ------ ---- ---- ------- normal backoff 57m (x179 over 100m) kubelet back-off pulling image "nginx" normal pulling 7m33s (x22 over 100m) kubelet pulling image "nginx" warning failed 2m30s (x417 over 100m) kubelet error: imagepullbackoff
发现，获取nginx镜像失败。可能是由于docker服务引起的。
于是，检查docker是否正常启动
systemctl status docker
发现，docker服务启动失败，手动尝试重新启动。
systemctl restart docker
但是，重启docker服务失败，出现如下报错信息。
[root@m1 ~]# systemctl restart dockerjob for docker.service failed because the control process exited with error code.see "systemctl status docker.service" and "journalctl -xe" for details.
执行systemctl restart docker命令失效。
接着，当执行docker version命令时，发现未能连接到docker daemon
[root@m1 ~]# docker versionclient: docker engine - community version: 20.10.17 api version: 1.41 go version: go1.17.11 git commit: 100c701 built: mon jun 6 23:03:11 2022 os/arch: linux/amd64 context: default experimental: truecannot connect to the docker daemon at unix:///var/run/docker.sock. is the docker daemon running?
于是，再次通过执行systemctl status docker命令，查看docker服务未能启动，阅读输出报错信息，如下所示。
[root@m1 ~]# systemctl status docker● docker.service - docker application container engine loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) active: failed (result: exit-code) since fri 2023-03-10 10:28:16 cst; 4min 35s ago docs: https://docs.docker.com main pid: 2221 (code=exited, status=1/failure)mar 10 10:28:13 m1 systemd[1]: docker.service: main process exited, code=exited, status=1/failuremar 10 10:28:13 m1 systemd[1]: docker.service: failed with result 'exit-code'.mar 10 10:28:13 m1 systemd[1]: failed to start docker application container engine.mar 10 10:28:16 m1 systemd[1]: docker.service: service restartsec=2s expired, scheduling restart.mar 10 10:28:16 m1 systemd[1]: docker.service: scheduled restart job, restart counter is at 3.mar 10 10:28:16 m1 systemd[1]: stopped docker application container engine.mar 10 10:28:16 m1 systemd[1]: docker.service: start request repeated too quickly.mar 10 10:28:16 m1 systemd[1]: docker.service: failed with result 'exit-code'.mar 10 10:28:16 m1 systemd[1]: failed to start docker application container engine.[root@m1 ~]#
通过上述输出显示，docker 服务进程的启动失败，状态为 1/failure。
✅接下来，尝试通过以下步骤来排查和解决问题：
1️⃣查看 docker 服务日志：使用以下命令查看 docker 服务日志，以便更详细地了解失败原因。
sudo journalctl -u docker.service
2️⃣ 通过输出ddocker日志分析，提取到了相关报错信息片段，发现是配置daemon中的/etc/docker/daemon.json配置文件出错导致的。
mar 10 10:20:17 m1 systemd[1]: starting docker application container engine...mar 10 10:20:17 m1 dockerd[1572]: unable to configure the docker daemon with file /etc/docker/daemon.json: invalid character '"' after object key:value pairmar 10 10:20:17 m1 systemd[1]: docker.service: main process exited, code=exited, status=1/failuremar 10 10:20:17 m1 systemd[1]: docker.service: failed with result 'exit-code'.mar 10 10:20:17 m1 systemd[1]: failed to start docker application container engine.mar 10 10:20:19 m1 systemd[1]: docker.service: service restartsec=2s expired, scheduling restart.mar 10 10:20:19 m1 systemd[1]: docker.service: scheduled restart job, restart counter is at 2.mar 10 10:20:19 m1 systemd[1]: stopped docker application container engine.
3️⃣此时，查看daemon配置文件/etc/docker/daemon.json是否配置正确。
[root@m1 ~]# cat /etc/docker/daemon.json{ # 设置 docker 镜像的注册表镜像源为阿里云镜像源。 "registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"] # 指定 docker 守护进程使用 systemd 作为 cgroup driver。 "exec-opts": ["native.cgroupdriver=systemd"]}
咋一看，配置信息没有什么问题，都是正确的，但仔细一看，就会发现应该在"registry-mirrors"选项的结尾添加逗号。犯了缺少逗号（,）导致的语法错误，终于找到了问题根源。
修改后：
[root@m1 ~]# cat /etc/docker/daemon.json{ "registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"], "exec-opts": ["native.cgroupdriver=systemd"]}[root@m1 ~]# cat /etc/docker/daemon.json{ "registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"], "exec-opts": ["native.cgroupdriver=systemd"]}
按下:wq报错退出。
4️⃣ 重新加载系统并重新启动docker服务
systemctl daemon-reloadsystemctl restart dockersystemctl status docker
5️⃣检查docker版本信息是否输出正常
[root@m1 ~]# docket version-bash: docket: command not found[root@m1 ~]# docker versionclient: docker engine - community version: 20.10.17 api version: 1.41 go version: go1.17.11 git commit: 100c701 built: mon jun 6 23:03:11 2022 os/arch: linux/amd64 context: default experimental: trueserver: docker engine - community engine: version: 20.10.17 api version: 1.41 (minimum version 1.12) go version: go1.17.11 git commit: a89b842 built: mon jun 6 23:01:29 2022 os/arch: linux/amd64 experimental: false containerd: version: 1.6.6 gitcommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 runc: version: 1.1.2 gitcommit: v1.1.2-0-ga916309 docker-init: version: 0.19.0 gitcommit: de40ad0
[root@m1 ~]# docker infoclient: context: default debug mode: false plugins: app: docker app (docker inc., v0.9.1-beta3) buildx: docker buildx (docker inc., v0.8.2-docker) scan: docker scan (docker inc., v0.17.0)server: containers: 20 running: 8 paused: 0 stopped: 12 images: 20 server version: 20.10.17 storage driver: overlay2 backing filesystem: xfs supports d_type: true native overlay diff: true userxattr: false logging driver: json-file cgroup driver: systemd cgroup version: 1 plugins: volume: local network: bridge host ipvlan macvlan null overlay log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog swarm: inactive runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc default runtime: runc init binary: docker-init containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 runc version: v1.1.2-0-ga916309 init version: de40ad0 security options: seccomp profile: default kernel version: 4.18.0-372.9.1.el8.x86_64 operating system: rocky linux 8.6 (green obsidian) ostype: linux architecture: x86_64 cpus: 2 total memory: 9.711gib name: m1 id: 4yis:fhsb:yxri:ced5:pjsj:eas2:bcr3:gjjf:fdpk:edjh:dvku:aiyj docker root dir: /var/lib/docker debug mode: false registry: https://index.docker.io/v1/ labels: experimental: false insecure registries: 127.0.0.0/8 registry mirrors: https://w2kavmmf.mirror.aliyuncs.com/ live restore enabled: false
至此，docker服务重启成功，pod节点恢复正常，nginx服务能够正常访问。
[root@m1 ~]# kubectl get podsname ready status restarts agenginx-f89759699-cgjgp 1/1 running 0 174m
查看pod详细信息，显示正常。
[root@m1 ~]# kubectl describe pod nginx-f89759699-cgjgpname: nginx-f89759699-cgjgpnamespace: defaultpriority: 0service account: defaultnode: n1/192.168.200.84start time: fri, 10 mar 2023 08:40:33 +0800labels: app=nginx pod-template-hash=f89759699annotations: <none>status: runningip: 10.244.3.20ips: ip: 10.244.3.20controlled by: replicaset/nginx-f89759699containers: nginx: container id: docker://88bdc2bfa592f60bf99bac2125b0adae005118ae8f2f271225245f20b7cfb3c8 image: nginx image id: docker-pullable://nginx@sha256:aa0afebbb3cfa473099a62c4b32e9b3fb73ed23f2a75a65ce1d4b4f55a5c2ef2 port: <none> host port: <none> state: running started: fri, 10 mar 2023 10:37:42 +0800 ready: true restart count: 0 environment: <none> mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-zk8sj (ro)conditions: type status initialized true ready true containersready true podscheduled true volumes: default-token-zk8sj: type: secret (a volume populated by a secret) secretname: default-token-zk8sj optional: falseqos class: besteffortnode-selectors: <none>tolerations: node.kubernetes.io/not-ready:noexecute op=exists for 300s node.kubernetes.io/unreachable:noexecute op=exists for 300sevents: type reason age from message ---- ------ ---- ---- ------- normal backoff 58m (x480 over 171m) kubelet back-off pulling image "nginx"[root@m1 ~]#
以上就是kubernetes中nginx服务启动失败如何排查的详细内容。