Debugging Network Issues in K8s

Troubleshooting networking issues in docker and kubernetes is often difficult and the root cause can be in different places including the underlying networking infrastructure. I have collected some information which might be helpful.

Posted November 25, 2021 by Adrian Wyssmann

Most importantly, is that you understand the concepts of networking in docker and kubernetes:

Docker images

In order to debug issues you need the right tools. The following docker images provide a set of tools. While netshoot is my preference, the others are also very helpful:

Docker Image Description
nicolaka/netshoot a Docker + Kubernetes network trouble-shooting swiss-army container
gopher-net/dockerized-net-tools Collection of Dockerized Open Source Network Monitoring, Performance and Management Related Tools
Praqma/Network-MultiTool Multi-arch multitool for container network troubleshooting

So you can use these docker images to test dns resolution or tracing a ping

Usage on Docker

You can start the container and then login to it

docker run -d nicolaka/netshoot --name netshoot
docker exec -it netshot /bin/bash

You can also run some commands directly rather than login to the container:

docker run -it -rm nicolaka/netshoot ping google.com

Run a command and then delete the container again:

docker run -it --rm gophernet/traceroute 8.8.8.8

Usage on Kubernetes

You can either create a single pod - without a deployment:

kubectl run netshoot --image=nicolaka/netshoot

Or you create a deployment:

kubectl create deployment netshoot --image=nicolaka/netshoot

Then you open the console of the pod

kubectl exec -it netshoot /bin/bash

Which tool for what?

As Linux user you might be familiar with most of the tools provided within these containers. If not, I really recommend to have a look at https://github.com/nicolaka/netshoot, which gives you a lot of details.

Kubernetes Commands

Here is a list of helpful commands

  1. Get the CoreDNS

    k get configmap -o yaml coredns -n kube-system
    

Edit this page