Profiling a service in Go
pprof
helps in collecting different type of performance profiles in Go, such as CPU, Heap, Goroutine etc. How to setup an application to generate these profiles is described at official go language documentation.
Making sense of Profile
To make sense of a profile, we can use go tool
chain. For example, if we have a CPU profile, we can analyze it using go tool pprof -http localhost:9000 cpu.profile
. This will open profile information in a browser and we can analyze it.
What if pod is not exposed as a service
If service is reachable and you can ssh
to it, it is easy to create a ssh tunnel
and analyze the profile from local machine itself. But there are cases when application is running in kubernetes and we can’t reach it using tunnel. In this scenario, we can get profile on node as then copy that profile to a machine that has go installed. Here are the steps to do that:
- Make sure pprof is enabled for application
- Choose a pod to get the profile from
- Exec to chosen pod using
kubectl exec -it <pod name> — sh
- Once in pod run
curl localhost:9000/debug/pprof/goroutine —output goroutine-file
- If file is created and saved, exit from pod
- From your local machine, run
kubectl cp <remote-pod-name>:<location-to-file> <local file location>/file-name
- Run
go tool pprof -http localhost:9000 <local file location>/file-name
to analyze profile in browser