Nsight Systems

Nsight Systems visualizes what your GPU and CPU is doing and when. It will help to understand where you can overlap execution or memory transfer, where you have an overhead and more.

A good article about using Nsight Systems can be found here

https://developer.nvidia.com/blog/sc20-demos-new-nsight-systems-and-nsight-compute-demos/

If you are more the visual type the following article contains some demo videos about Nsight Systems

https://developer.nvidia.com/blog/sc20-demos-new-nsight-systems-and-nsight-compute-demos/