GPU usage dropping to 0. Newbie at mining RVN

When mining RVN with NB miner. Whenever it says "Generate new opencl kernel" the GPU usage drops to zero. I don't think it's good for my card because along with the gpu usage dropping down, the fans also spin down. It's only for like a second. How can I fix this?