Mining question (not sure if a RVN question or mining setup question)?

I've tinker around in crypto on and off mostly just on the exchanges for a couple years but finally decided to dedicate one of my gaming machines as a mining rig. I'm currently pulling 20-22Mh on a GTX1080 at 113% GPU speed, 73% VRAM, and 67*c. With no changes to anything if I leave my machine alone for a couple days and return it'll only be hashing at 5-9Mh but a simple restart and it's back to "normal". I can't seem to figure out why because it never does it while i'm watching it only when untouched for a while. Anything obvious to look at?