Bug: Distraction pop-up repeats and won't disappear

I want the distraction pop-up to stop appearing once I've indicated that an activity is not a distraction. Currently, the pop-up appears over and over again and doesn't disappear even after I click "this is not a distraction". This is very disruptive to my workflow.