Boulding was brought down in the Brighton area seconds before Zesh Rehman's 55th minute opener.
That, according to Groves, was the changing point in the game and he told MarinersNet of his disappointment at the decision.
"Our view was that Michael had come inside, he's got a free shot at goal - I don't see the point in him going down," he said. "He's not somebody that dives. He's got a chance to let one go, and for him not to get a decision is somewhat bewildering.
"From there they've gone down there and scored a goal - there's the difference between something going for you and not."
Groves also, rather harshly, blamed Greg Young for the penalty that led to Brighton's third goal.
"The third goal, Greg's shown his inexperience, the lads held him off, he's turned him and Greg's not going to get the ball - induced the penalty," Groves added.