It was very much a game of two halves with Grimsby taking a deserved 2-0 lead before the break but then being completely out played as they lost their way in the second half.
Grimsby started in sparkling style and for the second week in succession striker Gary Jones was their two-goal hero. Both his goals were set up by Curtis Woodhouse who pulled the strings in midfield and were both superbly taken.
The first on 13 minutes was half-volleyed in from an angle seven minutes later Jones somehow connected with an overhead kick as a corner came over into a packed goal mouth.
Minutes later defender Ben Futcher almost put the game beyond Bury only to be denied by a Kasper Schmeichel save of which his dad, former Manchester United keeper Peter, would have been proud.
Then, at the other end, Futcher almost put the ball into his own net when he somehow diverted over a close range effort Dwayne Mattis.
But in the second half Bury took the game by the scruff of the neck and, as Grimsby sat back and backed off, it enabled the Gigg Lane outfit to take control.
Just before the hour debut boy Jon Daly, who had a short spell on loan with Grimsby last season, pulled a goal back with a delightful chip from 20 yards out.
After that Grimsby briefly came to their senses and, as they went in search of a third goal, leading scorer Michael Reddy hit the post and Gary Cohen had an effort well saved.
But that was a brief respite as Bury continued to push forward and the agony was not over until after the 4 four minutes added at time elapsed.