O'Grady calmly volleyed home in the 77th minute after a precise deep cross from Ian Sharps.
Grimsby played the last half-hour with ten men after Nick Fenton was sent off for his second bookable offence.
Ten minutes earlier Fenton had drawn Grimsby level, cancelling out Mark Hudson's first-half opener.
Fenton gave warning of his threat in the air as early as the second minute when his header was blocked on the line by Sharps.
Sixty seconds later Danny Harrison fired in a low 20-yard drive which the former Rotherham goalkeeper Gary Montgomery did well to save.
Fenton was then booked for a foul on Marc Newsham before Shaleum Logan played a one-two with Gary Jones on the edge of the area only to be denied by Andy Warrington.
Tom Newey curled in a dangerous free-kick which the evergreen Warrington tipped over the bar.
Stephen Brogan then put in a deep cross for young Rotherham striker Newsham but he headed over the bar.
On 39 minutes O'Grady and Newsham combined to set up Hudson who rolled in his first goal for the club after missing the first two months of the season with a knee injury.
On the stroke of half-time Jones missed a glorious chance to equalise when he failed to convert Logan's cross.
The equaliser arrived eight minutes into the second half when Fenton climbed highest to meet Danny Boshell's corner and power a header home.
Grimsby had the momentum ripped away from them ten minutes later when Fenton received a second yellow card for a clumsy hack on Harrison.
Jamie Yates and Pablo Mills tested Montgomery before, with 13 minutes left, O'Grady steered Rotherham to victory.