The visitors levelled in the 85th minute when Adam Proudlock dived to head home Jamie Clarke's right-wing cross from point-blank range.
But Richman had the final say three minutes later, placing a right-foot shot inside Phil Barnes' left-hand post after Louis Dodds had headed on Steve Thompson's right-wing cross.
The visitors dominated most of the first half and Proudlock had fired a shot a foot wide, before Nathan Jarman was inches from breaking the deadlock in the 18th minute.
Jarman spun on to a pass from JP Kalala and struck an 18-yard drive which Joe Anyon tipped against the crossbar and over the top.
But Vale got the tonic they desperately needed when skipper Marc Richards put them ahead with his ninth goal of the season in the 28th minute.
Neil MacKenzie's shot was deflected into his path and Richards fired a terrific left-foot finish into the top right-hand corner of Barnes' net from ten yards.
The goal should have lifted Vale, but Grimsby's response was swift as Anyon made a brilliant reflex save to tip Liam Trotter's close-range effort away for a corner.
Vale improved at the start of the second half and Richards almost claimed his second in the 54th minute.
Trotter fouled Ross Davidson 20 yards out and Richards' free-kick was heading towards the top corner before Barnes pulled off a fine save.
Richards then had another chance when Barnes collected a back pass from Robbie Stockdale eight yards out, but a group of defenders charged down his shot.
But Grimsby came again and Proudlock thought he had secured a share of the spoils before Richman pounced.