Furneaux told the Grimsby Telegraph that the meeting started without the intention to sack Groves, but the inevitable couldn't be avoided.
Graham Rodger, with the help of youth team coach Neil Woods, will take the helm until a replacement is named.
"The manager has been relieved of his duties after a full and frank meeting," Furneaux told the newspaper.
"We had a meeting to discuss the way forward and the outcome was for Paul to be relieved of his duties as manager.
"We're continuing to give him the facilities so that he can still train with the club.
"We're going to invite Graham Rodger to continue in his current role and take charge while a decision is made."
Furneaux went onto to pay tribute to Groves and thank him for his efforts.
"We thank him for his tremendous hard work and the service he has given us," Furneaux added.
"But there needs to be a change. I have tremendous respect for Paul and we didn't set out at the meeting with this intention in mind.
"But we have to find a way to take the club forward and we needed to do something."