Colback might have picked up a second yellow card for a 71st-minute foul on Victor Moses, but escaped punishment and three minutes later found the net.
Carver's side were denied victory, though, by Peter Crouch's 90th-minute header.
On whether Colback should have been sent off, Carver told Sky Sports News: "I think the referee used his common sense. Jack's not a nasty guy, he's competitive, but it works both ways and I think one or two of those lads on the Stoke team got away with one or two things."
Carver was disappointed his side were unable to hold on, with Gabriel Obertan and Ayoze Perez missing golden chances before Crouch struck.
"We shot ourselves in the foot," Carver added.
Crouch felt Stoke deserved to take all three points from the match.
"I thought we controlled the game, I though we deserved to win the game," the former England striker said.
"It was a cruel blow when they scored because I thought we were on top at that point."