A goalless draw would have been sufficient to take Trevor Francis's previously unbeaten team to the top for the first time in 11 years.
Indeed Francis, the City manager, admitted that a goalless draw had been the extent of his ambitions against the relegated Weirsiders, whom he tips strongly for a quick return to the Premiership.
"Although Sunderland made a slow start to the season", he said, "I think they will still go up. They were exactly as I expected them to be - well prepared, very well organised and with enough ability to go with their physical presence. We had to grind out a result and I thought the game was a banker to end nil-nil. I was very disappointed we gave away a goal the way we did, from a set piece."
It was a long throw in by the full-back Daren Williams which led to the goal, drilled in right-footed by Gray after both Martin Smith and Andy Melville had helped the ball on.
Gray was also involved when Birmingham inadvertently produced their only on-target goal attempt of the match. Six minutes before half-time Martin Grainger swung in a corner and the Sunderland defender Richard Ord, under pressure from Steve Bruce, back-headed the ball goalwards but Gray was perfectly positioned to head the ball off the line.
A goal at that stage, or in fact at any stage, of a scrappy, combative contest which contained six bookings, would have been more than the home side deserved.
Paul Furlong and Grainger both sent late efforts narrowly over the bar as they searched for an equaliser, but they were always second best to a Sunderland team who have found their First Division feet after seemingly suffering the post-Premiership blues in their opening matches of the new campaign.
They took this game to the home side from the early stages, when twice in a minute the midfielder Allan Johnston - first with a shot then with a header - forced Ian Bennett into good saves. Apart from failing to keep out Gray's winning goal, Bennett had to make no further saves.
Goal: Gray (0-1) 72.
Birmingham (4-4-2): Bennett; Wassall, Bruce (Johnson, 63), Ablett, Grainger; McCarthy, Hughes, O'Connor (Holland, 79), Ndlovu (Francis, 71); Furlong, Devlin.
Sunderland (4-4-2): Perez; Williams, Melville, Ord, Makin; Johnston, Ball, Clark, Gray; Philips (Mullin, 53), Smith (Rae, 88). Substitute not used: Bracewell.
Referee: J Kirkby (Sheffield).
Bookings: Birmingham: Wassall, Furlong, Francis. Sunderland: Makin, Ord, Gray.
Man of the match: Ord.Reuse content