With the deadlock over Brexit showing no sign of easing at Westminster and less than a month to go until parts of the country go the polls, Lord Hayward also claimed turnout could dramatically decline.
In England alone at the 2 May election there are 8,374 seats up for grabs, including 33 metropolitan councils, 119 district councils, and 30 unitary authorities.
Lord Hayward said on Monday this amounted to the election being "target practice on Tory territory" and suggested the Liberal Democrats are in the best position to make gains in May.
While he refused to be drawn on specific predictions regarding seat gains, on a Brexit effect the Tory peer said: "I have no doubt in my own mind there is a Brexit benefit to the government if there is a deal.
"The corollary of that is that there is a loss... there is a clearly a disadvantage, a disbenefit, a deficit to the Tory party to not having a deal," he said.
“Talk to any MPs and they will say exactly the same thing. They want a deal so they can go out on the doorstep and say we’ve got a deal. But if there isn’t a deal, the Tories have a disadvantage."
Lord Hayward also provided a bleak forecast in terms of voter turnout, suggesting there will be a "dramatic decline" due to the fact the last time these specific elections were contested, in 2015, they coincided with a general election, attracting voters in greater numbers.
The Conservative peer added that turnout would also be impacted by a large group of people who are "just so hacked off with politics in general they will just not go and vote".
In regards to party candidates, he claimed the Conservatives had managed to field a record level, with individuals up for election in 96 per cent of the seats.
“Considering there is this general perception that the grassroots of the Tory party are in total despair, and immobile, 96 per cent is the highest figure in this four-year cycle that’s ever been achieved," he said
By contrast, Labour are fielding candidates in 77 per cent of the seats, meaning they will not be challenging in nearly a quarter of council seats on 2 May. In 2011 and 2015, the party put forward 72 per cent and 75 per cent respectively.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies