Speaking with all the weight of his 12 years' experience in the post, he told the Independent: 'He's a mythical beast, the backwoodsman. You're asking me to define something I don't believe exists.'
In a rare public description of the Chief Whip's job, Lord Denham wrote in the May 1991 edition of The Field: 'It is suggested that, every time there is an important vote, the Chief Whip can summon up from the shires scores of peers who have never been to the House before and barely know their way into the chamber - let alone the division lobby.
'But since every year for the last 12 years there has been at least one occasion when this accusation has been made, one would have thought that the supply of unentered peers, or even those still at walk, would have been in danger of running out.'
Yet, whenever the Government majority in the Lords is thought to be at risk, the media turn to tales of backwoodsmen appearing from distant shores.
The very word 'backwoodsman' smacks of skulduggery and the dirty dealing of whips' offices through the ages.
But Lord Denham said this week: 'The last time I was accused of having brought in the backwoodsmen, I chose one day at random, when there was no vote on, and made a list of all those attending that day.
'I grouped them into those who normally attended once a week, those who were there once a fortnight, once a month, and less than 10 times a year - and a significant proportion had attended less than 10 times in the previous session. Every single day, rare attenders are frequent attenders in the House of Lords.'
Lord Denham was not alone in his scepticism. Viscount Whitelaw, former deputy prime minister and an ex-Leader of the House, also wondered at the idea of 'vast numbers of extraordinary people, over the hill' - waiting to ambush an unsuspecting government.
But behind the scorn, government uncertainty over tonight's vote was indicated yesterday by an increased war of words between the Tory machine and the pro-referendum dissidents.
Leaving little to chance, the Conservative Research Department yesterday issued a blow-by-blow response to statements by Lord Blake and Lord Pearson, of the referendum camp, accusing them of disingenuousness, and of lack of credibility and faith in the parliamentary system.
The last-minute briefing note also included a swipe at Baroness Thatcher, who plans to vote against the Government. Arguing that most of those supporting the referendum opposed the treaty, the department cited the 1975 words of the then Mrs Thatcher - 'Whoever tends to be against a Bill proposes a referendum.'