Deputy House of Commons Speaker Nigel Evans denies rape and sexual assault allegations relating to two men

Former chief whip Andrew Mitchell says Nigel Evans should retain his position while he fights the charges

Deputy House speaker Nigel Evans gives a press statement at his home in Pendleton Lancashire (PA)

Nigel Evans, a Deputy Speaker of the House of Commons, will give up chairing MPs' debates temporarily as he fights allegations of rape and sexual assault relating to two men.

But the 55-year-old Conservative MP, who strongly denied the claims, is refusing to stand down formally as Deputy Speaker while he tries to clear his name. He was arrested at his home in Pendleton, Lancashire, on Saturday on suspicion of raping one man and sexually assaulting another – both in their twenties – between July 2009 and March 2013.

He was questioned by police for several hours before being released on bail until 19 June.

Yesterday Mr Evans told reporters the allegations had been "made by two people who are well known to each other and until yesterday, I regarded as friends". He said: "The complaints are completely false and I cannot understand why they have been made, especially as I have continued to socialise with one as recently as last week."

The MP added: "I appreciate the way the police have handled this in such a sensitive manner and I would like to thank my colleagues, friends and members of the public who have expressed their support and, like me, a sense of incredulity at these events."

Mr Evans, who came out as gay in 2010, is a popular figure at Westminster among MPs in all parties. But a Cabinet minister voiced the doubts among some MPs that he could carry on as a deputy to the Speaker, John Bercow.

Philip Hammond, the Defence Secretary, told the BBC's The Andrew Marr Show: "I think that is essentially a question the Speaker will have to consider. I stick rigidly to the view that we should treat people as innocent until they are proven guilty, but it is quite difficult to carry out a sensitive and high-profile role while being under this kind of scrutiny."

Mr Hammond said: "I know Nigel well, I have known him for years. I'm obviously as shocked as everybody else is."

But Mr Hammond's former cabinet colleague Andrew Mitchell has said Mr Evans should stay put, telling BBC Radio 4's Today programme: "If he (Evans) were to resign now we would be in a terrible position where every public office holder would be open to blackmail and false stories in the media, triggering their resignation.

"But in this particular circumstance I am in no doubt that Nigel Evans should not resign and that he will be able to fulfil his duties as required by the Speaker.

"We have three deputy speakers in the House of Commons so if necessary there can be a degree of burden sharing.

"He has not been charged, he has not been found guilty, and we do still live in a country where you are supposed to be innocent until proven guilty."

He said he did not believe the allegations against a friend and colleague of 20 years whom he had known "in good times and in bad times".

Mr Mitchell himself lost a high-profile job after damaging allegations. He resigned from cabinet last year after police officers manning the gates to Downing Street accused him of calling them "plebs."

Mitchell admits there was an ill-tempered argument after the officers refused to let him ride his bike through the main gate, but denies using that word. Scotland Yard are still investigating the “plebgate” incident, after a Channel 4 documentary cast doubt on the original police account.

Mr Evans would normally have deputised for Mr Bercow during parts of a week-long Commons debate on the Queen's Speech, which will mark the start of a new parliamentary year on Wednesday. He has asked to be relieved of that role, which will now be shared by the two other deputies – Lindsay Hoyle and Dawn Primarolo.

Mr Evans could remain out of the spotlight until the Commons returns from its Whitsun recess in early June.

Commons officials said Mr Bercow could not dismiss Mr Evans or ask him to stand down until the police had concluded their inquiries, because he was elected by MPs rather than appointed. Commons sources said the issue would be kept "under review" and Mr Evans could face private pressure to stand down.

His solicitor insisted Mr Evans would carry on as a Deputy Speaker and as MP for Ribble Valley, which he has represented for 21 years. His local party said: "The officers and members of the Ribble Valley Conservative Association have got to know and respect him and are shocked about his arrest. We are believers in the rule of law and are aware that these are merely allegations. In our democracy everyone accused is innocent until proven guilty."

The Conservative high command will be anxious to avoid a by-election in the constituency. Although Mr Evans had a majority of 14,769 at the last election, last week's strong second by Ukip in the South Shields by-election suggests that it could win such a seat.

In 1991, when the sitting Tory MP resigned, the Liberal Democrats won Ribble Valley in a by-election, but Mr Evans won it back at the following general election.

Tory MPs rallied strongly behind him on Sunday. Andrew Bridgen, who was accused of a sex assault two years ago before police dropped their inquiries six days later, said: "Fortunately in this country we have a rule that says you are innocent until proven guilty and I think that should be maintained. I personally think that Nigel should be able to continue as Deputy Speaker while the police, quite rightly, carry on with their investigations."