Gordon Brown suffered another blow as the Conservatives overturned Labour's 5,000-plus majority to snatch Norwich North.
Chloe Smith, who at 27 becomes the youngest MP in the Commons, won by 7,348 votes in the first such test of public opinion since the Westminster expenses scandal.
Mr Brown described it as "clearly a disappointing result''.
The poll was forced by the resignation of Labour MP Ian Gibson, who claimed almost £80,000 in second home expenses on a London flat which he later sold at a knock-down price to his daughter.
He quit last month after Labour's expenses "star chamber" barred him from standing for the party, sparking fury among colleagues and activists who said he was harshly treated by the "kangaroo court".
The 45 per cent turnout, while well down on the last general election, is about par for a by-election of this kind.
Labour held on to second place with 6,243 votes, although candidate Chris Ostrowski missed the count with a case of suspected swine flu.
Ms Smith wished him a speedy recovery and said she hoped to serve the voters of Norwich North "with the same honesty and conviction" as Mr Gibson.
"Roll on the General Election when the British people also have the opportunity to vote for change," she said.
The Tories polled almost 40 per cent of the vote, a swing of 16.49 per cent over Labour and well up on their 33 per cent showing at the 2005 General Election.
In her acceptance speech, Ms Smith said: "This by-election has been a victory for honest politics. The people of Norwich North have rejected the old politics of personal attacks, of bickering, of smears and scare stories.
"They have voted for change and in doing so they have sent a message to Gordon Brown very loud and very clear.
"They have not just voted against Labour - they have voted for the Conservative Party."
She hailed Mr Gibson as a "strong, independent and local voice".
Mr Ostrowski's wife, Katie, who stood in for him on the platform, thanked all the medical staff who had looked after him and said he would be "one of the first to congratulate Chloe".
Deputy Labour leader Harriet Harman said the result was "very disappointing", but stressed that the turnout had been "appalling".
"It is really a reflection of the quite unprecedented circumstances," she told BBC Radio 4's World At One. "It is more of our voters staying at home or going to other smaller parties rather than a big shift from Labour to the Conservatives."
She went on: "I think that people are apprehensive about their own financial circumstances. There is no doubt that Norwich has done very well over the past decade, and there is now uncertainty... people worried about businesses they have built up, and whether their jobs are secure, and, of course, the Government always holds responsibility for that."
Ms Harman said she was "sorry" that Mr Gibson had chosen to spark the by-election by quitting as an MP immediately when he was blocked from standing again.
Asked about claims that Mr Gibson had been treated unfairly, she said: "Yes, there have been questions asked about the process of how we make sure that if there are allegations of wrongdoing, that we properly review them."
A repeat of the Norwich North result across the country at the next General Election would see David Cameron swept into Downing Street with a majority of 218, Press Association analysis shows.
Left-wing Labour MP John McDonnell said it was a "shocking" result for the party and demanded an apology from Prime Minister Gordon Brown for the treatment of Mr Gibson.
"What is clear is that the Brown/Mandelson strategy is not working. However hard they spin it, this is a shocking result for Labour," he said.
Conservative MP Theresa May, the shadow work and pensions secretary, said: "This seat would have been 162nd on our target list at a General Election. It's a fantastic victory.
"Given the controversy over expenses, you may have expected voters to say 'a plague on all your houses'.
"The fact that they didn't shows how strongly people want change and want to send a message to Gordon Brown."
C gain from Lab
Chloe Smith (C) 13,591 (39.54%, +6.29%)
Chris Ostrowski (Lab) 6,243 (18.16%, -26.70%)
April Pond (LD) 4,803 (13.97%, -2.22%)
Glenn Tingle (UKIP) 4,068 (11.83%, +9.45%)
Rupert Read (Green) 3,350 (9.74%, +7.08%)
Craig Murray (Honest) 953 (2.77%)
Robert West (BNP) 941 (2.74%)
Bill Holden (Ind) 166 (0.48%, -0.17%)
Howling Laud (Loony) 144 (0.42%)
Anne Fryatt (NOTA) 59 (0.17%)
Thomas Burridge (Libertarian) 36 (0.10%)
Peter Baggs (Ind) 23 (0.07%)
C maj 7,348 (21.37%)
16.49% swing Lab to C
Electorate 75,124; Turnout 34,377 (45.76%, -15.33%)
2005: Lab maj 5,459 (11.61%) - Turnout 47,033 (61.09%)
Gibson (Lab) 21,097 (44.86%); Tumbridge (C) 15,638 (33.25%); Whitmore (LD) 7,616 (16.19%); Holmes (Green) 1,252 (2.66%); Youles (UKIP) 1,122 (2.39%); Holden (Ind) 308 (0.65%)