Mr Harvey polled only 41,171 votes, well behind the next-lowest, Rob Farley, who gained 51,123 and was elected.
The five sitting board members seeking re-election were all returned and none of three candidates nominated by members won a seat.
Mr Harvey has been an outspoken critic of the society's lending policies and its foray into the estate agency business.
Two other candidates nominated by members, Benjamin Jacobs and Robery Leng, were also former employees.
Brian Davis, who replaced Tim Melville-Ross as Nationwide's chief executive at the meeting, defended the society's sales staff. This week, the society stopped its sales staff from selling Guardian policies after a routine inspection visit by Lautro. The staff are being retrained.
About 1,300 staff are affected, of whom half are dedicated to financial services; the rest have other duties as well. Retraining will take between two weeks and two months.
Representatives of Guardian, the life company to which Nationwide is tied at present, would be coming into branches to deal with customers who had initially talked to Nationwide staff.Reuse content