Khan, who offered £600m for the national stadium, expressed to the FA that a lack of support "from within the game" had discouraged him from following through with the deal.
The FA leadership had been working to push through a sale since February 2017, despite opposition among members of the governing body’s council.
In a statement released on Wednesday, the FA chief executive Martin Glenn said: "Shahid Khan has informed us today that he will be withdrawing his offer to buy the stadium – and we fully respect his decision.
"Mr Khan believed that his offer to buy Wembley Stadium would release funds to help improve community football facilities in England and that it would be well received by all football stakeholders.
"At a recent meeting with Mr Khan he expressed to us that, without stronger support from within the game, his offer is being seen as more divisive than it was anticipated to be and has decided to withdraw his proposal.
"There has been much deliberation on both sides of this debate and it has undoubtedly raised awareness of the issue that community football facilities in England need significant investment."
Both chairman Greg Clarke and chief executive Martin Glenn were in favour of the deal, which was approved in principle by the FA Board at a meeting in September.
The Department for Digital, Culture, Media and Sport, Sport England and the Greater London Authority also signalled their support for a sale.
But it is understood that many of the FA's 127 council members had raised serious doubts about the long-term benefits of the sale, despite FA executives arguing it would take the expensive running costs of Wembley out off their hands and allow them to ramp up investment in grassroots infrastructure.
But Khan, who also owns the Jacksonville Jaguars NFL franchise, explained that this limited support meant he would be unwilling to proceed with discussions.
He said: “I’ve been clear publicly as well as in my correspondence with the FA Council that it would require a proper partnership, with the full and enthusiastic commitment of all involved, to maximise the benefits to the FA and game of football by way of 100 per cent private ownership of Wembley Stadium.
"At this moment, following last week’s FA Council hearing, it appears there is no definitive mandate to sell Wembley and my current proposal, subsequently, would earn the backing of only a slim majority of the FA Council, well short of the conclusive margin that the FA Chairman has required.
“The intent of my efforts was, and is, to do right by everyone in a manner that strengthens the English game and brings people together, not divides them. Unfortunately, given where we are today, I’ve concluded that the outcome of a vote next week would be far from sufficient in expressing the broad support favoured by the FA Chairman to sell Wembley Stadium.
"Until a time when it is evident there is an unmistakable directive from the FA to explore and close a sale, I am respectfully withdrawing my offer to purchase Wembley Stadium."
Khan did suggest, however, that he could return with a future bid under the right circumstances.
“I cannot rule out revisiting the opportunity at another time when perhaps the Football Association family is unified in its views on the opportunity."
The Football Foundation, the UK's largest sports charity, expressed its disappointment at Khan's withdrawal.
“The news that Mr Khan has decided to withdraw his offer to buy Wembley Stadium should come as a huge disappointment to community footballers everywhere," CEO Paul Thorogood said.
"This would have been a once in a life time opportunity to make considerable inroads into probably the most pressing issue facing football in this country.”
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies