The suspects were detained by armed police shortly after midnight, on Thursday morning.
It came days after the UK’s terror threat level was raised to severe, meaning attacks are highly likely.
A spokesperson for the Metropolitan Police said the men had been seen “acting suspiciously in a car in the Westminster area”.
“They were stopped and arrested on Pall Mall by armed officers,” a statement added. “The police officers searched the men and the vehicle.
“They subsequently arrested the men — two 34-year-olds from east London — on suspicion of preparation of terrorist acts, contrary to section 5 of the Terrorism Act 2006.”
The offence is used to prosecute people accused of planning or attempting a terror attack.
The men, who have not been identified, remain in custody at a London police station.
Pall Mall is a historic road that connects to Trafalgar Square in central London, and contains several international embassies.
Commander Richard Smith, head of the Met Police Counter Terrorism Command, said: “Every day, police officers are on patrol across the capital, watching for suspicious behaviour and stopping people they suspect may be involved in criminality.
“The public can help us continue to keep London safe by being vigilant and reporting any suspicious behaviour to police.
“Following the increase to the UK threat level from terrorism last week — as a precautionary measure in response to recent events in France and Austria — I would like to emphasise that the public should be alert but not alarmed."
Last week, the security minister warned that the “temperature of the terror threat in Europe is rising” following a spate of attacks in France and Austria.
James Brokenshire told the House of Commons that intelligence agencies assessed the threat of attacks in Britain as highly likely.
Explaining the decision to increase the UK’s terror threat level from substantial to severe, he said: “There is a risk that the recent attacks in France and Austria could have a galvanising effect in parts of Europe, including the UK … they suggest the temperature of the threat in Europe is rising.”
Neil Basu, the head of UK Counter Terrorism Policing, said the change was “not based on a specific threat”.
Mr Brokenshire said police patrols would increase because of the raised threat level.
“The UK faces a serious and enduring threat from terrorism,” he told MPs.
“Recent events in France and Vienna have provided a stark and brutal reminder of the risks that we face and the continuing need to be resolute in the face of those who would wish to sow division and hatred.”
The spate of Isis-inspired attacks that hit England in 2017 was preceded by a string of atrocities on mainland Europe, including several in France.
Isis, al-Qaeda and other jihadi groups have been ramping up calls in recent weeks for supporters around the world to commit atrocities.
Counter-terrorism police believe that lockdown restrictions brought in to tackle the coronavirus pandemic may force a change to terrorist methodology.
Through its propaganda, Isis popularised the targeting of crowded places using low-tech methods including vehicles and knives.
Mr Brokenshire said that since March 2017, British security service had foiled 27 terror plots — 19 Islamist and eight far right.
A total of 12 attacks have been carried out in the same period, including the rampages by former terrorist prisoners at London Bridge and Streatham over the past year.
The count includes the Reading attack, where three people were murdered in June, but perpetrator Khairi Saadallah has denied having a terrorist motive.