The police and city officials deny having moved the beggars on. But an initiative by Lothian and Borders police, Operation Summer City, is thought to be the reason for their departure.
The police have been using hand-held video cameras to record down-and- outs they believe are being aggressive towards tourists.
The police are trying to make sure that the estimated 1 million tourists who visit Edinburgh this summer leave with a favourable impression of the city.
For the past year, City of Edinburgh Council has been trying to decide whether it wants to draw up a specific by-law banning begging from the city centre, or whether the police's existing powers are sufficient.
Police have general powers under the Civic Government (Scotland) Act to arrest beggars where breach of the peace, obstruction or fraud are suspected.
But some of those on the streets claim they are being victimised by the police simply for being homeless. Forty-one-year-old Ricky Scott lost his home in Edinburgh when he could no longer afford the rent. For six years he has slept in city doorways, in wind, rain, hail and snow.
The calls for a by-law have made Ricky and two other homeless men so angry that they have set up the Right To Beg Committee.
Ricky told the Independent On Sunday he had never been aggressive: "If I'm drunk, I tend to lie down and go to sleep. But, a beggar may be aggressive in response to a member of the public being rude. If you're just sitting on the street begging, saying nothing and somebody comes up and tells you - in no uncertain terms - to get a haircut, a job and a lot worse, the temptation to retaliate is huge.
"Nobody on the street is begging for the fun of it. Begging is humiliating; the most degrading thing in the world."
Ricky says his worst experience to date happened during a previous Festival. He was lying in his sleeping bag in a shop doorway on Princes Street when two men, shouting abuse, kicked him out into the main road as if he were a football.
"People who criticise us never ask us about our circumstances. They're not interested in the fact that a few coins might lie between us and destitution - especially not during the Festival. They don't want us spoiling the view."
Organisations which provide the homeless with meals say the Edinburgh Festival and all the tourists attract large numbers of beggars from all over Britain.
The police claim they are only videoing beggars who have been the subject of complaints by the public. A spokesman for Lothian and Borders police said: "We can't comment on what these videos reveal. They are sub judice because several beggars have been warned and charged as a result of our intelligence gathering. Of course, if councillors want to look at them, we will make them available."
But Susan Dalgety, deputy leader of the city council, said she had not known about the police videos.
"I support the use of closed-circuit TV for security reasons," she said. "But if the police are using hand-held video cameras, this is the first I have heard of it and I think it is somewhat excessive."Reuse content