Grig Richters said the Nazi symbol was daubed seven times on his vehicle after he left it in a car park in Hanover at night.
The vandals also wrote “refugees out” on the van, which is being driven by a supporter following the 31-year-old as he walks from Paris to Berlin.
The German-born filmmaker set out on 28 October to bring attention to the growing number of unaccompanied children in refugee camps in Europe and to call on western governments to offer them refuge.
He described his brightly painted van – driven by a different supporter on each leg of his journey – as a “petition on wheels” designed to bring attention to his journey.
The vehicle has been signed by thousands of supporters he has met during his walk.
It is thought to have been vandalised while left in a secured car park overnight.
“This been an emotional journey for me and I was devastated for a moment when I saw it,” he told The Independent.
Mr Richter, who lives in London, reported the graffiti to the police and was able to scrub the swastikas from his van.
He said he had also been threatened with violence by a man in Dortmund and believes someone tampered with his van’s brakes in Brussels.
“I was actually expecting a lot more of this,” he added. “The far right is on the rise and that’s partly why I felt I had to do this.”
During the course of his walk, Mr Richter has met political leaders in 16 German towns and cities. He said all wanted to take in more refugee children.
"Now we need the German government to take action," he added. "Every day we wait, another child dies in the camps. They are dropping like flies."
Last week Amnesty International urged the government of Greece to take urgent action to "restore human dignity" to migrants living in squalid conditions in Moria, the country's largest refugee camp.
Public health inspectors on the island of Lesbos have declared "dangerous for public health and the environment" and said there was an "uncontrollable amount of waste" at the site.
"I have friends who volunteer in these camps and witness this on a daily basis," said Mr Richter. "Many of these children try to commit suicide."
He added: "The only thing these children want is to live a normal life. Many of these children lost their families during the war and they went through hell to come to Europe."
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies