Investigators said they believe the attack, which took place by the Shrub Hill Retail Park, was racially motivated.
They added that the child sustained reddening to the face from the incident but was otherwise left unharmed.
The child's mother, who is in her 30s, was not injured.
"This was a nasty and unprovoked assault on a mother and her baby that has left the victims understandably shaken," said Sergeant Paul Smith, from West Mercia Police. "We're particularly concerned that the motivation for this is believed to be racial - there is no excuse for this type of behaviour and we will not tolerate it.”
He added that no arrests had been made but investigations were ongoing and CCTV footage was being reviewed to try to identify the vehicle involved.
"We realise this incident will cause concern amongst the community and want to reassure the public that we take reports of this nature very seriously and will continue to patrol the area,” he said. "I would appeal to anyone who may have witnessed the incident or saw anything suspicious in the area at the time of the incident to please get in touch."
Police data recorded in 2017-18 showed that babies and toddlers were among the victims of more than 10,500 crimes amid concerns over rising hate crime.
West Mercia Police have asked anyone with any information on the incident to contact them on 101, quoting incident number 289s of 2 September 2019.
Alternatively, anyone with information can call Crimestoppers anonymously on 0800 555 111.
Agencies contributed to this report