The Cricket South Africa chief executive, Gerald Majola, said: "It is very serious and if it continues, yes, we would look very seriously about whether we return."
Makhaya Ntini, Garnett Kruger, Herschelle Gibbs and Ashwell Prince, who are black, were targets of racial abuse during the first Test in Perth last month.
"It is not right when a country has a history like ours," Majola said. "The sad thing is, it has continued around Australia. It hasn't just been limited to one state or one city."
Cricket Australia's chief executive, James Sutherland, said a crackdown on racist fans was planned. "These types of people are not wanted at cricket. If there is a cost-effective method of keeping them out, we will certainly look into it."
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments