The TV personality, whose real name is Deirdre Kelly, has a 16-year-old daughter and a seven-year-old son, both of whom are mixed race.
Her manager, Barry Thomas, confirmed to the Huffington Post UK that she had relocated from her house on James Turner Street in Birmingham to a new residence in the city.
"It just got a bit silly," he said. "The abuse has got bigger and bigger. She's been getting quite a lot of hateful stuff on Twitter, and it's clearly going to get a lot worse.
"There was the comment online about her kids being liquorice allsorts, which is just vile and not at all acceptable. So, she's decided the family needs a bit of normality."
Her Rastafarian boyfriend has also been the target of online abuse.
Thomas went on to tell the publication that while White Dee is currently staying in a "secret bolthole", she doesn’t plan to leave James Turner Street permanently.
"I did try and get her to move five months ago, and told her it would only get worse," he said. "But she didn't want to - she loves the street. Dee is Dee, she loves sitting on that front wall having a fag and drinking a cup of tea, talking to people."
Her house had also been subjected to vandalism by "TV tourists", he said.
"We've had people throw eggs at the house, and there's been a brick thrown on one occasion when she was here doing an interview.
"It's just the small proportion of nut-jobs and idiots – 90 per cent of people love her."
White Dee came fifth place on Celebrity Big Brother during the grand final on Friday (12 September).
Meanwhile, Benefits Street will return to Channel 4 in 2015 with a new location and six more episodes.
Register for free to continue reading
Registration is a free and easy way to support our truly independent journalism
By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists
Already have an account? sign in
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies