But confidence had been declining ahead of the vote, the survey showed, falling 11 points from the second to the third quarters of the year.
Stephen Ibbotson, ICAEW director of business, said that businesses had reacted in the way you would expect them to.
"Time will help rebuild confidence as they adapt and understand the environment, bearing in mind that the UK will still be in Europe for at least two years. Companies need to make the most of any future opportunities that may arise and we will be working with Government to ensure the UK is the best place to do business," he said.
Construction, property and transport saw the largest declines in confidence, with manufacturing experiencing a smaller dent in sentiment. However the ICAEW said that these sectors were used to slow growth and had strategies in place to deal with it.
Consumer confidence has also been hit by Brexit. One survey showed that consumer confidence had fallen at its fastest pace in 26 years in the wake of Brexit.
An early GfK survey showed the biggest slump in confidence since 1994.
Business news: In pictures
Show all 13
The European Commission's consumer confidence gauge for Britain also suffered its biggest monthly drop in July since January 1991, hitting its lowest level since 2013.
UK GDP growth picked up to 0.6 per cent in the second quarter of the year, according to the Office for National Statitics.
But City of London economists said this was likely to prove a “last hurrah” ahead of a sharp slowdown and possible recession in the second half of the year.
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies