The bronze memorial to the 17th-century slave merchant was pulled down during a Black Lives Matter protest on 7 June, before being dumped in Bristol Harbour.
The Crown Prosecution Service (CPS) said Rhian Graham, aged 29, Milo Ponsford, 25, Jake Skuse, 32, and Sage Willoughby, 21, will appear at Bristol Magistrates Court on 25 January for their first hearing.
Following the protest in June, the controversial statue was recovered from the water by Bristol City Council and assessed to have suffered £3,750 worth of damage.
Avon and Somerset Police decided not to intervene in the protest and made no arrests at the time, but officers subsequently launched an investigation into the incident.
“The Crown Prosecution Service reminds all concerned that criminal proceedings against all four are now active and that they have the right to a fair trial,” a CPS spokesperson said.
“It is extremely important there should be no reporting, commentary or sharing of information online which could in any way prejudice these proceedings.”
The CPS said it had authorised charges following a review of a file of evidence from the Avon and Somerset force.
Although the statue was retrieved, Bristol City Council said it will not be returned to its former position and will instead be exhibited in a museum with the graffiti sprayed on it during the protest left intact.
Commenting on the recovery process, Fran Coles, the council’s conservation and documentations manager, said: “Our main concern is making sure that we can conserve the paint, the graffiti that's on him now, because that's actually become the most fragile part of the sculpture.
“It has become part of the story of the object, of the statue, so our job is to try and retain that as much as possible, while stabilising the statue for the long term.”
Earlier this month, a figure of Darth Vader was placed on the plinth where the statue of Colston formerly stood, in an apparent tribute to David Prowse, the Bristol-born actor who died in November.
Prowse provided the physical performance for the Star Wars villain, while James Earl Jones provided the character’s voice.
Additional reporting by PA