Brindabella had just taken over the lead from Morning Glory when her mast snapped near its base as the fleet sailed into a southerly front with winds of up to 40 knots and three-metre seas off Sydney's Botany Bay, a race spokesman said.
Three hours earlier Brindabella, Morning Glory, Hong Kong pocket maxi Exile and Australian maxi Amazon - all among the favourites to win the race - had been in a group of 25 yachts forced to restart after prematurely breaking the start line in light northerly winds in Sydney Harbour.
Syd Fischer, sailing in his 27th Sydney-to-Hobart race, led briefly but was forced to retire when his 50-footer Ragamuffin tore its sails.
George Steel's 43-footer Quest was forced out with a bent mast. Quest and Ragamuffin were members of the Australian team which won this year's Kenwood Cup regatta in Hawaii.
Others to withdraw were Australian yachts Prowler, Innkeeper and Drake's Progress. Ninety Seven withdrew after a collision at the start left her with a hole in her bow.
Race officials said Morning Glory, owned by a German industrialist, Hasso Plattner, led five hours after the start from the Australian yachts Sydney, BZW Challenge and Condor.
Sydney, designed and helmed by Australian America's Cup veteran Iain Murray, was earlier the first yacht out of Sydney Harbour after Exile was recalled by race officials and suffered a 20-minute penalty for sailing on after it broke the start. The crews aboard Morning Glory, Brindabella and Amazon all rounded the fleet to start again without being penalised. Officials said Exile later caught up with the rest of the fleet.
Raphael Dinelli, a French yachtsman in trouble in freak seas in the roaring 40s in a solo round-the-world race, has boarded a life raft dumped by an Australian air force rescue plane.