Lieutenant Commander Matthew Kroll said nine people were still missing following the pre-dawn fire.
Rescuers initially recovered four bodies just off Santa Cruz Island and 16 others were pulled from the water later in the day, while five have been found but not recovered due to unsafe conditions under the boat, the Coast Guard said.
Five of the six crew members escaped as flames engulfed the Conception by jumping into an inflatable boat they steered to a nearby vessel, the Grape Escape.
The 75ft (23m) Conception caught fire off the north coast of Santa Cruz Island at about 3.15am (11.15am UK time) on Monday.
“We should all be prepared to move into the worst outcome,” Coast Guard Captain Monica Rochester said at a news conference, as authorities searched for the missing people.
Mr Kroll said the four bodies that were initially recovered had injuries consistent with drowning.
Investigators have not yet determined the cause of the fire.
Bob and Shirley Hansen, the owners of the Grape Escape, told The New York Times that they were asleep when they heard banging on the side of their fishing vessel at about 3.30am and discovered the crew members.
“When we looked out, the other boat was totally engulfed in flames, from stem to stern,” Mr Hansen said.
“I could see the fire coming through holes on the side of the boat. There were these explosions every few beats. You can’t prepare yourself for that. It was horrendous.
“The fire was too big, there was absolutely nothing we could do.”
The couple added that the crew members told them they fled when the fire grew out of control.
The Conception was owned by Truth Aquatics, a Santa Barbara-based company founded in 1974.
Coast Guard records show inspections of the boat conducted last February and in August 2018 found no deficiencies. Earlier inspections found some safety violations related to fire safety.
A 2016 inspection resulted in owners replacing the heat detector in the galley and one in 2014 cited a leaky fire hose.
Records show all safety violations from the last five years were quickly addressed by the boat’s owners.
“It’s inconceivable that with all the safety regulations we have in place today, a fire on a boat can lead to the loss of life we saw this morning,” she said in a statement.
“We need to understand exactly how the crew was trained and, if they were awake and above deck, why they were unable to alert or help rescue passengers.”
Truth Aquatics referred questions about the accident to a joint media centre.
“This is still an ongoing search and rescue,” it said.
Agencies contributed to this report
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies