The 41-year-old got into difficulties eight miles off the coast of Dove - more than a third of the way into the 21-mile crossing to France.
Ted Ingham, a watch officer for Dover Coastguards, said the service received a call for help at 11am from her escort boat, the Mary Mayne, which is used by the Channel Swimmers' Association.
He said: "We got a call saying she had got into difficulty. Dover lifeboat got to her in 20 minutes and brought her back to shore. An ambulance was waiting at the docks for her, but paramedics pronounced her dead on arrival at the shore."
The woman was one of four swimmers who set off separately from Dover. The other three swimmers were still attempting to complete the challenge, but were said to be fine.
A post mortem is due to be carried out today.