The Mayor of London also announced a new alliance of charities in order to set up a single donation point for Londoners.
Work has also begun on a “rapid response” hub to help ensure first-time rough sleepers never have to sleep out for a second night.
Mr Khan said: “We’ve already started to make progress in London, but it’s still shocking that so many people in our city feel they have no choice but to sleep on the streets.
“We must continue to take action because one person sleeping rough is always one too many.”
The Mayor said that, under rules inherited from his predecessor Boris Johnson, shelters only opened when three consecutive days of freezing temperatures were forecast.
Now agreement has been reached with the 33 London boroughs for their shelters to be available on the first day of sub-zero temperatures.
The number of rough sleepers in London doubled from 3,975 in 2010-11 to more than 8,000 in 2015-16, but that rise has now been “effectively halted”, City Hall said.
Meanwhile, Mr Khan announced the bringing together of 18 experienced charities to form a coalition, called the London Homeless Charities Group.
It would provide an easily accessible single donation point, with all funds generated going equally to all 18 charities to direct vital services for rough sleepers where they are needed most. Donations can be made at www.london.gov.uk/help-rough-sleepers
“I know there are millions of Londoners who want to do something to help rough sleepers get off the streets for good,” Mr Khan added.
“As Mayor, I am investing in services right across the capital and doing everything within my power to tackle this issue.”
Mr Khan is launching the campaign on a visit to start work on the new “No Second Night Out” permanent hub in Hackney, to open early next year.
Work on a second hub in Lewisham, including short-term accommodation for people who cannot be helped immediately, will begin in the spring.
In the Commons on Wednesday, the Labour leader told the Prime Minister: “Homelessness is up by 50 per cent and rough sleeping has doubled. Homelessness and rough sleeping have risen every single year since 2010.”
In reply, Ms May argued the Government was “putting in place several projects that will deal with the issue of rough sleeping”.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies