Services on the east London line will run round-the-clock on Friday and Saturday nights from December.
The new timetable will allow passengers to connect with Night Tube services, which launched on the London Underground in August last year, and will take 24-hour trains to nightlife hotspots in the east of the capital.
Overground trains stop shortly after midnight under the current schedule.
Mr Khan said: "Our Night Tube services have been a real success right across the capital, so I'm delighted to announce that we'll be bringing the Night Overground to the east London line later this year.
"It will provide huge benefits to Londoners and visitors to our city, helping those working hard through the night and all those out enjoying everything London has to offer, as well as creating jobs and boosting our economy."
All-night Overground services will initially run between New Cross Gate and Dalston Junction before extending to Highbury and Islington next year.
Only a small proportion of the network is included in the project as most Overground trains operate on Network Rail tracks, which are also used by other train companies, freight and maintenance services.
The link between New Cross Gate and Highbury and Islington is managed by Transport for London (TfL).
TfL director of London rail Jonathan Fox said: "The east London route is one of the most popular parts of the London Overground network, particularly late in the evening.
"Expanding night services to this key part of London Overground will make it easier for customers to enjoy the vibrant night-time culture that east London has to offer, supporting the economy in this much-loved area of the capital.
"The Night Tube has already provided a boost to our economy and supported thousands of permanent jobs. We hope the Night Overground will build on this success even further."
The announcement was welcomed by the Night Time Industries Association, which represents bars and nightclubs.
Chairman Alan Miller said he was "delighted", adding: "London is making significant strides to become a smart, future-oriented 24-hour city in which transport is a vital component."
The London Overground launched 10 years ago after TfL took over a clutch of suburban rail routes from Silverlink.
Arriva won the £1.5bn contract to run Overground services last year.
All-night Tube services currently operate on the Victoria, Piccadilly, Northern, Jubilee and Central lines.
Parts of the Metropolitan, Circle, District and Hammersmith & City lines are set to be added once modernisation programmes are completed in 2023.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies