The Latvian was a 6-3 7-6 (8/6) winner against Ukrainian fourth seed Elina Svitolina, stumbling over the line after at one point having looked set to coast through.
Ostapenko, who turned 20 only two days before her Roland Garros triumph last month, needed eight match points to get over the line in her Court 12 tussle.
Five came and went when she served for the match at 5-3, and as her game became ragged Ostapenko dropped serve again to trail 6-5, in danger of being pushed to a decider.
But Ostapenko struck back, pouncing on Svitolina's serve and forcing the tie-break that had more tension in store.
Ostapenko opened up a 6-4 lead, giving her two further match points, but from the first she drove a return into the net before Svitolina then forced an error.
The eighth match point arrived thanks to an angled backhand that landed tight to the line, and Ostapenko clinched victory when Svitolina netted.
"Come on!" shouted 13th seed Ostapenko, later leaving the court all smiles, posing for photos with a host of camera-phone carrying fans.
Russian Kuznetsova first reached that stage in 2003, repeating the run in 2005 and 2007, and a fourth quarter-final appearance for the 32-year-old seventh seed was secured with a clinical performance against Polish player Radwanska, the 2012 runner-up.
Slovakian Magdalena Rybarikova extended her career-best grand slam performance, overcoming Croatian Petra Martic 6-4 2-6 6-3 on Court 18.
Until this year's championships, Rybarikova had paid nine visits to Wimbledon and suffered eight first-round defeats over that time, with the exception a third-round run two years ago.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies