Three minutes before O'Leary touched down for Quins' second try, the Ireland stand-off, David Humphries, could have snatched a remarkable success. But his 75th-minute penalty drifted agonisingly wide from 35 metres out as Ulster trailed Harlequins by a single point going into a tense final quarter.
Humphries looked on in despair as his late penalty veered off, and it was left to O'Leary to clinch a 21-15 victory when he was on the end on a rare threequarter line movement to charge over in the right-hand corner.
Leicester were also pushed to the final quarter - at Lansdowne Road by Leinster - before the Tigers made the game safe to complete two victories from two matches in the European Cup.
Leinster, trailing only 13-10 at that point, laid siege to the visitors' line but failed to make their territorial advantage pay and, once Leicester had rebuffed the onslaught, they piled on 14 points in the closing stages for a comfortable win.
"We failed to take our chances," the Leinster coach, Ciaran Callan, lamented. "We had them under great pressure, but failed to make the breakthrough when it might have counted."
Pontypridd were indebted to the replacement Simon Enoch, whose hat-trick of tries in the second half secured their second European Cup victory against a spirited Edinburgh side.
Elsewhere, the Scots enjoyed success over a Welsh club as Scottish Borders recovered from their comprehensive defeat by Pau to achieve a well-merited win against Llanelli.
All their points came from the diminutive left-wing Gary Parker, who kicked eight penalty goals as an indisciplined Llanelli were punished for persistent foul play. The Llanelli captain, Robin McBryde, ended the game with a yellow card after throwing a punch at the Borders' prop, Steve Ferguson.