Apart from minute contributions to global warming from our exhausts, the only time we effectively impose costs on our fellow citizens or other road users is when we drive into Oxford (as rarely as possible) or London (never, if we can help it). Higher petrol prices or vehicle taxes would merely increase our costs without contributing in the slightest to the real problems of urban and motorway congestion. You are right to support targeted measures such as congestion pricing or motorway tolls.
I hope that the urban perspective on road transport will not continue to dominate discussions. Blanket penalties on car users risk making rural life less viable. The result will be more pressure on the cities - not, I imagine, what any of us want.