After Johnson's death, Barber inherited most of his fortune and, with his wife and family, moved to Lichfield, his famous master's native city in Staffordshire.
Sadly, he seems to have been unable to handle his new fortunes. Attempts to set up schools in Lichfield and the surrounding area proved unsuccessful and he died in Stafford Infirmary in 1801. His descendants still live in Staffordshire.
Samuel Johnson Museum