Abdulhai Walizada, a local police spokesman, said there appeared to be more than one attacker, with a suicide bomber who detonated explosives and another who threw grenades at worshippers.
A further 64 people were reportedly wounded during the attack at the Jawadia Mosque and local authorities said the death toll is expected to rise.
At Herat Main Hospital, Dr Mohammed Rafique Shehrzai said as many as 20 bodies were brought to the hospital in Herat city, the capital of western Herat province.
But Mehdi Hadid, a lawmaker from Herat, who went to the site soon after the explosion, said there could be "at least 100 dead" as he estimated hundreds of people had either been killed or wounded.
The attack took place in the middle of evening prayers when the mosque was packed with about 300 worshippers.
He said he rushed to the mosque after hearing about the explosion and was told the attacker first fired on the private guards outside the mosque before entering.
When inside, he fired on the praying worshippers until his rifle jammed then he blew himself up, Mr Hadid said.
There was no immediate claim of responsibility for the latest attack to hit Afghanistan, which has seen more than 1,700 civilian deaths so far this year.
Afghanistan has traditionally been relatively free of the sectarian violence common in Iraq or Syria but hardline Sunni militants from the local branch of Isis have repeatedly attacked the mainly Shia Hazara minority in the past year.
Herat, which is close to the border with Shia-majority Iran, is considered to be one of the most peaceful cities in Afghanistan.
Additional reporting by agencies
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies