Police were called Co-op on Washwood Heath Road in the Ward End area of Birmingham after the man entered the store at around 4pm brandishing a knife and a pair of handcuffs. The manager of the store was taken hostage and handcuffed, with the man holding a knife to his back.
The manager suffered a stab wounds and was taken to hospital. His injuries were not believed to be life-threatening.
A police spokesman said armed officers were called to the scene. "The area was sealed off as negotiators tried to persuade the man to give himself up," he said. "As a result of the man's actions he was shot and died as a result.
"Two of the police officers who attended were armed officers from the force's Armed Response Vehicle which routinely patrols the city."
Witnesses said the dead man was known locally as a mental patient. Tahir Aslam, 25, said he saw a neighbour, "David" in the street shouting and making threats to his parents.
Mr Aslam, a railway worker, said: "He must have been at his parents because his mother came out screaming.
"His parents are both elderly and the father has arthritis. The woman was screaming, `Please, please, help me. Someone call the police.'
"David came out and was swearing and telling everyone to mind their own business before he grabbed his mother."
Mr Aslam said he and three friends went to help the elderly woman and pulled her away.
He said: "David kept on shouting that he would get a contract killing out on us and his father came across and said this wasn't the first occasion something like this had happened.
"We managed to get the two of them away from him and he was shouting about having knives and guns."