Mr Yeltsin sacked the former defence minister Igor Rodionov on Thursday in a staged dressing down, shown on television, in which he said he was fed up with the lack of progress on military reform.
General Sergeyev was the commander of the Strategic Nuclear Forces - 150,000 strong, including land-based missiles and navy and air force units - and is therefore a logical choice to replace him. As Russia's conventional forces have crumbled away, the Strategic Nuclear Forces, the most efficient, have remained as the backbone of Russia's defence. But it is uncertain whether General Sergeyev will be able to do any better.
Mr Yeltsin also sacked Mr Rodionov's number two, the Chief of the General Staff, General Viktor Samsonov, and replaced him with General Anatoly Kvashnin.
In spite of its appalling economic ills, the Russian Defence Ministry still has armed forces of 1.7 million, and, with armed soldiers belonging to 27 other ministries, including the Interior Ministry, and the Security Ministry, a total of 3.2 million men under arms. Western experts calculate that Russia can afford half a million men under arms in total - just twice the size of the British Armed forces. The strategic nuclear forces, which have been relatively free of corruption and disintegration, are seen as a model and starting point for the rebirth of Russia's armed forces.
"Yeltsin was using Rodionov as a scapegoat", argues Professor Sasha Kennaway of London University and the Conflict Studies Centre at Sandhurst. "My personal view is that the senior military have no intention of downsizing the armed forces. They have been using Nato enlargement as a bogeyman."
It is also understood that the Russians have just closed the research institute which was investigating how to restructure military industry on commercial lines.
General Rodionov recently admitted that Russia still had 1.7 million in the armed forces. Previously, the Russian defence ministry had said it would reduce personnel to 1.25 million. One third of those are officers - twice as many officers to men as in the British forces. But in some units, the balance is more like one-to-one.
When the President fired Mr Rodionov, he declared: "The soldier is losing weight while the general is getting fatter". Military prosecutors say about 20 generals and 100 colonels are being investigated for corruption. During Mr Rodionov's term as defence minister, the situation appears to have got worse, with junior officers driving taxis and young conscripts begging on the Moscow streets. The problem is that paying men off and attracting well-motivated professional soldiers and officers costs money, and there is none.
Like many Russian generals, General Sergeyev has taken an academic approach to his profession, and is regarded as an intellectual. Married with one son, he lists sport and classical literature as recreations. His career started in the navy, but in 1961 he joined the newly formed strategic missile forces, which Nikita Khrushchev had made into a separate service.