In his first major speech on the issue, Mervyn King said the IMF was guilty of issuing "ever more bland communiqués and meaningless statements".
He told an audience of leading economists in India: "If the mission of the fund is not examined and the institution revitalised, it could slip into obscurity."
He also criticised political leaders of members of groups such as the G7 - although he went out of his way to praised Gordon Brown, who chairs the IMF's main policy committee.
Mr King's comment will be seen as a contribution to a review launched by the IMF, in September. He told the Indian Council for Research on International Economic Relations: "We have an opportunity to return to first principles and ask some basic questions. Do we need an IMF? Not before time are those questions being asked in the corridors of international meetings which rotate endlessly around the world from one windowless room to another." The IMF was established 60 years ago to provide temporary financial assistance in a world of fixed exchange rates and closed capital markets.
Mr King said the openness of capital accounts meant private capital flows dwarf official flows, making current account deficits irrelevant in terms of financial stability.
"Certainly the fund's remit is unclear," he said. "Its lending activities have waned and its role in the international monetary system is obscure. Belatedly the G7 has started to discuss the future role of the fund."
The Governor laid out a plan to revive the fund by shifting its role to provide analysis on external risks to countries' domestic monetary policy goals.
This would include providing information on countries' balance sheets of all major to underpin an assessment of the risks to the world economy and encouraging them to be more transparent about national policy. He criticised the G7 for using high-profile meetings to discuss issues such as global imbalances without China and India, who both hold vast current account surpluses, at the table. "India and China have to be at the table," he said.
The IMF is based in Washington and no one was available for comment as yesterday was a public holiday in the US.