The story of how this Anglo-French tax exile has managed to push his way into public life reveals much about the anatomy of power in Britain.
The first explanation, most obviously, is money. For a man who is said to have made pounds 800m in one deal, the sort of sums spent on politics in Britain hardly ring his register. He has let it be known that he is prepared to spend pounds 20m on his single-issue campaign for a referendum on Europe between now and the general election. That puts the Referendum Party, of which he is founder, leader and ideologist, on the same sort of footing as the Tory and Labour parties - and heavily outspending the Liberal Democrats.
When it is suggested that Sir James's threat to stand candidates against Tory MPs is not serious, Jeffrey Archer, the Tory peer and former deputy party chairman, warns his friends: "James Goldsmith is a billionaire and he doesn't give a damn."
Sir James has also found a big issue: Europe. The reasons why he is bothered about Europe are not the usual ones, and his train of logic is sometimes difficult to follow. The root of his obsession is his concern about the effects of global capitalism.
As a supremely successful global capitalist himself, Sir James knows what he is talking about. It might be said that he is so keen to preserve national identity that he has two of them: French and British. He certainly has little in common with the yeoman English patriots who form the backbone of Bill Cash's troops on the Tory backbenches. Sir James has two houses in Britain, one in Paris and a hacienda in Mexico. He is a member of the European Parliament for a French constituency and is father-in-law to Imran Khan, who may or may not want to be prime minister of Pakistan.
Sir James is animated by the logic of the world turning into a single economic system in which the peoples of Western Europe will find themselves competing with the very lowest-wage countries. And his answer to that means dismantling the European Union and creating a different kind of trading block, protected from competition in world markets.
The first part of that programme is enough to excite the Euro-sceptics, and Sir James is canny enough to focus on one thing at a time - a referendum on the terms of Britain's membership of the EU is an issue that can unite the largest coalition of support and cause maximum creative chaos in British politics.
The third form of leverage Sir James has on the Tory Party is social. He is well-connected with the Thatcherite wing. He is friends with the buccaneering capitalists who backed Margaret Thatcher, because they thought she understood the free market and because she was not the Tory establishment.
In the Seventies, Goldsmith, along with Lord Hanson, Gordon White and Jim Slater, stalked the City, making money, going to the same clubs and being seen with the same glamorous women.
Most of his friends are outsiders in some way, but they all have money. Jacob Rothschild and Mark Weinburg are members of the City establishment, although Kerry Packer, the Australian former media magnate, and John Aspinall, the zoologist and casino owner, are flamboyant mavericks.
"You could imagine the plans for the Referendum Party being hatched in the back room of Aspinall's casino," says one observer. It is the sort of grand and bizarre scheme that rich men would enjoy plotting.
Sir James Goldsmith's salon reflects his varied interests and influences. A key figure is the Marchioness of Worcester, a former actress and model who has elevated green talking-shops from squats and coffee bars to the grounds of her vast Gloucestershire estate. Known in environmental circles as plain Tracy Worcester, the Marchioness has long been a close friend of Sir James.
At her London townhouse, Cabinet ministers and Whitehall mandarins have tea with green activists. Guests have included Sir James and his brother Teddy, along with William Waldegrave and Brian Mawhinney, Charles Secret, of Friends of the Earth, and Sir Crispin Tickell, former ambassador to the UN and chairman of Earthwatch Europe.
Prior to the Referendum Party, Sir James was a passionate ecologist. His financial assistance to the Ecological Foundation kept it afloat in the early Seventies and helped Teddy to found the Ecologist Magazine. In 1976, he was awarded a knighthood for "services to export and ecology".
One of his closest friends is John Aspinall. The pair met in 1949 while Aspinall was at Oxford. Later, they became partners in the Aspinall gaming clubs, which help to pay the pounds 4m a year cost of running Aspinall's zoos in Kent.
Geoffrey Wansell, Goldsmith's biographer, recalls that Aspinall was a flamboyant Oxford undergraduate, who organised gambling parties.
"One of the regular players at Aspinall's tables was Teddy Goldsmith, and it was only to be expected that he would bring along his younger brother Jimmy," he recalls. Then aged 16, young Jimmy made an impression on the table of gamblers by losing almost pounds 4,000, says Wansell.
Central to Goldsmith's circle of influence is his personal spin doctor, Patrick Robertson. At the age of 27, the founder of the ill-fated Bruges Group has already made a name for himself in right-wing circles. He has been credited with dreaming up the Referendum Party, a role he denies. "I do not work for the Referendum Party, but for Sir James Goldsmith personally," he said yesterday.
Other central figures in the Referendum Party include Judith Duckworth, a former Conservative Party agent who has recently been advising the New Democratic Party in Romania. Cheque-signing is the prerogative of Charles Filmer, a director of one of Goldsmith's companies and of the Referendum Party Ltd. Another great friend is Jacob Rothschild, the investment banker who also chairs the National Heritage Memorial Fund.
In 1984, Rothschild, Goldsmith and the Australian tycoon Kerry Packer formed a consortium to attempt to take over the St Regis Corporation, remaining close confidants ever since. Rothschild has said of Goldsmith: "Jimmy is completely international ... People aren't used to dealing with someone who is as independent and outspoken as he is. He doesn't compromise, he doesn't suck up, he isn't accommodating to the second-rate, and he doesn't like people who are hide-bound, or do nothing of interest."
And then he has access to the next part of the "amplification machine" - the media. He is on close personal terms with important media figures, especially - but not exclusively - in the Euro-sceptic right-wing press. There is a Now magazine diaspora of journalists who worked on Goldsmith's failed glossy right-wing weekly during its brief life in the early Eighties, including Frank Johnson, now editor of the Spectator.
Sir James's party invitation lists include Conrad Black, proprietor of the Telegraph, Nigel Dempster, who used to live close to his Richmond mansion, and William Rees-Mogg, the former editor of the Times.
Lord Rees-Mogg once wrote admiringly: "Over the years I have become an experienced observer of charisma; the best indicator is the way in which the public approaches the candidate ... People were coming up to Jimmy Goldsmith in just this way. It was like observing Margaret Thatcher at a Conservative Party conference, or Ronald Reagan in the Republican primaries of 1980."
Sir David Frost's status as a member of the inner circle was also confirmed when he joined Sir James at his palatial Mexico home for a holiday over Easter just before Sir James appeared on his programme.
The honour of an invitation to Mexico has also been extended to Sir Charles Powell, Margaret Thatcher's political secretary. Sir Charles and his wife, Lady Carla, attended both of Sir James's recent big social events - the party at the Ritz two years ago to celebrate his election as a Euro-MP and his wife's 60th birthday, also attended by the Princess of Wales, and the wedding of his daughter Jemima, Diana's friend.
Although Sir James holds unconventional economic views - he is closer to fellow billionaire-politician Ross Perot in opposing free trade and advocating protection - his social connections with right-wing business people intersect with the Thatcherites' anti-Europeanism.
Sir James's social connections with right-wing Tories were advertised by his invitation to the gracious Georgian home of the former Cabinet minister Jonathan Aitken in Lord North Street on Monday night.
Sir James apparently only made one comment in the debate on a presentation by Norman Lamont, the former Chancellor, on the question of whether Britain should pull out of the EU. But he was there, in a private discussion group which included Government ministers.
The Goldsmith-Thatcherite network was pulled together in the European Foundation, the anti-Maastricht think-tank chaired by Bill Cash and supported by "substantial" donations from Sir James. On its advisory board sit Lord McAlpine, the former Tory treasurer who regards Baroness Thatcher as so betrayed that the party might benefit from a spell in opposition; Robin Harris, of Lady Thatcher's policy unit and drafter of her memoirs; and Iain Duncan-Smith, a Euro-sceptic MP close to John Redwood.
The final reason why Sir James has such a hold on the Tory party is obvious - that the Government is massively unpopular. MPs may say they do not believe opinion polls, but you can be sure that the 175 who would lose their seats on present poll ratings are worried.
They take very seriously anything that could reduce their vote at the general election, and Sir James's threat to stand candidates against any MP who is not committed to his idea of a referendum - not just one on a single European currency - is one of the few things they could do something about.
Sir James insists that his initiative is neither left-wing nor right- wing, and it is true that some of his analysis of the evils of globalisation chimes better with the Labour Party's themes. But Labour MPs and candidates do not have to pay attention: they are winning. And besides, the Euro- sceptic theme is more likely to play with disillusioned Tory voters than old-style Labour anti-marketeers. Sir James's candidates only need to win 1.5 per cent of the vote to deprive the Tories of 12 seats.
Sir James may only be playing at politics, but he has the money, the issue and the connections to be a serious player. Some regard his views as dangerous. He certainly lacks neither opinions nor the language in which to express them. In his book The Trap he warned of the consequences of global free trade causing social divisions "deeper than anything ever envisaged by Marx". More recently he described the Government's signing of the Maastricht treaty as "tantamount to treason".
Others take a different view. One Tory MP said yesterday: "He pricks our consciences and shows our intellectual inadequacy."
Additional research by Ros Wynne-Jones.