Only the foolhardy would dare underestimate Sir James. This is the man who in 1987 sold most of his companies - just before the Wall Street crash that wiped out many other businessmen. Today he is the eighth-richest person in the UK, worth around pounds 1.2bn.
And now he has brought his high-octane brain and well-honed instincts to bear upon the world of politics. Two years ago in France he formed his own party, L'Autre Europe, and won 14 per cent of the vote in elections for the European Parliament. This week, his newly formed Referendum Party - which has a single goal: to force the Government to hold a referendum on whether the UK should pull out of the European Union - has sent the Tory high command into a spin.
There is some cause for concern. An anti-EU candidate in the Staffordshire South East by-election this month won more than 10 per cent of the Tory party's total votes. If Goldsmith's party - on which he has vowed to spend pounds 20m - replicated that at the general election, where it plans to field some 500 candidates, the Conservatives could lose 30 seats.
Sir James's party is gaining momentum. A former Tory party treasurer, Lord McAlpine (a key figure in the pro-Europe campaign in the 1975 referendum on Britain's membership of the Common Market), has pledged support. The right-wing challenger to the Prime Minister in last year's leadership election, John Redwood, has engaged in a public gavotte with the billionaire this week. And the Sun this week joined the referendum camp with a full- page editorial that parroted the Goldsmith line.
Mindful of the impact that businessmen like Ross Perot and Silvio Berlusconi have had elsewhere - the Italian business magnate became prime minister and the Texas billionaire took a big enough share of the vote to help Bill Clinton remove Mr Bush from the White House - the Government has been forced to respond. After scorn from Lord Archer was brushed to one side, John Major on Wednesday was forced to attack Sir James directly for "living in cloud-cuckoo-land".
Yet establishing how real a threat Goldsmith might pose is far from easy. Some reports at the weekend suggested that the only support for the party came from a "Dad's Army" of elderly xenophobes, blimpish ex-army officers, and other assorted Tufton Buftons. The Goldsmith camp has decided to keep mum on the identity of its candidates, either because that portrait is all too accurate or because it feels that media interest is peaking too early in a campaign that still has a long way to run.
The party's chief spokesman, the former Tory Central Office press officer Michael Gunton, insists that it is the latter. He promises there will be quite a stir when the party unveils its full list of candidates. So far it has announced only one, Professor Alan Walters, whose persistent advice to Prime Minister Margaret Thatcher against Britain's entry into a single currency led to the resignation of Nigel Lawson as Chancellor of the Exchequer. (Walters is to stand for the Referendum Party against Kenneth Clarke, the Chancellor, in his Nottingham seat in the election.)
But the party claims to have recruited 400 of the candidates it needs to challenge every MP who is not committed to a referendum. And it claims it is putting in place the machinery to support them, which means supporters' groups to help put up posters, deliver leaflets.
The idea for a referendum on Europe was first hatched seven years ago over a bowl of pasta in the home of Christopher Monckton, a former member of Margaret Thatcher's policy unit. Among those around the table in the tiny living room dominated by a life-size portrait of Monckton dressed as a Knight of the Roman Catholic order of the Knights of Malta, was Patrick Robertson, the enfant terrible founder of the Bruges Group. It was a good wheeze but not timely, they decided.
Four years later Robertson recalled the idea and, dressing it up in grander regalia, flogged the idea of a full-blown referendum party to a powerful backer: Sir James.
The Referendum Party was founded in November 1994 with one member of staff and two secretaries in a small office in Hammersmith. Two years on it has offices on three floors, 20 members of staff, 20,000 supporters and is about to move to Westminster. It has staff specialising in candidate recruitment, research and administration (though unlike most parties it does not need a fund-raising department.) Its main preoccupation is selecting 100 or so more candidates for the election. It estimates that it will not need to stand against 76 Labour MPs, 47 Conservatives and most Lib Dems, who are already committed to pressing for a referendum.
Its candidates have largely been drawn from a database of Euro-sceptics assembled by Monckton from the results of a nationwide telephone poll three years ago. Four full-time interviewers have been touring the country meeting those who responded affirmatively to a letter sent out to 30,000 potential supporters and weeding out those with extreme or unsuitable views. Those selected have then been given training sessions with Gunton on organisation, campaigning and handling the media.
The party's plan, apparently, is to announce the full list of candidates with fanfare at the time of its first public conference, which is timed to take place on 18-19 October, just after the Conservative Party Conference.
What is already clear, however, is that the motives of the motley bunch that has gathered around the party are extraordinarily varied. Yet they are all playing for high stakes. Goldsmith himself - who may fight Euro- phile minister Ian Taylor in Esher - does not seem motivated by power, as were Perot and Berlusconi, but by a consuming conviction that British politicians are in the process of surrendering utterly to an unelected dictatorship of Euro-bureaucrats. Others see in the Europe debate a convenient battleground on which to fight over domestic spoils, particularly the future of the Tory party.
Christopher Monckton concedes that the Referendum Party might hand a victory to Labour or turn a Labour victory into a landslide. "It could make Conservative Party unelectable for a generation. It could destroy the party for ever," he admits. But in the split that would follow in a defeated party "those who have no concern with retaining our democracy, the Heseltines and Clarkes who have always been corporatists, will go one way and the vast majority of the Tory faithful will go the other - which of the two factions ends up calling itself the Conservative Party time will tell."
It may all turn out to be the obsession of a handful of far-right Conservatives who are losing touch with reality even more swiftly than the rest of their colleagues. But it may just turn out to be the dominant issue of the forthcoming election. The BSE crisis, the predations of Spanish fishermen, and controversial rulings by the European Court of Justice may put electoral flesh on an issue that has hitherto been the preserve only of policy wonks. Only time, as Monckton says, will tell. But make no mistake, they are playing for high stakes indeed.
The team that aims to take us out of Europe
Sir James Goldsmith, the founder and the brain of the RP. Retired from active business in 1990 after selling out just before the 1987 stock market crash. The Eton-educated 63-year-old spends much of his time in France and Mexico. Active in French politics already as European Member of Parliament.
Judith Duckworth, director of campaigning, is the RP's chief administrator. She was formerly the Conservative Party agent for Elmet in Yorkshire, whose MP Spencer Batiste backed Michael Heseltine in the 1990 leadership election.
Michael Gunton, chief press officer, is also responsible for training RP candidates in campaigning and handling the media. His favoured technique centres around "10 reasons why we need a referendum". He was a Treasury press officer when Nigel Lawson was Chancellor. Later joined Conservative Central Office to complaints from Labour at top civil servants turning political. Quit the Tories over Europe.
Patrick Robertson, personal PR to Goldsmith. Age 27. Boy wonder founder of the Bruges Group (hon. pres. - M. Thatcher) which he ran from his college rooms at Oxford. But its big-name supporters resigned and withdrew funding after a series of blunders and acrimony over the group's money.
Later advised Neil Hamilton to ride out allegations of improper hospitality from Mohammed al-Fayed; Hamilton did and Major sacked him. Then wrote confidential memo to Jonathan Aitken suggesting that one more embarrassment could finish his career - and then promptly faxed it to a TV producer by mistake. Now a director of Taskforce Communications, which handled PR for wedding of Imran Khan and Jemima Goldsmith.
Additional research by Henrietta Norton
Businessmen in politics: how Goldsmith compares
Berlusconi, Forza Italia
Organisation: devoted to the leader; party meetings are rare and are stage-managed to allow maximum homage to be paid to the great man. Glitzy fundraising dinners and TV promotions are more Forza's style.
Launched: 10 January, 1994
Membership: the party has never made much effort to recruit; there is a network of national party clubs, but Berlusconi feels that a strong formal membership is not necessary to solicit votes.
Candidates: in the recent general election, the party fielded candidates in all constituencies except those in which they had agreements with other parties.
Finance: the party's finances seem in good health. Berlusconi is still embroiled in a court case, which led to the fall of the coalition government he headed in 1994, over allegations of bribery. There are limits on election campaign spending, but Forza's media blitz appears to many to exceed them.
Influence: Berlusconi has fingers in many pies - insurance, supermarkets, real estate, and television. He controls three TV stations, and his advertising agency, Publitalia, produces 90 per cent of all the TV advertising.
Political impact: was a vital force in shaking up the old corrupt centre- right establishment in Italian politics with reformist agenda. Has a broader political ambition than Goldsmith's single-issue campaign.
Ross Perot, Reform Party
Organisation: although very much Mr Perot's baby, the Reform Party is starting to acquire some life of its own. To this end, a nationwide modem- linked "cyber conference" in September will choose a party leader. In the unlikely event that this is not Mr Perot, he has pledged to step aside.
Launched: the Reform Party evolved late last year from United We Stand America, the group that supported Perot's 1992 presidential bid.
Membership: a network of loyal groups are slowly collecting signatures to allow the party to campaign in all states.
Candidates: all effort is devoted to the presidential race.
Finance: Perot, a computer billionaire, says he wants to cut the moneymen out of American politics by spending only his own money. In the last campaign, Perot lightened his wallet to the tune of about $65m.
Influence: Perot was at one stage ahead of both Bush and Clinton in the 1992 election race, but currently polls around 12 per cent. His shadow haunts the current presidential race (election in November) because Bob Dole, the Republican candidate, is so weak.
Political position: Perot is more like Goldsmith in the sense that he has a simple anti-Washington message, which focuses on balancing the federal budget. Shares with Goldsmith an antagonism to free trade.
BEN SUMMERSReuse content