To Britain and Spain, Gibraltar is becoming an irritating 'stone in the shoe'. But the people on the Rock are standing firm, determined not to crack under the threat of direct rule from Whitehall.

It is hardly surprising the siege mentality is deeply ingrained

Peter Montegriffo looked out from his sixth-floor office over the Bay of Algeciras, which Gibraltarians call the Bay of Gibraltar, his eyes as steely blue as the dancing sea below. His office in a law partnership is in the same building as the NatWest Bank, from whose cash machine I had extracted pounds 100 in crisp sterling notes on the way up.

"We're talking about economic survival," Mr Montegriffo said, explaining the concerns of his fellow Gibraltarians. "An economic gap was left by the departure of the Ministry of Defence. We just do not have a working relationship with Britain that would help us make good that gap, and we face stepped-up pressure from Spain."

Montegriffo's accent is cut-glass English, though a slight lilt - Welsh? Jamaican? - and his Genoese name speak of a Gibraltarian ancestry going back generations. It is a history that 30,000 Gibraltarians claim is unique to the Rock and which they defend with a passion that exasperates Whitehall and infuriates Madrid.

Earlier this month the Foreign Secretary, Douglas Hurd, warned Gibraltar's Chief Minister, Joe Bossano, that the colony must tighten up regulation of its financial services. Gibraltar's off-shore banking industry has allegedly become a centre for the laundering of criminal profits, in particular drugs money. The British government threatened to impose direct rule on Gibraltar, and now that threat has become imminent. To which Mr Bossano has said, quite simply: "We will not play ball."

Main Street, Gibraltar, is a good place to observe the peculiar mix of cultures that contribute to Gibraltar's self-perceived uniqueness - and its fierce desire for independence. The street is a sort of Mediterranean answer to the East End's Brick Lane, though its pace is more languid, its ambience more prosperous: a technicolour parade of merchandise includes silk pyjamas from China, duty-free electronic goods, including electric kettles unobtainable elsewhere on the Iberian peninsula, and exquisite embroidered tablecloths half the price of those in Madrid's knockdown flea market.

The souk-like atmosphere is interrupted every few yards by mock-mahogany pubs playing Seventies disco music and serving the same disgusting food that you would expect in identical establishments in any English garrison town. Here English tourists feel at home, and rosy men in shorts prop up the bar at lunchtime, downing neat J&B (English measures here: a splash of spirit at the bottom of the glass, rather than the Spanish half- tumbler).

All races mingle in Main Street. The Moroccan Muslim women robed from head to foot, the Orthodox Jewish men in homburgs and bushy beards, the scrubbed British matrons with pepper-and-salt curls and bare shoulders. The only missing element is anything Spanish, apart from the language - people swing in and out of Spanish and English within the same sentence. I tried to make a rendezvous for around 2pm to find that my potential lunch date was due back at work at 2.30. No concession here to the fierce Mediterrean sun that would have any Spaniard cowering behind the shutters until 5pm.

The pubs and everything else Gibraltarian grew up to service the British garrison, a tradition that began when British and Dutch troops seized the Rock in 1704 and the Spaniards who lived here fled to the village of San Roque, a few miles round the bay. "After 1704 there were no Spaniards left, basically," says Peter Montegriffo. "Their place was filled by Mediterranean merchants, the Maltese, the Portuguese, the Genoese traders, the Jews who had been banished from Spain came back from North Africa, all to service the garrison. We Gibraltarians are typical intermediaries."

But the squaddies who used to roam down Victualling Office Lane from the Cannon Bar to the Trafalgar Tavern are all gone now, and so are most of the jobs in what used to be the Naval Dockyard, not far from where Nelson's body was brought ashore after the Battle of Trafalgar, reputedly in a barrel of rum. Ten years ago the MoD accounted for 60 per cent of the Rock's economy; now that figure is less than 10 per cent, and still falling. The three armed services operate a joint command; the British Army has withdrawn its resident battalion and been replaced by a small local contingent of the Gibraltar Regiment; the Naval Dockyard has been sold, and all the Navy has left is a jetty and two small craft.

The departure of the forces ripped the heart out of this community of traders, fixers and middlemen and left a big hole to be filled. One possibility was to provide offshore financial services for the EU, now the subject of the dispute with Britain. Day-trip tourism and duty-free shopping, the other staple, depend crucially on Spain's vigilance at the frontier at La Linea; the signs are that Madrid is shortly to tighten the screw and increase the delays of those who wait to cross the border.

There remains the centuries-old tradition of tobacco-trafficking. A 19th- century Spanish couplet from the region goes: "Despite the guards/I have to be a smuggler/I have to sell tobacco/at the door of the barracks". Gibraltar calls it re-export, and some estimate that tobacco duty provides more than a quarter of the Gibraltarian government's revenue. Others call it contraband, and fear that it is a cover for drug-running.

Down on the bay, by the modern marina development of Water Gardens, set slightly apart from the glinting white yachts and pleasure launches, are moored a flotilla of smaller, scruffier craft, the fast motor launches, matt black or dark blue, their windscreens removed, huge motors hanging over the side, idly supervised in the heavy afternoon sun by a couple of lads sitting on the harbourside.

These are the vessels that on clear nights are suspected of "doing tobacco", picking up cartons of Winston cigarettes from Morocco and landing them on Spanish beaches along from the border town of La Linea. Spanish customs authorities complain bitterly at the ever-increasing traffic: 130 million packets of cigarettes and 50 tons of hashish smuggled into Spain with the help of fast boats based in Gibraltar last year, they say.

Joe Holliday, chairman of the Gibraltar chamber of commerce, takes a cautious view of this traditional money-making activity. "We should be tough against drugs, that's unquestionable, but the tobacco launch activity can't be eradicated overnight. A lot of people live off this trade, both here and in Spain. The social implications could be damaging. Britain must accept some responsibility and come to our help." But he believes more should be done to curb fast launch activity.

The Gibraltar authorities take a hear-no-evil, see-no-evil approach. "The fact that a boat leaves Gibraltar and goes to Morocco and loads up with drugs and takes it to Spain may be something we suspect," says Joe Bossano, "but if the drugs are in the boat in Moroccan waters and are in the boat in Spanish waters but when the boat is in Gibraltar waters there are no drugs in it, it is difficult to secure a conviction in Gibraltar."

Since Mr Bossano became Chief Minister in 1988, he has taken an increasingly assertive line with Britain. Mr Bossano, who started out as a factory worker, became a tough negotiator of the Transport and General Workers' Union, ploughing indefatigably through a four-year dispute with Britain in which he won the right of Gibraltarian workers to the same wages as locally employed Britons.

In the late Seventies he often used to visit Transport House in Smith Square, and when Joe - a stooped and shambling figure even then - shuffled into town, the brothers treated him with respect and affection. Today, still only 55, he resembles an embattled old tortoise, but behind those pebble glasses and grizzled whiskers lies a brain like a razor and a will of iron that must make the mandarins in King Charles Street weep with frustration.

He has been the driving force behind the Rock's desire to become an offshore finance centre, which he sees as the only way of plugging the economic hole left by the MoD. Down by the former Naval Dockyard is the Europa Business Centre, Gibraltar's Canary Wharf, where private industries are being wooed with tax breaks. In one of the warehouses, Dennis Matthews, a former teacher, runs a business servicing gambling machines for pubs and clubs on the Rock. He hopes to set up an assembly plant to exploit what he sees as Gibraltar's great asset as an offshore centre. "If I import the components and assemble them, I can export the machines to the EU as a European product." It is "screwdriver", or maquiladora, industry on the Central American model.

Mr Matthews is chairman of the Gibraltar Self-Determination Movement - "Joe's shock troops" in the words of one local commentator. "We Gibraltarians have been here for almost 300 years. Our forefathers lived and died here and there is no way anyone can tell me that this is not our home and this is not our land. We all feel passionately about this," he added, unnecessarily. "It's like saying 'who should decide your future?'. Clearly the answer is 'it's going to be me'. Spain has no right to claim Gibraltar. I think Spain and Britain are in cahoots on this, trying to push us into a corner."

Gibraltar has been subjected to sieges on and off for centuries. Deep into the limestone rock run tunnels that were cut in 1779 to supply artillery against a joint French and Spanish siege that lasted two years and ended when the Gibraltarians won. It is hardly surprising that the fortress mentality is deeply ingrained. "Gibraltar will never be Spanish," insists Mr Montegriffo, "other than by imposition. Gibraltar just is not Spain." Nor, despite the cute clunky street signs, the red pillar boxes and the sterling cash machines, is it Britain.

Probably too small for viable independence, Gibraltar is still able to stymie Britain's attempts to exercise greater control by consent. Spain's continuing sovereignty claim is what the Prime Minister, Felipe Gonzalez, calls "a stone in the shoe" of what was, before the recent halibut war, a harmonious relation with Britain. Neither London nor Madrid has been able to get anywhere in Gibraltar through negotiation. Both are now preparing to try harsher methods.