Craig Brown believes opposition to a Great Britain football team is down to a simple lack of trust in Fifa.
A row broke out after yesterday's "historic" announcement by the British Olympic Association that players from all home nations will be eligible for selection for Team GB at the London 2012 Games.
The BOA's statement came as a shock to the Scottish, Welsh and Northern Irish FAs who all denied being part of any agreement.
SFA chief executive Stewart Regan reiterated again his fears that a combined British team could lead in time to the four home nations eventually coming under one umbrella - even though he acknowledged that Fifa are prepared to give assurances that there would not be any future repercussions if Scotland do participate.
Former Scotland boss Brown backed Regan and claimed their fears were corroborated by the late David Will, former SFA president of the SFA and vice-president of Fifa.
"It's quite a harsh thing to say but I do think there is a lack of trust in the Fifa organisation," Brown told Talksport.
"Over the years there has been suspicion of the motives of Fifa and of course they would like to have one vote for Britain instead of four votes.
"The main argument is that Scotland would lose its identity.
"I think the feelings are that Fifa are desperate to combine the home nations and make it a Great Britain team and there would be no separate England, Wales, Scotland or Northern Ireland teams.
"When this issue first arose a few years ago the late David Will, who was former president of the SFA and vice president of Fifa, wrote to me and I still have the letter.
"He more or less said you could trust Fifa at the moment but successive committees and maybe a new president eventually, could change the whole thing.
"So it wouldn't be a binding commitment, it would be a temporary one.
"And I do genuinely believe that that's the case."
"I'm absolutely sure it's not an Olympic issue, it's a Fifa issue," added the Aberdeen boss.
"It's fear of what will happen if we agree to amalgamate for that one-off tournament.
"The next thing will be Great Britain for the World Cup and European Championships.
"We are very patriotic in this country, as you know, and everyone wants to remain Scottish and support a Scottish team.
"We want to keep our autonomy and even a less successful Scotland team is more worthy of support in Scotland than a combined Great Britain team."