Mr Colby, 76, who led the agency through some of its darkest hours under the White House reigns of Richard Nixon and Gerald Ford, went missing on Saturday evening.
He had taken to the waters in a canoe at his weekend home in the tiny community of Rock Point, where the Wicomico joins the Potomac river some 40 miles south of here.
All the signs point to him drowning, probably when his boat capsized in rough waters whipped up by the strong winds which were gusting at the time. The canoe has been recovered and searchers are now concentrating their efforts on Mr Colby's favourite routes, shown to them by his wife, Sally Shelton Colby, who was in Texas when the hunt for her husband began.
Alerted to his disappearance, neighbours found an unfinished meal and a glass of wine on the kitchen table of the yellow-painted home. A computer and other appliances were left on, suggesting that Mr Colby planned to return soon.
In a last phone call to his wife, on Saturday afternoon, he complained of feeling slightly unwell, but even so, enough peculiarities persist for police not to exclude foul play.
As the agency's director between 1973 and 1976, Mr Colby made more than his share of enemies both outside and within its ranks. He laid bare agency secrets to congressional investigators probing alleged CIA wrongdoing - involving secret assassination plots abroad and illegal wire-tapping of journalists and others within the US - all in the impassioned atmosphere of the unfolding Watergate scandal.
With his insistence that a clean-up was the way forward, Mr Colby earned widespread dislike in the CIA's upper echelons as he violated the agency's unwritten code of silence and forced out several barons of its bureaucracy who operated their own entrenched fiefdoms, beyond the director's control.
For all the controversy which surrounded him, the former wartime agent never doubted he was doing the right thing. "He had the most difficult job of any CIA chief in history, and did it extremely well, " his successor, Admiral Stansfield Turner, said earlier this week.
At Rock Point, Mr Colby lived a private, almost reclusive existence. The few residents who knew him said he was an excellent boatsman who would not take unwarranted risks with the weather. He invariably had a lifejacket with him.
Drowning was the presumed cause of death, said Fred Davis, the local county sheriff, "but we're not ruling out foul play - we never rule out foul play until we've found the body and perform an autopsy".