Mr Carroll had been visiting an Iraqi family in Baghdad yesterday to report on their reaction while watching the trial of Saddam Hussein on television. He had broadcast a live report on the trial to RTE, the Irish broadcaster, and on the Romanian news channel Realitatea TV.
Mr Carroll, 33, an Irish citizen, has in Iraq been nine months for The Guardian. His driver and translator were later released unharmed.
The Guardian said in a statement: "It is believed Mr Carroll may have been taken by a group of armed men. The Guardian is urgently seeking information about Mr Carroll's whereabouts and condition."
Mr Carroll's father, Joe, said the paper had told him three people had been with his son when he was abducted, "and one of them did get a bit roughed up, but he was the only one kidnapped."
Journalists have long been targets of kidnappers in Iraq, who calculate they are worth more than Iraqi businessmen. Other British journalists have been seized and briefly held by militiamen or insurgents but none has been held for a long time.
French and Italian journalists and foreign nationals have been detained and only freed after prolonged negotiations and the payment of large sums of money. Although all regular correspondents live in heavily defended hotels, they are vulnerable to attack when travelling to and from interviews. A second car is often used to see if the correspondent is being followed.
Joe Carroll, a former correspondent for the Irish Times, said his son had tried to reassure him about his safety in Baghdad. "He knew we were worried but he used to reassure us and say that it was not as dangerous as people outside think," he told BBC radio. "He said if you observed basic rules and security you would be OK. We knew he was playing it down for our sake and there was obvious danger."
The press freedom group Reporters Without Borders said that 72 journalists and their assistants had died in Iraq since the start of the war in March 2003. "Past experience with journalists being taken hostage in Iraq showed that a significant expression of support in the first few hours after the kidnapping were vital," the group said.
"Unfortunately, the safety of journalists is still far from being assured in Iraq and there are grounds for suspecting that tension linked to the start of Saddam Hussein's trial are having repercussions on the press."
The last reported abduction of a foreigner was in September, when a video posted on the internet showed Garabet Jekerjian being held at gunpoint. The Lebanese man works for a supply company. He has not been located.
Peter Hain, the Northern Ireland minister, said the Government would help in any way it could to secure Mr Carroll's release, but urged his abductors to remember that he was Irish.