He said there is little chance that the weapons — whose alleged existence provided the main basis for war — will be found independently, or that top officials will provide useful information.
"I never believed that we'd just tumble over weapons of mass destruction in that country," Rumsfeld said in a broadcast interview, echoing President George W. Bush's comments Saturday.
"I'm not frustrated at all," Rumsfeld said later in a separate televised interview.
Because Iraq's weapons program was central to the coalition's argument for invasion, finding them may be crucial to proving the administration's case, reports CBS News Correspondent Gretchen Carlson.
"If Saddam didn't have weapons of mass destruction, why didn't he invite the inspectors in and cooperate fully? And if he did have them ... why didn't he use them?" Lee Feinstein, of the Council on Foreign Relations, told CBS News.
The defense secretary also said he does not know if the ousted Iraqi president and his sons are alive or dead. Whatever their fate, they are out of power and no longer threatening Iraqis, Rumsfeld said. "They're either in a tunnel someplace or in a basement hiding," he said.
Rumsfeld said weapons of mass destruction will not be easily found in Iraq because Saddam hid them from U.N. inspectors.
"We're going to find what we find as a result of talking to people, I believe, not simply by going to some site and hoping to discover it," said Rumsfeld, just back from a tour of Iraq, Afghanistan and the Persian Gulf region.
So far, high-level Iraqi officials, such as Tariq Aziz, one of Saddam's closest deputies, have not cooperated or provided information about the weapons, U.S. officials said.
"We're going to have to find people not at the very senior level who are vulnerable, obviously, if they're in custody, but it will be people down below who had been involved in one way or another," Rumsfeld said.
Asked if any of these lower-level officials are cooperating, Rumsfeld said: "Are they telling us something substantive? We don't have anything substantive to announce at the present time."
Secretary of State Colin Powell also expressed confidence that evidence will be found, and he denied a suggestion he overstated the case against Iraq and its arsenal. "I'm absolutely sure that they had weapons of mass destruction, and I'm sure we will find them," he said on CBS television.
On other topics, Rumsfeld on Sunday:
Said it is unclear what sort of trial or tribunal will be convened to try former Iraqi leaders. Options include a world court, a U.S. proceeding or an Iraqi-convened trial. "I don't even know that one size will fit all," he said.
Said it is "an open question" as to how many U.S. troops will remain in Iraq and how long they will serve. "We're going to have as many people in there as we need for as long as we need them."
Declined to rule out a military strike against North Korea, which is believed to have nuclear weapons. "I don't know what'll happen there," he said.
Predicted nations that are aiding terrorists or developing weapons of mass destruction may have learned a lesson from the U.S.-led war against Iraq. "I think that that message that's gone out is a solid one, it's a healthy one, it's good for the world, and we may see some behavior modification."
Said the Pentagon will evaluate its military needs before deciding what weapons to purchase in rebuilding its stockpile. Asked what future needs would include, he said: "We simply have got to be able to move in hours and days and weeks, rather than months and years. We need to be swift." Still, he said, the military will always need heavy armor but those assets may be positioned more strategically.