Former Afghan President Rabbani was negotiating with the Taliban but was killed by a suicide bomber purporting to be a Taliban peace emissary.
US President Barack Obama has renewed calls for Pakistani action against militants of the Haqqani network.
Mr Karzai, speaking to a group of religious leaders, said there were no partners for dialogue among the Taliban. It was not possible to find the Taliban leader, Mullah Omar, he added.
"Where is he? We cannot find the Taliban Council. Where is it?" he said.
"A messenger comes disguised as a Taliban Council member and kills, and they neither confirm nor reject it. Therefore, we cannot talk to anyone but to Pakistan," Mr Karzai told the meeting.
"Who is the other side in the peace process? I do not have any other answer but to say Pakistan is the other side in the peace talks with us."
A statement by members of the nationwide council of religious scholars praised Rabbani's efforts to bring peace to the country, and condemned his killing in the strongest terms.
'Terrorist hot bed' Last week, the US military accused Pakistan's spy agency of helping the Haqqani militant network in a recent attack on Kabul.
Pakistan's foreign minister responded by warning that the US could lose Pakistan as an ally if it continued to publicly accuse Islamabad of supporting militants.
Late on Friday, President Barack Obama renewed calls for Pakistan to take action against the group.
"My attitude is, whether there is active engagement with Haqqani on the part of the Pakistanis or rather just passively allowing them to operate with impunity in some of these border regions, they've got to take care of this problem," said Mr Obama.
Pakistan has long denied supporting the Haqqani group, but BBC correspondents say it has a decades-old policy of pursuing foreign policy objectives through alliances with militants.
Although Islamabad denies the network has safe havens inside Pakistan, the country's former national security adviser told the BBC that it was operating in North Waziristan, in Pakistan's restive tribal belt.
"Today North Waziristan is a hot bed," said Retired Maj Gen Mahmoud Durani.
"It's not just Haqqanis. Everybody who is anybody in the terrorist field is there. Although there is military (there)... I think they have a fair amount of freedom of action."
He said the army was too overstretched to take on the Haqqani group.
The BBC's Orla Guerin says that, privately, officials admit that the group is not a target for Pakistan because its members don't kill and maim inside the country.
US officials say they are close to deciding whether to label the group as a foreign terrorist organisation, and the Treasury Department on Thursday announced new sanctions on five individuals it said were linked to "the most dangerous terrorist organisations operating in Afghanistan and Pakistan".