The AU High Level Ad Hoc Committee welcomes Colonel Gaddafi's acceptance of not being part of the negotiation process," AU leaders said in a statement after Sunday's meeting.
"Following the suspension of hostilities ... the Libyan parties should begin the national dialogue for a comprehensive ceasefire, national reconciliation, transitional arrangements, as well as the agenda for democratic transformation."
Al Jazeera's Peter Greste, reporting from Pretoria, said AU leaders called for an immediate end to hostilities before anything else.
"It says that will all lead to an interim government with the support of the African Union and the United Nations," he said.
"Of course, all of that is a very long way off. We have to get to those talks in the first place. But ... it does appear to be a major breakthrough."
There was no immediate confirmation from Libya that Gaddafi would step out of any negotiations - but if confirmed, the agreement would remove a major obstacle to peace.
'Ceasefire coming soon'
Al Jazeera's Zeina Khodr, reporting from the eastern Libyan stronghold of Benghazi, said opposition leaders had dismissed the AU proposal.
"What they would like to see as the starting point is for Gaddafi to pull back his forces, for his forces to return to the military barracks, as well as for Gaddafi to release the scores of prisoners that they are holding," she said.
"For them, there is no solution to this conflict unless Gaddafi actually leaves office [and] leaves power."
Jalal El Gallal, a spokesman for the opposition Transitional National Council in Benghazi, said opposition leaders are also unwilling to talk to anyone in Gaddafi's family or his inner circle.
"It's very difficult to speak with anybody that has blood on his hands," Gallal told Al Jazeera.
"If there's going to be any negotiations, first they have to adhere to the resolution of the United Nations by withdrawing their forces to the barracks, allowing humanitarian aid and allowing people to demonstrate."
Many AU leaders, including Zuma, have voiced reservations over NATO's ongoing air campaign against Gaddafi forces.
Gaddafi is a long-time backer of the AU and a forceful advocate for stronger continental integration. He held the pan-African body's rotating chair in 2009 and has twice held talks with members of the panel.
The UN had sanctioned the air assault with a view to protect civilians from a security crackdown that the Libyan leader unleashed in response to the uprising against his rule.
Zuma's government, which currently holds a non-permanent seat on the UN Security Council, did vote in favour of the UN resolution, accusing Gaddafi of committing a "heinous violation of human rights against his own people" with his crackdown on the anti-government protests.
But he has since been critical of NATO for pursuing regime change, straying far outside the resolution's civilian protection focus.
Earlier this month, Mauritanian President Mohamed Ould Abdel Aziz, who chairs the AU panel on Libya, told AFP that Gaddafi "can no longer lead Libya," and that "his departure has become necessary".
The president and other African leaders have repeatedly called for a ceasefire and a diplomatic solution to the conflict, although they have so far failed to come up with a truce proposal that meets the rebels' and NATO's pre-condition that Gaddafi and his inner circle must leave power.
In recent days, rumours have been rife that the Libyan leader may consider leaving Tripoli and that rebels could accept his internal exile to a remote location.
The rumours have been fuelled by a military deadlock on the ground and a steady trickle of defections from Gaddafi's forces.
The rebels said on Saturday that 38 Gaddafi officers - including six high-ranking officials - fled to Tunisia a day earlier.
Source: Al Jazeera and agencies