Parliament has reportedly refused to accept the resignations.
Security sources have told the BBC that Yemen's intelligence chief, Ali Hassan al-Ahmedi, has also stepped down.
There are reports of explosions in the southern port city of Aden, where President Hadi has a strong following.
Houthi rebel figures publicly welcomed the resignation of the president with one proposing the creation of a ruling council.
The council would include Houthi-led groups, Abu al-Malek Yousef al-Fishi was quoted as saying by Reuters news agency.
Houthi leaders had previously committed themselves to withdrawing from key positions around the presidential palace and the home of President Hadi.
The US, which is helping fight al-Qaeda militants in Yemen, said it was still assessing the implications of President Hadi's move.
Analysis: Sebastian Usher, BBC World Service
The resignation of the Yemeni president and his government is likely to plunge an already unstable country into uncharted territory.
Mr Hadi has been surrounded in his home in the capital, Sanaa, by rebels for several days. His resignation may be the last card he had left to play after being pushed to the brink.
The rebels have publicly welcomed his resignation but it is not what they wanted. As minority Shias from the north, their authority will not be accepted in the south, where many want to return to having their own state.
With no president and no government, there is also a fear that al-Qaeda's powerful branch in Yemen could make new gains from the political vacuum.
'Political maze' Local officials in Aden told Reuters news agency that unidentified gunmen had attacked two military vehicles in the city early on Friday, local time.
Three explosions were heard during the attack, which was followed by the clashes, said one of the officials, who declined to be identified.
Explosions and clashes were also reported by Qatar-based broadcaster al-Jazeera, which said they were happening on the city's Martyrs' Square.
In his letter of resignation, seen by the Associated Press news agency, Mr Hadi said the parties had reached a "deadlock".
"We found out that we are unable to achieve the goal, for which we bear a lot of pain and disappointment," he said.
A government source told the BBC that ministers were resigning in protest at the rebels'
challenge to Yemen's sovereignty and their seizure of state institutions.
In his resignation letter, Prime Minister Khaled Bahah said the cabinet did not want to be dragged into an "unconstructive political maze".
Earlier this week, Houthi gunmen fired on Mr Bahah's convoy and then laid siege to the presidential palace, where he was staying.
Then on Wednesday the home of President Hadi was shelled, shattering a ceasefire that had been agreed only hours earlier.
The ceasefire deal had met a series of rebel demands including the expansion of Houthi representation in parliament and state institution.
In return, the rebels said they would pull back from their positions and free the president's chief-of-staff, whom they have held since Saturday. But so far they have not done so.
The Houthis, who follow a branch of Shia Islam known as Zaidism, have staged periodic uprisings since 2004 in an effort to win greater autonomy for their northern heartland of Saada province.
Since July the rebels have inflicted defeats on tribal and militia groups backed by the leading Sunni Islamist party, Islah, and battled al-Qaeda as they have pushed into central and western provinces.