Abdul Mahdi's decision, announced in a statement on Friday, came in response to a call for a change of leadership by Iraq's top shia leader Ayatollah Ali al-Sistani.
"I will submit to parliament an official memorandum resigning from the current prime ministry so that the parliament can review its choices," he said.
Earlier on Friday, Al-Sistani said that parliament, which elected the year-old government of Abdul Mahdi, should "reconsider its options".
"We call upon the House of Representatives from which this current government emerged to reconsider its options in that regard," al-Sistani said in his weekly sermon delivered in the holy city of Najaf via a representative.
Abdul Mahdi said he "listened with great concern" to al-Sistani's sermon and made his decision in response to his call and in order to "facilitate and hasten its fulfillment as soon as possible".
The developments came a day after more than 50 people were killed by security forces in one of the bloodiest days of violence since the anti-government protests erupted in early October.
Hundreds of people have been killed and thousands wounded since the demonstrations started amid a fierce response from security forces, who have used live ammunition, tear gas and stun grenades against those taking to the streets.
The protests, sparked by widespread anger over official corruption, mass unemployment and failing public services, have gripped Baghdad and several cities in Iraq's south.
On Friday, celebrations erupted in Iraq’s Tahrir Square following the announcement.
Al Jazeera's Mohammed Jamjoom said the atmosphere in Tahrir Square was transformed.
"This is an extremely vibrant scene and it really goes on to show that right now in this particular moment you have these demonstrators here in Baghdad who feel this is a win for them," he said, reporting from the capital.
"Some people wonder what exactly is going to happen next, but at this particular moment people are extremely happy by this development," he added.
Yanar Mohammed, the co-founder of the Organization of Women’s Freedom in Iraq, told Al Jazeera that Abdul Mahdi's resignation is "a first step to a victory over a corrupt sectarian and criminal government".
"This feels like the victory of the uprising of the people, that finally the power and the willpower of the people of the uprising has overcome," she said.
However, protesters are not content with just the prime minister's resignation, and vowed to continue demonstrating in the streets until all their demands have been met.
"The condition is that all the ruling political parties have to step out of the political formula," Mohammed said. "They are not acceptable anymore. They all took part in the criminal killings of the demonstrators."
"We will have to continue the uprising until all our conditions are met."
Source: Al Jazeera and agencies