The mosque is in the heart of Mogadishu's Bakara Market, a stronghold of Al-Shabaab, the militant group waging a war against the government in an effort to implement a stricter form of Islamic law.
The bloody incident took place about 1 p.m., according to Ali Muse, a local ambulance service director. He said hundreds were wounded, and he expects the casualty figures to rise.
Fighting ensued in the city hours after the incident, when a pro-government militia group slugged it out with militants.
An Al-Shabaab spokesman, Ali Dhere, initially blamed "foreign elements" for the attack, but an AU peacekeeping official said no group has claimed responsibility.
Dahir Mohamud Gelle, the government information minister, called the attack "barbaric" and said it illustrates "a total lack of wisdom and a disrespect to the holy places."
The special representative of the chairman of the AU Commission for Somalia, Ambassador Boubacar Gaoussou Diarra, deplored the killings of innocents and any attacks targeting mosques and other public places.
"Indiscriminate attacks on public places like today's incident cannot be condoned. I, on behalf of the African Union, would like to call upon all warring parties in the Somali Conflict to stop such barbaric attacks on innocent civilian population," Diarra said. Video: Fighting Islamic militants in Somalia
The Horn of Africa nation has not had a stable government since 1991, and fighting between the rebels and government troops has escalated the humanitarian crisis in the famine-ravaged country.
Al-Shabaab is on the U.S. list of terror organizations because of its ties to Osama bin Laden's al Qaeda network.
A local journalist said two explosions detonated on the lower floor of the two-story Abdala Shideye Mosque when people assembled for midday prayers.
The mosque was still standing but seriously damaged, said the journalist, who saw the results of the carnage.
Survivors told him that two high-ranking Al-Shabaab officials died and another was seriously wounded. The journalist said everybody in and outside the lower floor was either killed or injured.
The AU said it is unclear whether the explosions were caused by suicide bombs.
Al-Shabaab fighters guarded the mosque, described by several local journalists as a facility strictly serving the militant movement.
After the blast, the journalist said, gunmen with Ahlu Sunna, a pro-government Sufi Muslim militia, fought near the market with members of Al-Shabaab and Hizbal al-Islam, another Islamist extremist group. Ali Muse said he counted at least two dead and 13 wounded in the fighting.
Heavy fighting engulfed the swath of city for two hours, and sporadic gunfire and mortar blasts rang out as fighting died down.
Witnesses said the Sufi militia took ground from the militants and was moving toward the market.