After seizing state TV, an army spokesman announced it was targeting people close to Mr Mugabe who had caused “social and economic suffering”.
Messages attributed to a ruling party Twitter account described the takeover as a “bloodless transition”.
The move came after Mr Mugabe sacked his deputy, Emmerson Mnangagwa, in favour of his wife.
A statement read out by a general on air denied it was a coup and said Mr Mugabe was safe, but did not say where.
Heavy gun and artillery fire could be heard in northern parts of the capital Harare early on Wednesday.
Soldiers overran the headquarters of the ZBC broadcaster after armoured vehicles took up position on roads around Harare on Tuesday.
Maj Gen Sibusiso Moyo went on air to say the military wished to “assure the nation that his Excellency the president… and his family are safe and sound and their security is guaranteed”.
“We are only targeting criminals around him who are committing crimes… that are causing social and economic suffering in the country,” he said.
“As soon as we have accomplished our mission, we expect that the situation will return to normalcy.”
Other key points of the statement included:
- Citizens should remain calm and limit unnecessary movement
- The military assures the Zimbabwean judiciary that its independence is guaranteed
- Security services should “co-operate for the good of our country” and any provocation would “be met with an appropriate response”
- All leave for the defence forces is cancelled and personnel should return to barracks immediately
It is not clear who is leading the military action. Army chief Gen Constantino Chiwenga had said the army was prepared to act to end purges within the ruling Zanu-PF party.
A government source told Reuters news agency that Finance Minister Ignatius Chombo had been detained.
He is a leading member of a faction of Zanu-PF led by Mr Mugabe’s wife Grace.
Alex Magaisa, former adviser to Zimbabwean opposition leader Morgan Tsvangirai, told the BBC he believed the military’s claim that they hadn’t carried out a coup was untrue.
“They have decided not to call it a coup because they know that a coup does not sell, it will be condemned,” he said.
“But as far as authority is concerned it seems very clear that President Mugabe is now just a president in name and authority is now residing in the military.”
Zanu-PF had accused Gen Chiwenga of “treasonable conduct” after he issued his warning that he army might intervene.