The deputy provincial governor of Ghazni province Mohammad Ali Ahmadi said one militant blew himself up at the prison gates, allowing others to force open jail cells.
The attackers were well-organised and wearing uniforms, he said.
The Taliban confirmed it had carried out the early morning attack.
Both Taliban spokesman Zabihullah Mujahid and the deputy governor confirmed that three of the attackers were killed.
The deputy governor said seven police officers were also injured. He said only 80 prisoners were left inside the prison.
It was not immediately clear who the prisoners who escaped were, nor where they had gone.
Ahmadi said he did not reject the possibility the attackers had had help from inside the jail. He added that most of the prisoners were Taliban, with some petty criminals and drug addicts.