England-born Smith will be replaced by chief financial officer Shayne Elliott on January 1, 2016 but will serve as a non-executive adviser to the Australia and New Zealand Banking Group's board for at least a year due to his "invaluable understanding of and connections in Asia", the bank said.
Chairman David Gonski praised Smith for his efforts in steering the company -- which has a market capitalisation of Aus$78.1 billion -- through the global financial crisis and for transforming "ANZ into one of Asia-Pacific's leading banks".
"Mike has been a champion of ANZ and of Australia globally but particularly in Asia," Gonski said in a statement.
Shares in ANZ, one of Australia's four major banks, rose 1.81 percent to Aus$27.57 in midday trade.
Gonski said New Zealander Elliott, who previously held senior positions at Egypt-headquartered investment bank EFG-Hermes and American firm Citibank, was chosen after a detailed review of candidates.
"Shayne's international banking and financial experience, together with his background in Australia and New Zealand makes him the right person to lead ANZ in a challenging global environment," he said.
Asia-focused ANZ operates in 34 countries and reported a 4.3 percent increase in cash profit to Aus$5.4 billion in the nine months to June 30.