He was born in Logierait, Perth and Kinross, Scotland. He emigrated to Canada in 1842 after completing an education in public schools at Perth, Moulin, and Dunkeld, Scotland. Mackenzie married Helen Neil (1826-1852) in 1845 and with her had three children, with only one girl surviving infancy. In 1853, he married Jane Sym (1825-1893).
When the Macdonald government fell due to the Pacific scandal in 1873, the Governor General, Lord Dufferin, had to call on someone to form a government. There was no clear leader of the Liberal Party. Mackenzie was the fourth person called upon, and the first to accept, the post of Prime Minister. Mackenzie formed a government and then asked the Governor General to call an election for January 1874. The Liberals won, and Mackenzie remained prime minister until the 1878 election when Macdonald's Conservatives returned to power with a majority government.
As Prime Minister, Alexander Mackenzie strove to reform and simplify the machinery of government. He introduced the secret ballot; created the Supreme Court of Canada; established the Royal Military College of Canada in Kingston, Ontario in 1874; created the Office of the Auditor General in 1878; and struggled to launch the national railway. After his government's defeat, Mackenzie remained Leader of the Opposition until 1880, when he relinquished the party leadership to Edward Blake. However, he remained as a Member of Parliament until 1892, because he died in Toronto, Ontario, from a stroke that resulted from hitting his head during a fall. He is buried in the Lakeview Cemetery, Sarnia, Ontario.