Gzowski was born in St. Petersburg to a noble Polish father who served with the Russian military.

He emigrated with his family to the United States after the Polish revolt against Russia in 1830. Trained as an engineer, he was involved in railway construction in the United States. Eventually he was hired as an engineer to help in the construction of the New York and Erie Railway.

In 1841 he moved to Canada to work on the Welland Canal, and also helped finish the building of Yonge Street in southern Ontario. In 1853 he also helped build the Grand Trunk Railway. As president of the Toronto Turf Club, in 1859 he was a pime factor in the creation of the Queen's Plate, the first organized thoroughbred horse race in North America. He was the first Commissioner of the Niagara Parks Commission.

He was knighted by Queen Victoria in 1890, and briefly served as Lieutenant Governor before Oliver Mowat took power in 1897.