The park was originally the site of the British garrison, as well as the cricket grounds. The garrison was expanded with new buildings during and after the Upper Canada Rebellion in 1837. The British troops withdrew to Europe in 1853 to train for the Crimean War, but their barracks were used to house escaped slaves from the United States, as one of the end stations of the Underground Railway. The troops returned in 1861, fearing that the American Civil War might spread to Canada. In 1874 the park was transferred to the city and renamed Victoria Park, after Queen Victoria.
Although designated for recreational activities, the park was still used as a military garrison when necessary. As London was the centre of the Western Ontario military district (District No. 1), troops were stationed in the park during the Boer War, World War I, and World War II; there was some minor rioting in the park during the Conscription Crisis of 1944, when conscripts demanded to be sent to Europe.
In 1907 three cannons from the Crimean War were placed in the park. In 1912 a statue was built as a memorial to the Boer War, and a cenotaph was built in 1934. A Sherman tank (known as the "Holy Roller") used in World War II was placed there in 1950.
Many annual events are held in Victoria Park. These include the London International Children's Festival, a 3-on-3 basketball competition, Sunfest, the Home County Folk Festival, Rib Fest, and Fiesta del Sol. Every winter, there is an annual vigil for the Montreal Massacre, the trees in the park are decorated with Christmas lights, the "Lighting of the Lights" and Snowfest is held in Febuary.
The park is notable for the presence of a large number of melanistic (black) Eastern Gray Squirrels (Sciurus carolinensis). In February 1961 a group of these was taken to Kent State University, in Ohio, USA, where they have become firmly established.