The son of a master plasterer of Pollokshaws, Glasgow, he joined the Primitive Methodist church there, most of whose members were master tradesmen - builders, joiners, bricklayers, plumbers, painters and plasterers. He hoped to become a minister, but when his father died he took over the business, Gray's Ferro Concrete, which undertook public works contracts. This was successful and he moved to 'Holmwood', a larger house in Cathcart, where he and his wife lived from 1919 to 1931. The house, now in the care of the National Trust for Scotland, was completed in 1858 to the designs of Alexander 'Greek' Thomson (1817-1875). During Gray's occupation alterations were made to the house. The young people of the church were allowed to use the tennis court and given refreshments.
Gray's strength as both advocate and administrator was recognised. He was elected a Glasgow town councillor in 1928 and was its bailie from 1932 to 1935, as well as Deputy Lieutenant for the County of the City of Glasgow. He stood unsuccessfully for Parliament as a Liberal. Lloyd George was his guest at Holmwood on visits to Glasgow. Gray was a local preacher for over fifty years and leader of the Pleasant Sunday Afternoon movement. He was Vice-President of the Primitive Methodist Conference in 1917 and of the Methodist Conference in 1936. He retired from business in 1946.