Sloss became involved in railroads in the 1850s and fifteen years later ended up as president of the Nashville and Decatur line. During this postwar period, Sloss not only promoted the development of Southern rail, but became one of the chief proponents of Alabama’s postwar industrial development. In 1871 he struck a deal with the L&N Railroad to complete a 67-mile gap of the South and North Railroad between Birmingham and Decatur. Ultimately reaching the Gulf of Mexico, the L&N invested more than $30 million in furnaces, mines, wharves, steamship lines and other Alabama operations. By 1888 it was hauling annual tonnage of iron, coal, and other mineral products outweighing the nation’s entire cotton crop.

Sloss’s decision to bring in the L&N transformed Birmingham from a squalid jumble of tents, shanties, and boxcars into a thriving community. Anxious to tap the rich mineral resources surrounding Birmingham, Sloss, along with fellow Birmingham promoters Henry DeBardeleben and James Aldrich, acquired 30,000 acres and formed the Pratt Coal and Coke Company. Pratt soon became the largest mining enterprise in the district. In the early 1880s, with the backing of Henry DeBardeleben, Sloss founded the Sloss Furnace Company, and two years later ‘blew-in’ the second blast furnace in Birmingham. Called City Furnaces, the plant was located at the eastern edge of downtown, at the intersection of two major railroads. The majority of Sloss pig iron ended up in Cincinnati, Louisville, Chicago, and Cleveland. Pig iron costs in Northern plants averaged $18.30 per ton in 1884 while pig iron in the South could be produced for $10-$11 a ton. By the 1880s Birmingham was booming and had earned the nickname The Magic City.

Sloss retired in 1866 and sold the company to a group of financiers who guided it through a period of rapid expansion. The company reorganized in 1899 as Sloss-Sheffield Steel and Iron, although it was never to make steel. With the acquisition of furnaces and extensive mineral lands in northern Alabama, Sloss-Sheffield became the second largest merchant pig iron company in the Birmingham District.

James Withers Sloss continued to be interested in iron and steel-making until his death in May of 1890. Praising Sloss, an obituary in the national trade journal, "Iron Age", stressed "his farseeing discernment, indomitable energy and modern ideas."