The next place we find James Collins is in the 1870 US Census. He is now living in Richwood Township, Richland County which is located in southwestern Wisconsin.
He is listed as a farmer, owning $2000 worth of land. He now has 4 children with his wife Elizabeth. This census also lists his birth place as Indiana.