The previous page (image 3 of 14) of that 1880 census that Kite posted has a family transcribed by Ancestry as:
Carroll J. A. 47
Carroll Elvira 39
Carroll Mary 13
Carroll Cintha 10
Carroll J. B. 5
Carroll Sarah 3
J.A. Carroll family
If he's the same James A., the age is about right as he was age 15 on that census of 1850 living with his parents Benjamin and Doshy Carrell.
Also, is that the same Doshy, now spelled Dosie Carroll, listed as Mother on image 13 of 14, Line 9? She's with a Mary Mitchell, and next door on the same page starting on Line 1 is the family of Milton Carroll.
There are more Carrolls on image 5 and image 6, and on image 7 is a family of Carrolls starting line 20 with the wife named Julyann b. about 1840.