In 1841 there is a George Spence working as a Male Servant for a farmer, James Waddington, in the Knaresborough district:
http://search.findmypast.co.uk/recor...1%2f0009556983
This would fit in with George becoming a groom. His age is given as 15, but the ages of the servants are all multiples of 5!
Edit: I don't think this is him now.
See 1841 Census below.