Could my West family be aristocratic?

[The Wannabe Toff]

One brick wall I have been BANGING my head against for nearly two years now is the family of my four-times-great-grandfather, Jobe Henry West. He was a brewer, born in Chatham, Kent in 1802 and died in Croydon on 23 July 1880. Beer will always be a business, however in his will he left behind close to £21,000 - £2.5 million in today's money! This makes me think he may have been from a well-off family. Someone who was working on the same branch sent me evidence of an aural tradition passed down through my West family - according to Jobe's sister Sarah, their father was a son of a peer who had large estates in Camden Town, but the son - a John West - never claimed his inheritance as he was apprenticed to a builder who business he took over. He apparently abandoned that business to escape compulsory service in the Napoleonic War and took a job in the Chatham dockyard, returning to London later to be buried in St. John's, Hoxton. More of the tradition is that Jobe's brother James claimed a link to the Sackville-West family, but unfortunately it was said written evidence of a noble connection was stolen from James' son never to resurface. I DO know that John West definitely lies in the St. John's, Hoxton graveyard, so it may not be such a tall tale as it looks. I have found daguerreotypes of Jobe's siblings Edward, Mary, James and Eleanor uploaded by various people on their Ancestry trees, so I'm wondering if Jobe also exists in someone's family treasures somewhere. If you're descended from this family in any way, I'd like to know if you have pictures or viable research/evidence.