The new HBO series The Gilded Age, from Downton Abbey creator and writer Julian Fellowes, is introducing a new generation to the historical figure of Ward McAllister. Famous for being an arbiter of New York’s “high society” of the 1860s–90s, McAllister used his list of “the 400” to advise Caroline Schermerhorn Astor a.k.a. “Mrs. Astor” on whom should be “in” and whom should be “out.”

But, before arriving in New York in 1858, Ward had spent the dawning years of the 1850s in San Francisco, where he lived in the same house with his older brother Hall McAllister, who arrived in the city in 1849 and remained until his death in 1888 — a period during he which he became of the most eminent and respected attorneys of his generation.

It’s very likely that, in the early 1850s, Joshua Norton — then at the height of his prosperity and influence — socialized with the brothers McAllister in their home, together with his friend Joseph Eastland, a founding partner of the company that went on to become PG&E.

In fact: It was Hall McAllister who — in 1853–54 — represented Joshua’s opponents in the rice affair.

It’s a fascinating set of connections that (a) reveals Joshua Norton to have been a guest — but never really a member — of a world of privilege and power that would become closed to him once his life took a different turn, even as it (b) shines new light on one member of that world who never forgot him.