I can't find an official quote but I once saw a televised question and answer session with Warren Buffet. When an audience member asked what it was like being one of the world's richest men.
In Warren's reply he said: "One benefit is, I don't have to deal with people I don't like." Warren then went on to explain with grandfatherly charm that his days were spent dealing with people he admired or otherwise liked because he could afford to select those people to surround him. He explained that finding good people and investing in them was one of his keys to success.
Only the most wealthy of people can seriously claim that they don't have to deal with people they don't like. For everyone else in the world dealing with people who annoy you is just a fact of life. There's no real way out. Few of us can afford to ignore or avoid our bosses, coworkers, or authority figures if we want to.
Not having to deal with people you don't like is an expensive luxury.