Wikitionary: unwashed masses (plural only)
(idiomatic) The collective group ("mass") of people who are considered by someone to be somehow uneducated, uninformed, or in some other way unqualified for inclusion in the speaker's elite circles.
It's implied. I'm trying to think of a group of unwashed above average income people. Burning Man attendees are the only group that comes to mind. Maybe you live in a country where rich people don't wash up. Edit: then there's the filthy rich.