The rich. They crap up everything.
I kid! But this says in cities where income inequality is high, the poor pay significantly more for housing. From Brookings:
Inequality at the local level may be undesirable for a variety of reasons. It may diminish the ability of schools to maintain mixed-income populations that produce better outcomes for low-income students. It may narrow the tax base from which municipalities raise the revenues needed to provide essential public services and weaken the collective political will to make those investments. And local inequality may raise the price of private-sector goods and services for poor households, making it even more difficult for them to get by on their limited incomes.
You can read more here. And thanks, Leftover, for the link.