Wrong side of the tracks

Meaning: A poorer or less socially respected area of a town or city.
Example: He grew up on the wrong side of the tracks but worked hard to succeed.