Wrong side of the tracks

The poor side of town.

The man married a girl from the wrong side of the tracks because he loved her.