What happened in 1980?
The cause of literally everything of course, Suburbanisation, urban renewal and the decline of American Industry in the 70s
While he did ruin so many things, the drop we are looking at was before his election as it took place between 1970 and 1980 and therefore he wasn't a direct cause in the population decline of Philadelphia, Nixon on the other hand...
Here is the metro area population over time for comparison, where Philadelphia grew by about 200,000 people, so my theory is that a lot of the decline came from the growth of the suburbs