Richmond, VA got lucky in that of the two nice stations it had, one is back in service (Main Street Station was taken out of service following Hurricane Agnes flooding the whole neighborhood, though Amtrak had wanted out for a while) while the other is the Virginia Science Museum (though IIRC the tracks are still there to Broad Street Station).
And the answer is that not many cities in their right minds would knock down those stations...but a lot of cities took leave of their right minds sometime in the 50s. In other cases, the city wasn't part of the equation (since unlike most commuter terminals, most of the old stations were privately owned and in some cases the alternative would have been a massive deteriorating eyesore like happened in Detroit...more specifically, with Michigan Central Station) and it didn't make sense for anyone to maintain a massive station for one or two trains per day.
Let's also not forget the saga of Washington Union Station (which lost passenger service for quite a while in lieu of the putting a smaller building on property, if I'm not mistaken, and which wasn't kept up well for a while). I think that might be illustrative...you had plenty of cases where the alternatives were public ownership (expensive for light service), forced private ownership that would probably have found ways to let the building collapse while insulating themselves from liability (cheaper but long-term problematic), and demolition (unfortunate).Does Richmond Hull Street (former Southern Railway Station still exist? I have seen pictures, but never saw it live. I travel from both Main Street and Broad Street.