That's not true, at least not between Baltimore and New York. The drive is at least 4 hours. Even the Northeast Regional does it in 2 hours 40 minutes, and the Acela does it in 2 hours 20 minutes. One can arrive at the station at any time before departure, in my case, I usually like to get there 2 minutes early, but when I was commuting, I'd sometimes get there about 5 minutes before train time. There is really not need to increase the maximum speed on this route, though clearing up bottlenecks to reduce total travel time would be beneficial.
I'm not sure about this need to always increase maximum speed, just because someone else is doing it. Even the airlines are slowing down their airplanes. It doesn't add that much travel time and it saves a lot of money in reduced fuel consumption, not to mention reductions in greenhouse gas emissions. Who needs to fly at 500 mph when 450 mph works perfectly well?