There are a couple of factors here. One is that this segment is owned and dispatched by Metro North and is therefore optimized for the running of commuter service for which speed is less of an issue due to frequent stops. It is also a very busy railroad, probably the busiest in the county. I think they do try to keep Amtrak moving but if an Amtrak is a few minutes late and loses its slot on this busy piece of railroad it can get delayed further.Recent trip from NYP to BOS on Acela, I had forgotten how SLOW this and other trains actually go between New Haven and New Rochelle, YET we still came in to New Haven TEN MINUTES under the advertised on the schedule - during rush hour. I’m not sure if we ever hit 70 mph - it was stop and go the whole time. We were even TEN MINUTES early into Boston. Been a long haul of upgrades on this section, and over a pretty long time. Not sure how close progress is towards some sort of improvement. But long over due and now with recent speed increases and work done on other portions of NEC, this section really stands out - to me at least. Good show though overall, with Acela and onboard staff….
Another factor is that this segment has been perpetually under construction due to the need to catch up with decades of deferred maintenance, including replacement of catenary and bridges. This often requires tracks be taken out of service and what is normally a 3 or 4 track railroad is down to 2 tracks in some places.
A third factor is that the right of way is fairly curvy to begin with and has lots of bridges with slow speed limits.
In short, while someday the maintenance will get caught up and that may help, I don't think you will ever see speeds more than say 80 mph on this segment.