Obviously, it is a passenger's responsibility to make the trains. The thing is, given that responsibility, people are likely to choose not to ride if they don't think they will be able to reach the train station and catch a train within a convienent amount of time- convienent being in the eyes of the rider- within the level of guarentee the rider personally wants.
That is, reduce frequency and reduce the number of hours the service operates, the less riders will be confident they with a) find it convienent, and b) not get stranded. As that diminishes, so does ridership. Even if the ridership at the hour you have decided to discontinue is minimal. The knowledge the train will be there if it is needed is actually more important then the actual people moving of that particular run.