The primary deciding factor, in my opinion, is whether you want to see the Front Range of the Rockies departing Denver in the daylight.
If so, you want the California Zephyr westbound. Otherwise the scenery by daylight is about equal in both directions.
I will add that as cool as climbing the Front Range is going west, it is also breathtaking going eastbound on the California Zephyr seeing the lights of Denver below you as the train works it's way down out of the mountains. You're in for a treat either way.
I think on this trip I would be inclined to go west on the SWC and east on the CZ. This would insure maximum daylight viewing over Raton Pass which is not likely to host the SWC much longer. A late eastbound SWC would put you through here in the dark.