BART was partly a replacement for the Key system and used some of its right of way, but as originally designed (i.e. with Marin and San Mateo counties), and even as eventually built, it went beyond what the Key system did. The lower deck of the Bay Bridge was designed to support the Key system, which was essentially light rail. It wasn't for freight or other heavy rail uses.
In retrospect, using a different gauge was probably a mistake, but it made sense at the time. BART was envisioned to be a space age replacement for commuter trains – computer controlled (drivers would only have an emergency stop button) and with 90 second headways, which is why the original cars didn't have straps or grab bars – there would be so many trains that everyone would have a seat. At the time, making BART tracks compatible with legacy rail made as much sense as designing the cars to be able to run on highways.
BART was the Hyperloop of its day. We can only hope things turn out better for Elon.