Plenty of countries are able to build public sector projects efficiently. Historically that included this one: the Hoover Dam was built for $50 million, or $800 million at today's prices. Could we even build a bridge of that length for below a billion today?
The problem isn't necessarily that we contract out projects rather than having three Army Corps of Engineers build them, either. In the 1860s - with the Civil War causing massive shortages of manpower, steel, and gunpowder - the Transcontinental Railroad cost $30,000 per mile to build, paid out to the Union Pacific / Central Pacific. Yes, there was enormous graft, but even then that still comes out to $800,000 per mile today - on an all-new alignment while fending off attacks from hostile Indian tribes, without access to bulldozers, tunnel boring machines, etc. Compare to the proposed Uinta Basin Railway in Utah, crossing 120 miles of similar sparsely-populated terrain on an all-new alignment, for an expected cost of ~$3 billion - if it ever gets built, which after three years in court is looking increasingly unlikely.
The reason for our exorbitantly costly public-sector projects is first and foremost the downright Byzantine regulatory environment surrounding them.