OK, checked your references. As said, I believe you understated costs, and all of these confirm this. First, the track reference: It says "siding", that is the cost of an industrial track, not a main line. In its description is says used rail, and several other things that would not be acceptable for a main line track. Here is another set on this one:
From:
https://www.freightwaves.com/news/commentary-do-you-want-to-build-a-freight-railroad
Here is a broad rule of thumb when trying to estimate what a railway line’s new construction might cost.
Ignoring the land cost, the basic rails + ties + ballast + sub-compaction and grading might as a capital budget expense come to between $3.5 million and $4.5 million for each route mile when built as a single-track main line.
If you build a parallel second main track, then consider adding another $1 million to $1.5 million per mile to the budget plan.
Are you building this track in an urban area? Prudently add another $2 to $4 million a mile to the capital construction budget. Add more if the terrain requires tunnels and bridges. Add even more if utilities or structures need to be relocated.
While this says freight, once you get to a solid track structure for main line traffic, there is very little cost difference between heavy duty track for freight or for high speed passenger service, other than allowed tolerances in deviation from perfection. By the way, the "grading" in this reference is fairly incidental and does not include significant cuts and and fills, drainage structures, utility issues or bridges.
Your highway references are from Florida DOT. There are few states, maybe some of the fairly flat farming states in the western plains where road construction costs could possibly be lower. The numbers in the reference do not include any significant grading beyond basic smoothing, so area times depth of cut or fill and cost per unit thereof needs to be added. Also there is an allowance for some minimal culverts, but not for any bridges or significant drainage structures, overpasses, underpasses, etc.
For a single track line, you can use their two lane highway number, subtract pavement, add track, for double track, use the three lane or four lane undivided highway number, again subtract pavement add track.
I stand by my original statement, you cost numbers for both rail construction and road construction are low. In fact, I would say significantly low.