Take for instance Monk's Mound. Shouldn't there be an equivalent hole somewhere? I did a cursory Google search, but came up with nothing. All of these mounds must mean a whole lot of holes as well. Where are they?
It depends on the mound and it depends on how the mound was constructed. Not all mounds were constructed in the same manner. And not all mounds were constructed using the same soils. Many mounds do have borrow pits, which are shallow depressions in which earth was extracted for construction. These borrow mounds are often less than 100 meters from the mound itself. But now all mounds used readily available earth so close. Some mounds went out of their way to target specific soils, either for their properties to aid in construction, their color, or both.
Take, for instance, the initial mound at Poverty Point. The mound was constructed using only E-horizon soils that could only be found in select locations within the region. The E-horizon is just one of many soil layers within the stratigraphy of the region, but it was nonetheless, selected for construction for a particular purpose. The builders of Poverty Point had to dig up to a meter or more beneath loess in order to find E-horizon soils.
At Shiloh, the mound had distinctive red layers within its construction. This red sediment, found 2 meters below the surface, was used to cover the mound and give it a red appearance. What makes the red sediment at Shiloh interesting is that the red sediment is found deeper than typical borrow pit depths. It is also underneath a discontinuous iron and chert pebble conglomerate that is difficult to breach and requires a lot of effort to reach. However, the nature of the red sediment may have aided in repelling water from the mound. Whether the water proofing or color was more significant is uncertain. It could be that both of these attributes were equally important in selecting material for mound construction.
I mentioned previously that mounds can be constructed differently. We typically think of mounds as simply piles of dirt made up of basketloads of material that are dumped in one spot and then tamped down. However, that is not always the case. Sod blocks have been found to have been used in mound construction, as well. Not only were sod blocks used, but different types of soil as well. Within a mound you can find more clay-like soils sandwiched between soils with more gravel. The clay layers allowed for a solid and firm foundation for mound construction, but clay swells when it is wet. If a mound was constructed completely from clay it would swell to a point in which the clay would then split apart. The gravel layers allow for the clay layers to swell and allows for water to drain from the mound and keep it from breaking apart.
You would have to look at each mound individually and understand its construction before you could begin to look for where the material may have come from. It may not be as simple as borrow pits using surface soils, but instead be much more complex involving specific soil types found in a limited number of locations within the region.
Sherwood, Sarah C., and Tristram R. Kidder. "The DaVincis of dirt: Geoarchaeological perspectives on Native American mound building in the Mississippi River basin." Journal of Anthropological Archaeology 30.1 (2011): 69-87.