Jackson sold himself as a man of the people as opposed to the political elites of his time, but that narrative is inaccurate in many ways, the fact that he was a land speculator, supported and made money from the slave trade, was a well paid military officer who abused his soldiers and brought them into unnecessary battles, and so much more would prove that he didn’t come from or support the “common man” he claimed to. So why was he so popular? What appeal did Andrew Jackson have to poor whites across the country?
You're right that Jackson certainly wasn't poor but his actions throughout his life were extremely popular and seen as beneficial to the common man. One of the big differences between Jackson and all of his predecessors was that Jackson was associated with the West/Frontier, Tennessee very much being the west at the time of his coming into political prominence. The Virginia and Massachusetts Presidents all lived in settled portions of their states (what is now West Virginia and Maine being very much frontier provinces). The West was viewed as a means of achieving wealth and prestige for the common man in contrast to the more restrictive voting systems in eastern states (about 30-40% of white male Virginians couldn't vote prior to 1831) and more expensive land. Jackson greatly expanded land available for settlement in the red stick war with the treaty of Fort Jackson opening up more than half of Alabama and much of Southern Georgia for settlement. Jackson also continuously advocated for the removal of Native Americans throughout his political life which would open up more land for development. Democrats and Jackson also supported selling these new western lands at very low prices whereas more developed eastern states and politicians generally wanted a slow move west. The massive population move shouldn't be understated, I have seen estimates that by 1840 a third or more Virginians and South Carolinans lived outside of their states. Other policies like lowering tariffs were also seen as beneficial to the common man.
Finally I don't think you should undersell his popularity from the war of 1812 and especially the battle of New Orleans, without control of the Mississippi the West couldn't bring their goods to market and indeed New Orleans just might have been the single most important piece of land in the entire United States. Subsequent expeditions to Spanish Florida, where he may or may not have exceeded orders, ending a slave rebellion and hanging some British agents, played well with many people despite raising serious alarms in Congress.