I can't believe no one's mentioned Bill Clinton!
Clinton's father was a travelling salesman who died three months before he was born, his mother left him with her parents when he was a baby to study to become a nurse. She returned when he was 4 and married a car dealership owner who was an alcoholic and gambler and who abused her, with Bill sometimes having to intervene to protect her from him.
He went to college on scholarships. While his step-father's occupation meant they probably never went below the breadline, he certainly did not have a privileged upbringing.
I'm not sure you'll find great data here, specifically because of the "average income" part. So you may need to look at things qualitatively due to a lack of quantitative data. For example:
From that, you can draw out that Washington grew up in an above-average family, successful land-owners, but perhaps not elite plantation owners and probably lacking major political clout.
Now, just to cherry-pick some others that I know about off the top of my head...
Jackson was born somewhere in the Piedmont region of NC/SC, his Irish father having passed away shortly before he was born. He claimed later that he was raised on his uncle's plantation. The family was not considered elite or particularly prosperous, but being raised on a family plantation implies at least some above-average circumstances perhaps. Of course, any wealth or privilege would have been his uncles, not his or his immediate family's.
Polk was born near Charlotte, his father was a slaveholder and successful farmer. Sounds a lot like Washington, probably above average, but not elite. I've been to his boyhood home before, it's a fun site - clearly not one of the largest estates from the era, but he certainly came up better than most.
T.R. was definitely elite - his father was a businessman in NYC and a major philanthropist - family money.
Eisenhower's father ran a general store that failed, and they moved around a lot during his childhood. I think it's safe to say they were average/middle-class at best.
Kennedy - I probably don't need to explain this to anyone here, but the Kennedy's were a powerful family, even before JFK. Father was head of the SEC before being the Ambassador to the UK, clearly JKF had a privileged childhood.
Reagan was the son of an immigrant, a salesman, and they lived their first few years in an apartment building. Probably average at best.
Just a few examples, if anyone wants to provide a more comprehensive list I would defer to them. But I think my main point is... I doubt you can find much solid quantitative data prior to 50 years ago, 100 at best (foundation of the Census Bureau was sometime in the early 1900's, so I'm thinking that's the earliest you'll find good data if you really want solid numbers). But the bigger problem will be finding data around the family's income, as that's not often tracked or recorded very well. So, again - qualitative looks might be the way to go - and I'm sure there are people here more qualified than me to come up with these short childhood profiles.
NO GUESSING!
We have had to nuke almost every posting here because there have been a lot of unsubstantiated guesses and outright WRONG ANSWERS!
Seriously. If you are going to link to wikipedia as your source, at least READ THE ARTICLE you are listing. Much of the information directly contradicts what you are saying!
Have you looked at Edward Pessen's The Log Cabin Myth? It is outdated, and it is more interested in social class than actual inflation-adjusted income, but its central contention is that most American presidents came from at least middle class backgrounds: Jackson, Fillmore and Nixon are the only exceptions. At the same time, he contends that American political culture has mythologised the idea of many of their political leaders coming from humble origins, to play up the "social mobility" of their system.
The book won't directly answer your question as it doesn't contain the kind of hard financial stats it sounds like you're looking for, but it does give an interesting contrast between perception and reality of presidential family backgrounds.
This is an issue I had with James Loewen. In Lies My Teacher Told Me, he references Edward Pessen's The Log Cabin Myth and said that all but three or six presidents came from "privilege".
So I went through the biographies of every single president and tabulated which came from middle-upper class backgrounds or better, and which were middle class and lower.
The resource I used was the Miller Center's American Presidents Project, which has detailed information on each president, including their early lives, as well as other sources.
The upshot is that about half of them came from privilege, and half did not. Most of the early presidents were part of the early American elite, but as early as Andrew Jackson you had a kid raised by a single mother and orphaned at the age of 15 (several presidents were orphans or raised in single family households, actually), followed by Martin Van Buren, the fourth of six kids, whose parents could could best be described as middle-class innkeepers. (The Miller Center uses the word "struggling".)
In more modern America, you had your presidents born into privilege like the Bushes (their father/grandfather was a senator) and JFK (who said the only experience he had of the Great Depression was what he read in a book), but then you also had Nixon (who got accepted into Harvard on a full ride, and still couldn't afford to go), Truman (whose father had financial difficulties and ended up farming for a decade), Clinton (grew up with a widowed mother), and so forth.
When I confronted Loewen about it over lunch, he sort of flinched and said that he'd "taken some heat" for it, and was just referencing Pessen, so blame him instead.
Richard Nixon's family was poor. Their family business(ranching) failed when he was still a kid. Worked in the subsequent family business, a grocery/gas station. 2/4 of his sibblings died from illness so he was left to pick up the slack, giving up sports and stuff so they could keep the store running.
It's also why he was initially so pro-welfare. He supported the Family Assistance Plan heavily, which would give basic income to poor families, because his family could have made use of it. Conservatives thought the program was too expensive, liberals and humanitarians didn't like the work requirements or "stinginess", so both sides shot it down. Nixon couldn't wrap his head around the welfare system during his presidency, all he could picture was his childhood with a hardworking but still poor family that he wanted to help. His definition of the underclass was hard working but still poor, and thought welfare aid would compliment a long workweek and temporary until the family's finances were sound. He was more cynical as he got older when he saw "welfare queens" that would live off of welfare and not work.
Andrew Johnson would certainly meet the criteria of below average income. He was born in a simple cabin, he was a indentured apprentice to a tailor(when he ran away the words used int he adverts looking for him were "legally bound"), headed West, and eventually started his own Tailoring concern while making real estate investments. He could be described as more or less a self-made man, although his wife helped him improve his writing and math skills, which no doubt helped him immensely.
A related question is are they all rich by the time they run for president?
As for the number of presidents who grew up wealthy, I researched the individuals elected during the 20th and 21st centuries, and where they attended college. Of the 18 presidents, 14 attended a top ten ranked university where tuition and lodging costs far exceed the national average. Only three of the 18 attended small public schools - Warren Harding (Ohio Central College), Lyndon Johnson (Southwest Texas State Teachers College), and Ronald Reagan (Eureka College). Harry Truman is the only president serving in the 20th and 21st century who did not attend college.
The remaining fourteen presidents attended Harvard, Yale, Stanford, Princeton, Columbia, and Amherst. I realize it is unfair to generalize an individual's economic standing by alumni status, but to successfully reach the position of president contemporarily requires attending a school that provides a degree with a pedigree.