I am assuming mixed-race children in those days were all the product of rape, or is that wrong? From what I read, almost all the white ancestors of African-Americans were male, so I am wondering who these white ancestors were. Were they the wealthy men that owned African slaves, or were they from the other poorer white men that worked on plantations or as household staff?
The Plaçage system may have played a large part in this. This allowed the European men in French and Spanish colonies within North America and Caribbean to enter what was essentially common law marriages with non-european women. The reason for this was due to the shortage of white european women as many people who came to the new world were often male explorers or colonists.
The first women to enter these marriages were either Native Americans or African slaves, but soonafter a class of Free people of color was created for white european men to marry. The men these women would marry were often wealthy i.e Noblemen, planters, military men, However, administrators and merchants were also known to enter Plaçage marriages.
I'll offer up a few cases to look at, as they show that race relations in colonial N. America tend to be a lot more involved then we consider today.
First, Salem in 1653 the case of James Thomas, for fornication with Katherine, "negar" servant of Daniel Rumball, was to pay 18d. a week towards raising of her bastard. The courts charged her with either 40s. fine or to be whipped. Daniel Rumball agreed to pay the fine.
Interestingly, Thomas shows up elsewhere in the records as a servant of Daniel Ray, and to be whipped for stubbornness and disobedience to his master. He is also charged at a much later date for intoxication.
This example places the two of them in a lower rungs of Puritan society, and not in the same household or a master/slave role.
In Boston, these sorts of relationships, where equal social position tends to outweigh race, happened much more frequently. I'll suggest Unfreedom by Jared Hardesty if you're interested in the topic. His work is phenomenal.
The rest of these references are coming from Philip Morgan's Slave Counterpoint and should be considered as quotes. In brief they give an idea of the numbers and nature of inter-racial relationships in the Chesapeake, with his work painting a much different relationship than traditionally considered. I highly suggest it.
In Virginia in 1671, Francis Skiper, a white pays levies and tithes on his wife, "she being a negro".
In the early 18th century, in Maryland, a white woman persisted in a common law marriage to a slave - bearing 7 children.
In the mid 1700s, Sarah Clarke, a white indentured servant, gave birth to "two mulatto bastard children during her servitude."
In the mid-1700s one white woman accused another of being "a negro whore and negros strumpet [who] ... would have jumpt over nine hedges to have had a Negroe." (Off-Topic, this quote is amazing)
In 1769 Mary Skinner of Maryland gave birth to a black child, her husband being a prominent gentleman and proclaiming his disgust at her "polluting" his bed.
1795, Captain James West called off his marriage to Peggy Whitaker of Maryland because her sister had a black or mulatto husband.
1738, Tamar Smith, a white woman, paid a fine and served sixth months in jail in order to marry Edward Hitchens, a mulatto man.
In 1779, Thomas Wright lived openly with his "very black" slave Sylvia, who bore him 4 mulatto children. He eventually freed her, their children, and her two previous children. After his death his oldest son with Sylvia, Robert, inherited the plantation as well as six slaves, and married a white woman.
Obviously, all of these examples are only a small sample of the relationships which occurred - and rape of course would have been a part of these. The argument can also be made that cases such as Thomas Wright and Sylvia might still be seen as non-consensual - after all how can a slave execute free-will? - but the case still stands that our own beliefs and the historical record don't always ally themselves.
Ultimately, the reason for the number you've given (which I'll say I find suspect) are due to a myriad of reasons. There are mentions of slavers raping female slaves during the Trans-Atlantic Voyage, of slaves being raped by owners, of being raped by poor whites, of having relationships with white owners and servants, and even cases where white woman are having relations with black and mulatto men. Ultimately, humans are sexual beings and where they are in close contact with each other or have opportunity to do so, sex isn't an unlikely occurrence.