The short answer is (of course) "it's complicated." It would be so much simpler for historians if domestic family law was consistently and fully coordinated at the federal level, but it's not, and it would be easier for research purposes if family services were strictly governmental, but they never have been in the US. So there's always a hodge podge of different laws and practices and approaches to any area of social welfare.
And of course a lot depends on how you define "orphanage." There are "group homes" for youths still today, of course. ... Basically, the practice of "warehousing" children in orphanages started being criticized in the mid-19th c. with the sentimentalization of the child, and the rise of "placing out"--sometimes at great distances (viz "Orphan Trains") which many view as a proto-foster care system. Most orphanages started dwindling in the early 20th c., but there was a big uptick in the Depression (with parents often using them as temporary housing/board for their children--which has always been the case of orphanages in most countries, and still is today). Most US orphanages closed their doors or evolved into group homes and other youth services in the post War period / 1950s-60s.
Also, by the way, the idea of having orphanages is pretty new--it didn't really happen until the early 1800s--before that displaced children were just lumped in with adults in "county homes" or "the poor house." Or placed out in involuntary indentures. Or if infants, put on "baby farms" where they would most likely die, because pretty much no one was interested in adopting an infant until the rise of the companionate family in the late 19th/early 20th c.
Some good sources: I edited a volume entitled Children and Youth in Adoption, Orphanages, and Foster Care (Greenwood P, 2006)--a reference book written for the general reader, with primary documents and a good bibliography. Online: Ellen Herman, THE ADOPTION HISTORY PROJECT: http://pages.uoregon.edu/adoption/
This question just made me remember going to an orphanage in 1970. It was a big old house just like you'd see in old movies. Then I remember the mid 60's and my neighbor having 2 foster children. From what I recall the orphanages I knew about were supported by churches, like the one I went to in 1970, and the Foster system is government run. So I would have to say for some period they existed together. You can probably be sure the Govt had something to do with eliminating orphanages and churches maybe not getting the donations from parishioners seeing the govt was willing to take on the burden. I'm sure there was government run orphanages in earlier years and that transition to Foster system would have a date. I don't know what date that would be, but answering your question in general was all I actually know and seen myself.