I don't know of any experiments but there are several cases of children being brought up in isolation, they are known as "feral children". There are numerous books on different feral children and their stories but one case that might be of interest to you is the case of "Genie".
Genie was subjected to child abuse early on in her life, predominantly from her father who thought she was "mentally retarded" causing him dislike her.
At 20 months old, Genie's dad decided to socially isolate her by keeping her locked in a room tied to a kids toliet ("potty") or to a crib making sure she was completely immobilised. This continued until Genie was roughly 13.
Whilst this was going on Genie's dad forbade anyone (this included her mum and older brother) to interact with her. This caused Genie to not be able to acquire language properly. Because of the surrealism of this unique case it gave psychologists and linguists a good chance to study various aspects of human development including child language acquisition and psychosocial development. The case of Genie led to hypothesises in that we now know there critical periods to acquire language (just before the age of 7 being one of them if you intend on acquiring a second language). Even though she made great progress only months after professionals found her in 1970, she only ever managed to develop nonverbal skills specifically sign language never a spoken language.
However by 1978, Genie's mother suddenly forbade near enough (apart from 1 person) all contact and made the decision for Genie to live with her. This decision was incredibly detrimental in Genie's progress as she began to regress. Genie began to even lose the sign language she'd been taught.
If you'd like me to go in any more detail or if you have an follow-up questions, please feel free to ask :)
Source:
Genie: A Scientific Tragedy ~ Russ Rymer
What you refer to is often called the "Forbidden Experiment", a term coined by Roger Shattuck in his book by the same title which mostly focuses on a feral child found in 1800, and it has been reportedly attempted a few times in the past, all in attempts to discover what language they would speak, usually using deaf-mutes, or else people instructed to not communicate at all.
In "Politics of Deafness" by Owen Wrigley, he briefly notes three monarchs who did so:
Pharaoh Psamtik, who ruled in the 600s BCE, raised two children in isolation, as related in Herodotus. Reportedly, they started speaking Phrygian, (but likely because the person caring for them didn't follow instructions well). The entire story can be found in Book Two of his Histories.
Frederick II is supposed to have done so as well, but the children all died young so there was no result. Little other information is conveyed, and the source seems to be the 13th century Chronicle of Salimbene.
Also said to have is James IV. He did it for political reasons, and the fact the Scottish children started speaking Hebrew was an orchestrated stunt to "demonstrate the ancient origins of his country".