My question has two parts:
How was a successor chosen in the Ottoman Empire? I've heard various explanations, most of them suggesting that there wasn't really a standard procedure. Is it really true that a new sultan would frequently have to kill many of his brothers to ensure his place? Was there ever a period in Ottoman history where a standard succession law was put in place?
I have read that potential heirs, claimants and deposed Sultans would live in "the cage"--a part of the imperial palace which basically served as a comfortable prison where they would be isolated from the world and under constant surveillance by palace guard. How many sultans came out of this environment? How did they develop the skills needed to lead an empire without any real experience governing territory or leading armies (or even interacting with humans other than residents of the palace)? What was the point of it--did it come about due to the disorganized nature of Ottoman succession?
How was a successor chosen in the Ottoman Empire?
From what I have learned at university, there was actually a quite standard procedure for choosing a new Sultan. A Sultan had a harem which was filled with some of the most beautiful women of the empire, whom he could choose to have sex with. These women would not be with any man except the Sultan and were generally effectively imprisoned in his palace. This was done to make sure succession was clear, and that no bloodline except that of the house of Osman could become sultan.
Is it really true that a new sultan would frequently have to kill many of his brothers to ensure his place?
Yes very much so, it happened at almost every switch from one Sultan to the next. The women from the harem gave birth to a lot of boys who were trained to be Sultans. After the reigning sultan died, the princes would rush to Istanbul try to get the support of the Imperial Guard and Royal Court and the first one to do so would become sultan. He would then kill off all, or the vast majority, of his brothers to ensure that nobody had a claim to the throne.
I have read that potential heirs, claimants and deposed Sultans would live in "the cage"--a part of the imperial palace which basically served as a comfortable prison where they would be isolated from the world and under constant surveillance by palace guard. How many sultans came out of this environment?
This change largely came in the 17th century, where the practice of killing brothers was discontinued. I'm honestly not sure why they stopped as it seemed effective enough. You can say that the majority of sultans after this period came out of this "golden cage"
How did they develop the skills needed to lead an empire without any real experience governing territory or leading armies (or even interacting with humans other than residents of the palace)?
They didn't, simple as that. Due to the degree if inbreeding and lack of stimulus found in the cage, some princes tended to become very unstable. The earlier sultans were placed out in the world were they would learn how to rule, and each time there was effectively a very small-scale war of succession between the princes. However, this new system of having the sultan have sex with a large amount of women, pick the boy who seems the most promising and kill the rest removed possible disagreements between princes and caused a huge lack of experience or ability in the sultan. This meant the power of the Grand Vizier increased, as he had executive authority in the name of the sultan and was often the effective ruler during the reigns of especially weak or ignorant sultans.
What was the point of it--did it come about due to the disorganized nature of Ottoman succession?
The "golden cage" did ensure that succession was clear, as a new sultan (however ignorant he was) was chosen and all the other princes killed, together with their mothers to ensure that nobody could ever have a claim to the throne. However, while it made the question of succession much easier, it also, as mentioned, meant the new sultans were often inexperienced and could not rule effectively.
Sources: A History of the Modern Middle East by W.L. Cleveland and university lectures