Setting aside modern issues of private military contractors, I am interested in learning more about the time when private, for-hire soldiers were sometimes a major part of a ruler's army, fighting on the front lines and performing most (all?) of the duties of that ruler's "own" troops. First, did such a time actually exist, or is it a fiction? Second, what changed for that to stop being a viable defense policy? Why did mercenaries fade from the battlefield?
Note that this answer is from a European perspective.
Yes, mercenary armies were common from medieval times and onwards. Generally, mercenary forces were well-trained, well-equipped and with a good unit cohesion. They were professional soldiers often hardened by battle and, most important, immediately available.
There's a blurry line between the hird, retuny or personal guard of early medieval societies and mercenaries. The former includes men such as the hard core of the vikings (the hird), gallowglasses in Ireland, the Housecarls in Anglosaxon England, the personal retuny of a knight (his castle's garrison, his men at arms and squires) and the Varangian Guard and Scholae or Tagmata in the Eastern Roman Empire. These are all permanently employed professional soldiers.
As we progress in time armies grow, and the feudal levies of knights and men at arms as well as the personal retunires of the knights are simply not enough - and they tend to come up short against more cohesive units. The Ottomans create the Janissary Corps, a state-provided large professional force, which can probably be called the first European modern army (at least after the fall of the Marian Legionary system). They were certainly superior to anythign the various Balkan states, including the ailing Eastern Roman Empire could throw at them (yes, I know that all Balkan states have one last stand or superb victory against the Ottomans, but the Ottomans tended to be back the next year and win then).
Mercenaries, as opposed to feudal levies, could train and equip in a uniform manner and create excellent unit cohesion. They had no other duties (they did not have to be home in time for the harvest or things like that). The Swiss pikemen became popular mercenaries after their victories against the Habsburgs and the Burgundians (the Papal Swiss Guard stems from Swiss mercenaries used by the Papal States during this era). Likewise, Scottish pikemen became popular mercenaries, as did German landsknechts (often equipped with polearms). Italian Condottieri were extensively used and were mostly superior to the militia the various counties, duchies and cities raised.
Mercenaries continued to make up most of the armies in the early modern era, even if states tried to create national armies. Due to the horrible losses to attrition, national armies tended to be whittled down quickly, and losses were replaced by mercenaries.
During this era, mercenaries also started switching sides when defeated - they had no loyalty beyond that of their pay and their own life and could become devastating scourges to their previous employer if not paid. During the various pauses in the Hundred Years' War, roving bands of mercenaries out of employment would roam the countryside, plundering, pillaging and raping as they saw fit.
The English tried to create a kind of national army with their extensive system to create a class of able longbowmen and it served them well throughout the Hundred Years War, but they still relied on mercenaries a lot.
Both the Dutch and the Spanish used mercenaries during the Eighty Years' War.
Mercenaries were less common in the east, but never unheard of. The Poles relied mainly on their Crown Army - the nobles of the land had to serve as heavy cavalry in the Crown Army, and brough retainers and servants along. The core of this army was the famous Winhed Hussars, heavy chock cavalry that completely over-ran the mostly mercenary Swedish army at Kirkholm in Livonia 1605. The Poles did use mercenaries, especially infantry and artilleyr, during their interventions in the Russian Times of Trouble, while the Russians themselves relied on their traditional militia, the Streltsy that were equipped with an arqebuis and a halberd. The Streltsy were initially a full-time professional force, but lack of equipment, pay and training had effectively degraded them to part-time troops of heraditory status. The Russians also used noble cavalry like the Poles did.
During the Thirty Years' War both the Imperials and the Protestants used almost exclusively mercenaries to fight.
It was onto this scene that Gustavus Adolphus introduced his reforms. Buildin on the reforms of Mauritz of Oranje, he created a cohesive national army with its own supply service, artillery, cavalry, infantry and siege artillery.
This started the end of the mercenary era. While mercenaries would still make up the majority of any nation's army for a long time still, they were from the early victories of Gustavus Adolphus not hired as cohesive units, but as individuals and put into a national system already in place. And when the mercenary, who demanded regular pay and could switch allegiance at the drop of a hat could not provide any significant advantage over a national recruit (since both would be trained in the national system) but the aforementioned disadvantages, he was slowly phased out.
The Swedish army that entered the Great Nordic War 1700 can be said to have been the first truly modern national army that faced off against the best mercenaries in Europe (used by the Danish and the Saxonians) and came out victorious at almost every occassion - and never had better odds than being outnumbered 1 to 1,25.
So, in short. Mercenaries provided a significant advantage over other available forces from the late medieval era to the late 1600s in most nations, and were extensively used. They faded because national armies created systems where their advantages (training, unit cohesion, uniform equipment and drill) could be achieved without their disadvantages (high cost, disloyalty).