It seems that these complex infantry formations started to came in part in response to Cavalry but similarly in South Asia Mughals, and in China Qing were a cavalry based armies, so why didn't something unique appeared on the battlefield to challenge that even though these regions did have access to gunpowder weapons.
I shall try and provide an answer for South Asia.
To answer this question we need to first consider the time period here, which according to your question seems to be focused on the mid 16th century to 17th century. Therefore, to understand why the formations, tactics and weaponry used by South Asian kingdoms and Empires during this period developed in such a unique fashion, we need to look at the military systems and traditions of the states that existed and competed for supremacy in south Asia during this time period. Since it is by looking at their conflicts, their experiences and solutions to pressing military matters that we can answer the aforementioned question. To simplify this task, we shall look at three states or cultures, the Rajputs in the Rajputana, who were major contenders for power in North India before their Confederacy suffered a major defeat at the Battle of Khanwa in 1527 by the founder of the Mughal Empire Zahir-ud-Din Mohammed "Babur", the Afghans who after the defeat of the Afghan Lodi dynasty at the First Battle of Panipat 1526 reunited and remerged as the strongest contender for supremacy in North India under the Sur dynasty and finally the Mughals, who would subjugate both the nations/cultures and emerge as the state which controlled all of North India, modern day Pakistan, Afghanistan and Kandahar by the end of the 16th century.
THE RAJPUTS
In the beginning of the 16th century, the Rajputs had been divided into several kingdoms, each ever vigilant against the other and seeking to exploit the other's weaknesses and expand their border and influence at the other's cost. However, the actions of one state and it's ruling house would change these attitudes and provide these scattered states and petty kingdoms with a united purpose and strong leadership. This was the House Sisodiya of Mewar.
This house of Rajputs traces it's origins from the legendary Suryavnshi lineage. But while records to back up such claims are obviously questionable, the historical foundation of this dynasty lies in the rise of Rana Hammir Singh, the founder of the Sisodiya Cadet Branch of the Guhila dynasty. The Guhila dynasty was extinguished by Alauddin Khalji, Sultan of Delhi, after he besieged and conquered Chittor in 1303, their capital. But Rana Hammir Singh had taken back Chittor and since then reclaimed control of the region and re-established the dynasty under its cadet branch of the Sisodias by 1326. Owing to the legendary exploits of their kings and being one of the few Hindu noble houses that had remained independent during the successive reigns of various dynasties at the helm of the Delhi Sultanate, the House of Mewar carried weight amongst Rajput nobility.
Apart from Rana Hammir Singh, two rulers in particular, Rana Kumbharna Singh (1433-1468) and his great grandson Rana Sangram Singh (1508-1528), had raised the prestige of the House of Mewar to astronomical heights by not only defeating neighbouring Sultanates in Gujurat, Nagaur, Delhi and Malwa, but infact under the reign of Rana Sangram, actually conquering Gujurat and Malwa. Therefore, by 1526, most Rajput states had formed a Confederacy under the leadership of Rana Sanga.
RAJPUT MILITARY SYSTEM :
By the 13th century, the Rajputs were well entrenched into a feudal system of civil and military organisation. The land of a kingdom was divided into villages headed by rais, who were themselves subordinated to either individually or in groupings known as thikana to thakurs which initially was a term which referred to Rajputs or Kshatriyas exclusively, but eventually became a title and was used by Brahmins of the same feudal rank. Next were Samantas special ministers appointed by kings to administer a number thikanas or to serve in special roles in the administration. There were also a number of other feudal lords and chiefs, subordinated to the Raja, and several Rajas would be further subordinated to a Maharaja or a Great King.
The army itself was raised out of 6 military labour pools namely :
Maula or hereditary troops
Bhrita or regular forces/mercenaries
Sreni or troops supplied by guilds, towns etc.
Mitra or Suhrd or forces of allied states
Dwisad or Ari or soldiers alienated from the enemy camp, by bribery, or recruited after defeating the enemy
Atavika forces raised out of tribal groups such as Bhils
Among these groups the most important pool of military labour is Maula. This refers to those soldiers whose previous generations have owed loyalty to and have shown exceptional service towards the king, who in recognition of these services appoints them as part of his Maula troops. These troops were also often drawn from close and distant family members. For example, a king might have many brothers, who might have many sons and those sons might have married into families and those families might have several capable men of fighting age as well. The king would appoint men from this labour pool into his Maula force as he saw fit and each soldier would be replaced by a brother or son when they died. Hence making them hereditary troops both in terms of their replenishment and the fact that each successive generation inherited these troops. These men, being close friends and family of the king, meant they could be given independent charge, could advise him in war and could be appointed as commanders of any number of units.
I shall not go into the details of military organisation of the Rajputs any further, since that is not the concern of this answer, however, to give a brief overview, units were organised into 5, 30, 100, 1000 and 10,000. There did exist a corps of high teir officers to look after the management of the logistics, law and order and command of the entire army. Armies until the 11th century were composed of elephants, infantry and cavalry.
Between the 8th and 11th century, the last Brahmin Empires and kingdoms of North India such as the Pratiharas, the Shahis, the Sindh dynasty, had struggled against and eventually faded out due to the repeated incursions of Arab forces. And while the Arabs could never break through the Brahmin kingdoms to reach into the North Indian plains, the invasions of Muslim powers continued under Turkic and Iranian dynasties such as the Ghaznavids and the Ghurids, until eventually, the Delhi Sultanate emerged as a Muslim power in North India in 1206. Between the 13th and 14th centuries, the continued conflict of the Rajputs with the successive dynasties that ruled over the Delhi Sultanate forced the Rajputs to adopt to their enemies and their new style of warfare. The Rajputs began to change the composition of their armies from poorly armoured but highly skilled infantry, elephants and light cavalry, to cavalry heavy armies or cavalry exclusive armies.
The concept of Rajput chivalry, prowess and honour was closely associated with horsemanship and effectiveness as a cavalryman. The Rajputs relied more and more, on outmaneuvering the enemy, on heavy cavalry charges, to swoop the enemy off the battlefield and to rout and destroy their numbers. The typical Rajput campaign as it came to take shape in by the mid 14th century during the campaigns of Rana Kumbharna Singh, was one where the Rajputs utilised their mobility to surprise the enemy and choose a battlefield of their taste, then waiting for the enemy to take positions on a plain battlefield well suited to charges, and then charging along the entire front or in some order, to defeat the enemy.
The Rajput cavalry of the period, covered the heads of their horses with artificial elephant trunks to make it seem to the enemies elephants as if a horde of young elephants were charging towards them, thus discouraging any attacks from the beasts. The typical equipment of a Rajput cavalryman in the period consisted of mail armour for the torso, covering the arms as well, a steel helmet, two tulwar swords ( single edge or single+false edge curved swords with disc hilts and bulbous grips, an native innovation and an answer to the Persian shamshir), or a tulwar and a Khanda (a double edged broad sword, with a basket hilt, and bulbous grips), along with axes, maces, and eventually in imitation of their Turkic enemies, bows and arrows. Depending on the income and status of a cavalryman, more complex armour such as plated mail, plate gauntlets and greaves, chest and back plates etc were also a feature.
Therefore, the military system of the Rajputs as it existed in the 16th century, was a result of 3 centuries of conflict with the Delhi Sultanate, it's armies which were composed of infantry, elephants and Afghan heavy cavalry and the attempt of Rajput kings to devise solutions to counter the threats they faced on the battlefield.