Well this is certainly an interesting and fraught answer. I will be very cautious with how I word this one and would advise you to be as well if it ever comes up. First up Śiva is only the most powerful god in the Śaiva branch of Hinduism, though he is important in the Śaktaḥ and Smārta traditions as well. Most powerful god is kind of a weird thing to discuss in modern Hindu thought since what we think of as Hinduism is a bevy of VERY different religious traditions GENERALLY originating in the Vedic religion, some of which are self‐consciously "monotheistic" and most of which are to some degree monistic.

That said, Śaivists are either the largest or second largest group of Hindus counting all of the various and extremely diverse sub‐sects from orthodox Śaiva to the Bhakti focused Nayanars to the iconoclastic and Veda rejecting Lingyats to the subversive Aghori to the almost Buddhist (and nearly extinct) Kaśmīri Śaivism, and that's not even half. In any case, Śiva is generally a pretty big deal both in and outside of India so you are right, but only kind of.

Anyway, lets go back –waaaay back– to a time before Śiva, before Hinduism, hell before the pyramids. In present day India and Pakistan, along the now typically dry Gaggar‐hakra river lived a civilization which we have somewhat confusingly called the Indus Valley Civilization. They were the first major civilization in the region. But they aren't really relevant to the story of Śiva so redirect your attention to the Pontic steppe and meet the Yamnaya horizon people, who are probably the Proto‐Indo‐Europeans. Okay, okay, I probably need to discuss one thing that you may hear about the IVC people and that is the Paśupati seal.

So Paśupati is one of the many, many titles for Śiva and means something along the lines of "Master of Animals.". In 1928 a man named Ernest Mackay uncovered the seal at Mohenjo-Daro and it was sent to John Marshall, archeological director of British India. He saw the figure on the seal as "the prototype of the historic Śiva." His reasoning is listed from pages 52 to 63 of his report, but to sum it up, he sees the posture as "yogic" and notes that Śiva sometimes has three faces. Needless to say, scholars generally reject this today, but the name stuck. The reasons will probably become obvious later on, but suffice to say that the antecedents of Śiva will have far less a direct connection with the IVC than was speculated at Marshall's time.

But moving back in time again and to the Yamnaya. The Proto-Indo-European people were, as their name suggests, the ancestors of the Vedic people though at this point quite distant. These people hold a particular myth involving a god who wields a club infused with lightning who kills a serpent representing chaos. The god became a hero sometimes and the head of the pantheon elsewhere. This myth spread to the Semitic peoples and most famously is depicted in YHWH's slaying of Leviathan or Herakles slaying the hydra. This hero‐god is powerful and often leads a pantheon. Later, he may have acquired characteristics of fertility and masculinity.

A few thousand years down the line he became known to the Indo-Iranians of the Andronovo cultural horizon as Indra. Here we also see the development of the fire cult which in the South will coalesce into the worship of fire personified as Agni. Agni is the god of fire, but also of light, sacrifice, and transition. Shortly after around 1600‐1500 BCE, Rudra appeared.

Rudra is now an epithet for Śiva generally and is typically used as the fierce aspect of Śiva by his devotees. In the Ṛgvēda though, Rudra is somewhat different. In early layers he carries a bow and is the bearer of medicine. He is fierce and powerful, ruling others through violent storms and being the father of the Rudras. He is terrifying and capricious, but dedicated worship could bring fortune in battle or cure disease. In the late Vedic era, 1200-1000 BCE, the Vedas became almost monistic and developed a strong focus on a supreme deity, which was sometimes Indra, sometimes Agni, sometimes Rudra, but typically Iśvara and later Prajāpati. From the late Vedic to late Mahājanapadā era the principal Upaniṣads were written an further the monistic tendencies developed in the last layer of Vedic Saṁhitā texts.

This is all very well and good, but what do Indra, Agni, Iśvara, and monism have to do with Śiva? A lot actually. Since the Gupta period, when Hinduism took on a far more recognizable form with the composition of the Āgamas and the Purāṇas, Śiva has principally been associated with Rudra, but he bears just a much resemblence to other Vedic deities. This isn't to say Rudra was not a major part od the development of Śiva, he was but he wasn't the only component. Scholars such as Wendy Doniger have linked Śiva, especially as depicted in the Purāṅas, with Indra. Like Indra, Śiva was frequently depicted as ithyphallic and associated with male fertility. His position and rank as well as association with the universal mantra AUṂ (rendered frequently as Oṃ) and portrayed as slayers of ignorance. Agni is another component, being portrayed as the root of things and in late Vedic texts is identified as Rudra especially in the form of fire.

The early ideas of a supreme god began with the middle Upaniṣads and late Vēdās in the form of Iśvara and later Prajāpati. Prajāpati is described as Brahman itself or as the origin of all things in the principal Upaniṣads. Like Śiva, Prajāpati is said to emanate the world from his forehead, much like Śiva's use of his third eye.

After the Śramaṇas (Buddhists, Jains, Ājīvikas, Cārvākas, and so on) the late Upaniṣads began to utilize Śramaṇic ideas and terminology within a Vedic framework as well as developing new means of competing with these "heterodox" traditions. Two of the major works of this period are the Maitrāyaṇīya Upaniṣad and the Śvetāśvatara Upaniṣad. The former is possibly a direct response to the challenges presented by Buddhists and contains a polemic attack against them while utilizing the language of suffering and the emphasis on meditation versus ritual common to Śramaṇic texts. While the Maitrāyaṇīya Upaniṣad advocates no particular deity and can be interpreted as non-theistic, the Śvetāśvatara advocates a position of monotheism. To the composer(s) of the Śvetāśvatara, all is contained in Prajāpati who is Rudra, known to be kind (Śiva). While both texts bear Yogā-Sāṃkhya influence, they advocate a very different metaphysical view of gods. Futhermore, the Maitrāyaṇīya advocates a non-dualist position while the Śvetāśatara advocates a dualist view of reality as is common with the Sāṃkhya tradition. However, this was not nearly the problem it sounds to be. Among advocates of the Sāṃkhya tradition (and indeed others) the apparent contradictions were only surface level, one could interpret the divisions of Brahman in the Maitrāyaṇīya as the Puruṣa and Prakṛti, or perhaps the three divisions outlined as the three divisions of Prakṛti. Furthermore, just because the Sāṃkhya orthodoxy, if such a thing could be said to exist, advocated non-theism didn't mean that all those who drew influence from Sāṃkhya believed that. In fact, the Yoga Sūtras of Patañjali which formed the basis of later Yoga practice (Yoga here being the philosophical school of practice in India, not the Haṭha yoga of Mathsyendranath) firmly advocates a monotheistic position where god is identical with the jīva or soul.

It is probably from this Yoga tradition that the first Śaivas emerged. We have no idea exactly when, but we can say probably post-Śramaṇic but pre-Aśoka the Śiva cult emerged out of the fusion of deities into Rudra who was gradually moved from his initial name to his reflex name of Śiva. Given the evidence we have, I would place this cult's development in Northwest India based principally on Numismatic evidence. Granted, this is pretty poor evidence as we have no real way of seeing the development of Śiva, but it is something.

The earliest philosophical school that we know to have advocated and systematized Śaiva doctrine was the Pāśupata who were an ascetic cult who were know for covering themselves in ashes. They appear to have been a west Indian sect and were famous as opponents of the then more popular Vaiṣṇava and Bhagavata cult. However, the invasion of the Kuṣana in the 2nd century seems to have increased the prestige of the Śaivas and they proliferated throughout northwest and west India, where Kuṣan influence was greatest. Śaivism spread to Tamil Nadu some time later and became popular, especially through worship of Skanda, his son. Around the same time, the cult spread to Bengal where it became a major rival to Buddhism, eventually replacing it almost completely by the 12th century.

However, in the post Kuṣan era, most dynasties seemed to favor Viṣṇu, especially with the composition of the Puraṇas during the Gupta period. From the time of the Guptas to the Muslim invasions, Viṣṇu seemed to hold the most royal support among the Central Indian rulers while Śiva worship existed on the fringes. This is not to say that there were not rulers in Central India and the Deccan that were Śaivites, but that the majority of royal houses were dedicated to Viṣṇu. Famously, the Hūṇās were Śaiva during their conflict with the Guptas and Mihirakula was said to have been especially devoted to the destructive aspect of Śiva which fueled his persecution and destruction.

In any case, these as the periphery grew in importance, so did Śaivism. Today Śiva is most popular in Nepal, Tamil Nadu, Karnataka, Bengal, and Bali.