As others have pointed out, the word "Hindu" was originally a demonym (and an exonym by the Persians). Till the early British era, you could find writings that mentioned "Hindoo Christian" people (try a Google Books search for this term) -- "Hindoo" referred to the person's ethnicity, while "Christian" referred to his/her religion.

In ancient India, there were a number of religious and philosophical traditions as well as animist tribal cults, which were later coalesced under the term "Hinduism" to distinguish them from Islam and other foreign religions. Syncretism gradually led to the development of a coherent "Hindu" identity. As for the Mauryans:

No concrete numbers

The chief aim of the Mauryan census was to gather information for taxation purposes: it was mainly centered around collection of occupational information. We don't have any concrete numbers on the population's religions.

The contemporary texts (native and foreign) are not very reliable either, as they are full of inaccuracies, exaggerations and propaganda. For example:

• The Buddhist text ''Ashokavadana'' claims that Ashoka built 84,000 stupas (an obvious exaggeration) and boasts about massacre of 18,000 non-Buddhists by Ashoka (a common tactic used by ancient Indian sects to prove their dominance over other contemporary sects).
• The Jain texts of Tapa Gaccha claim that Samprati constructed 125,000 new temples and renovated 36,000 old temples (another obvious exaggeration)
• Greek records claim that certain Indian tribes worshipped Herakles and Dionysus, who is also claimed to have visited India (obviously not true)

Guesses

Chandragupta Maurya's period:

• Vedic practices were dominant, while Buddhism and Jainism were mainly seen as monastic sects / doctrines.
• The Greek writings based on accounts of Cleitarchus and Megasthenes mention two categories of Indian philosophers: Brachmanes (presumably Brahmins, followers of Vedic religion) and Germanes / Sarmanes (possibly Sramanas - Buddhists, Jains and Ajivikas). The Brachmanes were held in highest regard, indicating their dominance. No direct references to Buddhism and Jainism are found in accounts of Cleitarchus or Megasthenes.
• Similarly, the Arthashastra doesn't mention Buddhism and Jainism (at least directly), indicating that these sects weren't prominent.
• Chandragupta Maurya is believed to have died a Jain ascetic. However, a few scholars believe this to be Jain propaganda, and state that the evidence is not convincing enough. Anyway, Jainism was probably seen as something that old people take interest in, at the time of sannyasa.
• Buddhism was most probably confined to Kosala and Magadha, with small communities in Mathura and Ujjain. The accounts of high attendance in the pre-Ashokan Buddhist councils are most probably exaggerated.

Bindusara

• Bindusara patronized Ajivika, but not with the same zeal as Ashoka favoured Buddhism.
• Limited by the ancient means of communication, this patronage probably didn't translate into a mass adoption by his citizens. But, going by the later Buddhist accounts, the Ajivikas numbered in thousands (if not more) in Pundravardhana (North Bengal) alone.

Ashoka

• Under Ashoka's patronage, Buddhism became an actively proselytizing religion.
• The geographical spread of stupas, inscriptions and literature indicates that Buddhism spread far and wide.
• It had a direct appeal to the common people, who weren't great fans of the traditional rituals and sacrifices. Whether it was limited to the educated/royal/trading urban elite or managed to become a religion of the rural masses is anybody's guess.

Post-Ashoka

• Buddhism definitely remained a major force. This is indicated by the fact that several post-Mauryan kings were Buddhists, including neighbouring foreigners such as Menander I (Milinda) and Kanishka.
• Ashoka's descendant Samprati is believed to be the "Jain Ashoka". Jain proselytism is beleived to have reached its peak during his reign. Several merchants, especially those in Western India, were Jains during his period. Again, it's hard to say anything about the common masses, though.

Age of the Nandas and Mauryas by KA Nilakanta Sastri has a chapter on Religion. The Mauryan Polity by VRR Dikshitar also has a chapter on religion: it has a certain right-wing Hindu bias and stops just short of claiming that the Buddhist claims of Ashoka's patronage are fabrications. A good read for the sources it mentions, nonetheless. Megasthenes and Indian Religion by Allan Dahlaquist has Megasthenes' accounts in Greek with English translations (skip the fringe parts like "Krishnaism and Christianity", though).

Not sure about percentage of whole India but majority of population of present day Pakistan (historic part of greater India) was Buddhist based on number of Buddhist archaeological sites such as Temples and Stupas found in Pakistan from that period. But at the time Hinduism was not a strict religion rather a collection of different religious practices for example at the time a person may be practising completely different religious practices from a person in another part of the country but both will be called Hindu as Hindu used to be a geographic nomenclature for people of India rather than a religious one at the time.

As a related question, how fluid were the categories of Hindu, Buddhist, and Jain during the Mauryan period? Were they viewed as distinct religions as they are today?

The word Hindu is nothing but Persian word for Indus from sanskrit Sindhu(as Persian pronunciation of 'H' in place 'S'), Its has no relevance to Hinduism other than classification which was started by British colonials to categorize , India in general was mostly about philosophies or schools, There was no distinction between Hindu Buddhist and Jain only in modern sense, But people always accepted different schools of thought which all paths lead to same thing. Buddhist and Jain are mainly thought as different school of philosophy just like Advaitha , Vaishnavism.