So this is a bit outside of my geographic area, but it is within my time period/thematic wheelhouse to some extent, and since you haven't gotten a good answer to what seems to be a very important question for you personally and not just historically, I did some research to see what I could come up with.

The legal status of homosexuality in Portugal varied during the period of its history prior to the establishment of Salazar's regime (the Estado Novo) in 1932. Homosexuals had been persecuted during the era of the era of the Inquisition (from the 16th century to the early 19th century), but homosexuality was decriminalized in 1852. However, it was recriminalized again in 1886, carrying a three-year prison sentence. The law didn't explicitly state which sexual acts were illegal; the 1886 and 1912 laws that were used to prosecute homosexuals simply referred to "addictions against nature", which also encompassed other undesired sexual acts like prostitution. The Estado Novo didn't make any sort of changes to the existing laws criminalizing homosexuality until the revised Penal Code in 1966 specified a two-year prison term for sexual assault against someone of the same sex.

It's worth pointing out that from a legal and ideological perspective, the Estado Novo was somewhat different from the archetypal fascist states like Italy, Nazi Germany, and Francoist Spain. Unlike Hitler and Mussolini, Salazar was a deeply religious man and he rejected the mainstream fascist ideologies, which he viewed as irreligious (and in the case of Nazi Germany, paganistic). The 1933 Constitution, which established the Estado Novo, created a corporatist system of government based on Catholic social teaching, in which society was organized into different interest groups ("corporations"). The goal of this system was to essentially eliminate the possibility of class conflict and create a society organized primarily around social rather than economic interests. The new constitution included a fundamental right for laborers to organize and an extensive social welfare system. Salazar's ideal was basically an apolitical society, rather than the type of mass-party political mobilization practiced by Mussolini and Hitler. It's also worth noting that the Estado Novo didn't really subscribe to the type of racial-biological nationalism that animated Nazism and other fascist movements, and that Jews weren't systematically persecuted in Portugal; in fact, Portugal was a vital transit point for refugees fleeing from Nazi-occupied Europe, as was famously portrayed in Casablanca.

Of course, despite its ideological differences from Italy and Germany, the Estado Novo was nonetheless a strongly socially conservative, nationalist, authoritarian, and dictatorial regime, which harshly repressed its political enemies and people it deemed to be socially undesirable according to Catholic religious doctrines of the time. The repressive nature of Salazar's regime really became accentuated in 1936, in response to the regional instability generated by the Spanish Civil War. There were quite a few legitimate threats to Salazar's fledgling regime during the mid-30s, including both monarchist and leftist opposition groups (the latter supported by the Spanish Republican faction). Salazar's secret police force, the PVDE (which later became the PIDE) was highly active in repressing perceived political enemies, as well as undesirable social elements, which included homosexuals.

Salazar's official public position was that homosexuality didn't exist in Portugal, believing that that which was not discussed in public couldn't really exist. Any newspaper, magazine, or other publication that mentioned homosexuality was censored. However, despite Salazar's claim that homosexuality didn't exist, the PVDE (and later the PIDE) were active in persecuting homosexuals in Portugal from the early days of the Estado Novo. The police raided known gay bars and clubs, as well as private homes of those suspected of homosexual activity. These people were often taken to black sites known as mitras and beaten or tortured. Both gay men and lesbians were actively prosecuted and imprisoned or sent to forced labor during this time. The police also engaged in entrapment of gay men, posing as members of the gay community and then arresting them. Some homosexuals were sent to the concentration camps that Salazar's regime operated. Historians have documented cases in which private citizens denounced other private citizens to the PVDE/PIDE for various crimes, presumably including homosexuality, which would track with the relatively conservative social attitudes of most Portuguese during the Salazar regime and reflects the general hostility of society toward the gay community at that time.

After World War II, Salazar took on a more conciliatory tone in his government, promising political liberalization, including protections for free speech and fair, multiparty elections. In addition, during this time, the scientific community's views on homosexuality in Portugal began to shift; the country's most prominent medical professional, the neurologist Antonio Egas Moniz (who won the Nobel Prize in Medicine...for inventing the lobotomy, not the Nobel Committee's finest moment), had publicly stated that homosexuality shouldn't be treated as a crime to be prosecuted, but rather as a disease to be treated and cured (which was a common belief in the medical community well into the 1970s). Obviously these attitudes seem regressive now, but they were par for the course in the postwar period.

However, despite some nominal liberalization measures, in reality, Salazar's regime remained dictatorial and repressive until and even after he stepped down in 1968. If anything, the PIDE was more ruthless and efficient than the PVDE had been, both in sniffing out potential pockets of resistance to the regime and in repressing those the regime deemed to be socially undesirable. The previous repressive policies toward gay people continued, including raids on gay establishments, arrests of individual gay people, as well as torture and extrajudicial detainment, even after Salazar was replaced by Marcelo Caetano in 1968. Even after the Carnation Revolution in 1974, gay liberation didn't immediately follow; homosexuality wasn't decriminalized in Portugal until 1982. It should be said though that nowadays Portugal has some of the strongest protections for LGBT rights in the entire world and that the public attitude toward homosexuality is very liberal, which is an impressive amount of progress in the space of 40 years.

As far as your grandfather is concerned, I obviously can't really tell you anything. I don't know anything about him beyond what you've said and I can't go do archival research to look at his service records or any other type of empirical evidence. All I can really tell you is that those types of things--police raiding gay bars and clubs and beating and torturing gay people--did occur during that period of Portugal's history.

Sources:

Felipe Ribeiro Meneses, Salazar: A Political Biography (Enigma, 2009).

Avraham Milgram, Portugal, Salazar, and the Jews (Yad Vashem, 2011).

Sasha Roseneil, et al., The Tenacity of the Couple-Norm (UCL, 2020), p. 96.

Duncan Simpson, "Approaching the PIDE from Below: Petitions, Spontaneous Applications and Denunciation Letters to Salazar's Secret Police in 1964," Contemporary European History 30, no. 3 (August 2021): 398-413.