The three crucial points in this issue are the legality of Macau was disputed, the Portuguese never asserted full control of the Macau people, and that the Chinese government wanted Macau to be a gateway to the West.
Macau was surrendered to Portugal by the Treaty of Peking, and in that treaty, different interpretations occurred with both sides claimed Macau's sovereignty. But the Macau was a de facto Portuguese territory.
After the Carnation Revolution, the new Portuguese government admitted that the legal status of Macau is not a colony, but a rented Chinese territory as their treaty with the Qing Empire stated. So in a purely legal perspective, Macau is no longer a colony from that point.
They did try to return Macau to China but Chinese government rejected. The reason is that the Chinese government needed a territory that has close ties to both China and the West in order to do trade with Western countries without using China's name. Compared to Hong Kong, Macau was a better choice because
UK was declared China's main enemy and Sino-British relationship was very bad. Portugal, on the other hand, did not suffer too much from the daily Chinese propaganda.
Portugal never had full control of Macau. Unlike Hong Kong, Macau had close political and economic tie to the Communist China from the very beginning. During the WWII and the earlier years of communist reign, Macau provided critical supply of money laundry and rare items from the West (e.g. medical supply) Many people in Macau still identify themselves as Chinese over Portuguese (or Macanese). While in Hong Kong... Well that was another story.
Today you may see china vases that had the label of "Made in Macau". They are the ones made in the Communist China and then sent to Macau for exportation. These items provided the important foreign currencies that China needed.