In some sense, the Polish-Lithuanian Commonwealth DID have a "colonial empire" in the sense that it established peripheral subject territories in what is today Romania and Ukraine. The PLC was not a very centralized state, by comparison to it's contemporaries in Western Europe, or even to Russia.

However, I don't think it's correct to suggest, as /u/Pseudohistorian does below, that the lack of access to the global ocean is a useful criterion for explaining or defining colonial empires. There have been plenty imperial polities that extended their reach to their nearby neighbors rather than across the oceans, as the geography dictated.

Beginning in the late 17th century, both Russia and Qing China (and to a much lesser extend Safavid Persia) were engaged in project of imperial expansion into Central Asia, with both resource and settler colonialism taking place. Much of the Russian conquest of Siberia and the Chinese conquest of what is now Xinjiang, fit within the idea of colonial empire-building on a infra-contientnal, rather than trans-oceanic, basis.

One of the reasons was geographic- county simply did not have open access to high seas. The only entrance to sea PLC had was Baltic coast, hence all naval activities outside it would depend on the good will of Denmark.

Secondly, bout- Lithuania and Poland- were rather landlubber nations. While Baltic tribes such as Prussians and- most notably- Curonians had a naval tradition rivaling Scandinavian Vikings, Lithuania did not inherited that tradition, and for most of it's history did not even had a access to the sea- GDL had (relatively) only town of Palanga, wedged between Livonia and Teutonic Prussia. Lithuania short-time control Black sea coast is even less of a thing to speak about (albeit, Vytautas the Great did funded a port there).

After wars with Order ended, towns of Palanga and Šventoji established themselves as trading ports, temporarily even rivaling Riga, however the trade activities were maintained mostly by foreigners (privilege to build port in Palanga was given to English merchants in 1589) and all that was destroyed during Deluge.

I can not speak with confidence about Polish naval history, but Poland more or less lost access tot he sea during Fragmentation period and did not regain it until Peace of Thorn (1466), hence story could not be much different to them.

Third, PLC was not centralized state. England, Spain or Portugal had a strong central government to push for establishing and maintaining colonial empire. Commonwealth went another direction on this regard, as we all know- Crown's power was constantly checked by nobility, Lithuanians were constantly rivaling Polish etc., and in latter centuries situation become more and more anarchic.

Meanwhile Tatars incursion were still a thing, Lithuania had Mexican stand-off with Sweden and Muscovy over Livonia that was resolved only in early 18th century- in general Commonwealth had plenty of land-based problems and little of powerful governments to start colonial adventures.

P.S. Jacob Kettler, Duke of Courland and Semigallia, ruler of small Commonwealth vassal state in Livonia established a small colony in mouths of the Gambia river in 1651, but it was taken by English after a decade. Another colony was founded in Tobago, but it was soon abandoned, then reclaimed, lost during war, regained via diplomacy, abandoned again and unsuccessfully attempted to reclaim for a third time. But that was Jacob's personal affair, nothing more.

Our vassal had colonies in Gambia and Tobago, does it count?