The Polish-Lithuanian Commonwealth incorporated the Grand Duchy of Lithuania into a much larger polity; the map provided by Wikipedia is in this respect inaccurate, as it depicts Polish territory as a Lithuanian holding.

The Commonwealth itself presents an interesting case, however. The terms of its constitution granted extraordinary powers to members of the aristocratic 'sejm', including essentially universal veto powers (phrased as a requirement for unanimous approval of royal proposals).

Though nominally an achievement in safeguarding against the tyranny of monarchs, it had the effect of paralyzing the governance of the Commonwealth by requiring broad political cooperation to effect policy across an enormous geographic range. These constraints on decisive political action limited the Commonwealth's ability to respond to invasions by the Tsardom of Russia, the Habsburg Holy Roman Emperor and the Kingdom of Prussia (an elector of the Holy Roman Empire). The ensuing "Deluge" (so termed in the historiography of the period of rapid territorial cession and eventual disintegration of the Commonwealth) saw the former Grand Duchy incorporated into the Russian Empire. By the time of its first period of independence in the 20th century, the Republic of Lithuania could not effectively claim the territorial expanse of the Grand Duchy, which had itself been a culturally composite polity held not on the basis of nationalism (an anachronistic concept in any case) but rather military potency.

Alas, Lithuania! Consider that, historically, the idea of Lithuania was (more or less) always tied to Poland. If you consider Polish history (and the ravages set upon it by being sandwiched between the Swedes, Prussians, Austrians, and the Russians), you start to get a picture of how things went poorly.

Poland and Lithuania tied themselves together in 1569 (keep in mind, this is almost a hundred years before the idea of the modern state sprang into existence), and the Commonwealth was, relative to the times, a progressive state. Sadly, it was not to last.

The Swedes (as was their wont) invaded and devastated the country during the Northern Wars (middle 17th century). On top of that, the other two great decisive factors in history - plague and famine - descended upon the Commonwealth, further weakening it after the Swedish rampage. It's estimated that something like 40% of the population died during this phase - 40%! It's very difficult for a government - especially a fairly liberal one for the times - to survive that kind of mass die off. It wasn't long before the neighboring powers (who had grown steadily in influence as the power of the government waned) intervened, and we have the Partition of Poland.

After this, most of Lithuania fell to the Russians, who, during the reign of Nicholas I, introduced steady policies of Russification that sought to suppress Lithuanian nationalism (any form of nationalism was of great concern to the tsar and his ministers). It wasn't entirely successful, but you can see how the once proud nation had its identity subverted.

At this time, large numbers of Lithuanians bailed on what was an increasingly bad situation in Eastern Europe (this is the runup to WWI, when Russia became aware of the problems it had on its Western borders), and in their new American homes they re-established their sense of national pride and a desire to have a Lithuanian state independent of the Russians. Fast forward to WWI, and during the chaos that was tearing the Russian nation apart (something something Bolsheviks), the Lithuanians declared independence...but there was still this problem of being neighbors with two (diminishing at the time, but soon to be powerful again) great power countries. The road to Lithuanian independence was a complicated one, especially given that they were dealing with the still-solvent German Empire at the time, who was reluctant to give up territory.

Ultimately, the new Lithuanian borders became the product of haggling between the nascent state and its great power neighbors. Territory lost is rarely regained as a gift from those that took it, and I think the short answer to your question lies in that essential truth.

Speaking about the time before Lithuania joined Poland (the one others have delved in): Keep in mind that most of the territory ruled by the Grand Duchy of Lithuania at its peak, now in Russia, Belarus and Ukraine, was not actually inhabited by ethnic Lithuanians. The Lithuanians through this period lived more or less in the same area inhabited by Lithuanians today.

These other areas had been ruled previously by different East Slavic (Rus') principalities. These principalities were devastated in the 13th century by the Mongols. The vacuum left by the Mongols allowed the Lithuanians to expand and take over these lands over the 14th century.