I think the only places that were kept was Puerto Rico and some island chains other than that nothing else.
At least in the Pacific, the US also took the Philippines and Guam as a result of the Spanish-American War. The Island chains in the Pacific I think you may be referring to are the Marianas and Caroline Islands which were not taken by the US (though they would become governed by the US later and for other reasons). I know little about the Caribbean theatre of the war, so hopefully someone else can address that.
In the Philippines, the change in administration between Spain and the US was immediately followed by the Philippine-American War; the nascent Philippine Republic had hoped that the US would support their independence but quickly discovered that the US had other intentions for their new overseas territory in the Philippines. Skipping ahead to why the Philippines are no longer part of the US; by 1935 the Philippines had transitioned to Commonwealth Status within the United States and on the path towards independence. World War II either interrupted the process, or sped it up, depending on your interpretation and faith in the US to keep its word on independence; and in 1945 the Philippine Republic was reborn. The US maintained a large military presence in the nation until 1991 when the "U.S. Bases Extension Treaty" was not renewed and the US given until 1992 to withdraw from the remaining military instillations. In more recent years the US and Philippines have negotiated a return of US forces to the region, and a potential redeployment to part of the Subic Bay Naval Base- though I haven't been following much how the negotiations went.
Guam became a US Navy governed territory and the residents of Guam became American-nationals. Guam was used a refueling (coaling) port for US navy ships and a couple of smaller bases were built and manned on the island in the early years. The base was important for US power projection in Asia- Guam had provided an important stopping point during the Spanish galleon trade across the Pacific and so it did for the US as well. Unlike the Philippines, the US never had much interest in incorporating Guam into civilian administration and the navy governed the colony until 1950 when the passage of the Guam Organic Act finally gave citizenship to the people of Guam and allowed the election of a local government. It took many deaths fighting for the US against Japan during WWII and active protests against Naval rule before Guam was able to secure local civilian control. Guam is still part of the US today and its motto it "Where America's day begins." Guam still have a very contested relationship with the US military which seized large amounts of the island territory- and still controls a great deal of it (about 2/3s). The bad blood for some started as soon as the US took the island. The early naval governors banned the local language, interracial marriage, and instituted pretty harsh health measures against the local people believing them to be the corrupted, devolved offspring of Spanish interracial mixing.
Otherwise the Marianas Islands and the Caroline Islands; which had been part of the Spanish Empire were sold to Germany after the war. German commercial companies already dominated the economic life of the islands and Germany had previously attempted to claim them. The US was less interested in these islands because they were less developed by the Spanish colonial empire; Guam and the Philippines had been under Spanish rule for over 300 by that point whereas the Carolines and Marianas had felt Spanish rule much less- in fact the Carolines had been left virtually alone until Germany began poking about and Spain had to shore up its control over the islands to maintain the illusion of sovereignty over them. Guam was an important coaling station and naval base and the Philippines an important economic link to East Asia- the other islands were not seen as terribly important- and Germany already dominated them to a great extent. The US had less commercial interest in the burgeoning copra trade that was taking place in the region than Germany did.
The Marianas and Caroline islands were later both claimed by Japan during WWI. As an ally of the British they moved against Germany forces in the Pacific and captured these island regions among others and after the war the League of Nations recognized a Japanese mandate over the former German colonies.
After the Second World War both the Carolines and Marianas island did come under US jurisdiction as part of the "Trust Territory of the Pacific" which the US controlled up until the 1980s. So eventually the US is in fact in charge of these territories which changed flags as a result of the Spanish-American war. The Trust Territories still enjoy a close relationship with the US because of the Compact of Free Association that ended their status as Trust Territories.
The Marianas are today the Commonwealth of the Northern Mariana Islands and are part of the US Commonwealth with a status very similar to that of Puerto Rico.
If you want to read more; either about the war, or the status of these distant parts of the US:
Cogan, Doloris Coulter. We Fought the Navy and Won: Guamʻs Quest for Democracy. Honolulu: University of Hawaiʻi Press, 2008.
Hattori, Anne Perez. Colonial Dis-ease: US Navy Health Policies and the Chamorros of Guam, 1898-1941. Honolulu: University of Hawaiʻi Press, 2004.
Hezel, Francis X. Strangers in Their Own Land: A Century of Colonial Rule in the Caroline and Marshall Islands. Honolulu: University of Hawaiʻi Press, 1995.
McHenry, Donald. Micronesia, Trust Betrayed: Altruism vs Self Interest in American Foreign Policy. New York: Carnegie Endowment for International Peace, 1975.
Anderson, Benedict. Under Three Flags: Anarchism and the Anti-Colonial Imagination. 2005