The US military's presence in the Caribbean can basically be chopped up into two categories, those prewar bases which existed, and those acquired in the run-up to war and during it. And in purpose to 3 basic categories, existing presences, those with an Anti Sub focus, and those with a training focus, though often it was a mix of both in any location.

To start with there were long standing US Army and Naval presences in 3 main locations before the war, Guantanamo Bay in Cuba, Puerto Rico, and Panama around the Canal Zone.

This was then augmented by leases on land for airbases in Commonwealth territory which was part of the US-UK Destroyers for Bases Agreement. The bases relevant here were in Bermuda, Jamaica, the Bahamas, Antigua, St. Lucia, Trinidad, and British Guiana. The other would be up in Newfoundland.

The forces though when war came were still small, underfunded, and undermanned. However additions were being made, especially as US war plans called for the US taking the lead in defending the Western Hemisphere and particularly the Caribbean from all comers(namely German U-Boats) while working to support the Allies in Europe against Germany first and foremost. Additionally a fortified Caribbean would also serve as a screen from allowing raiders into the Gulf of Mexico, where nearly all of the US's oil spent at least some time before heading elsewhere.

Forces were still hard to come by though, and the Canal was very much the initial focus of reinforcements pre and early war, in response to perceived vulnerability to attacks by Japan. About 60,000 US Army men were in Panama and the Caribbean (2/3 in Panama), a year later in late 1942 the number had risen to nearly 180k with roughly the same proportions.

The Navy too was in a sorry shape in its backyard just before the war:

The naval forces available for purposes of local defense were, like the Army garrison, concentrated in the Panama area, where Rear Adm. F. H. Sadler, Commander Panama Naval Coastal Frontier, had at his disposal a small and motley force of patrol planes, submarines, and subchasers. For "heavy" units, Admiral Sadler had two old destroyers and a gunboat, and the rest of his command consisted of 6 submarines, 3 converted yachts, 5 subchasers, 1 mine sweeper, and 12 patrol planes with their tender.

In the Caribbean, Rear Admiral J. H. Hoover's naval forces were divided between a patrol force consisting of two old destroyers, three small submarines, two subchasers of World War I vintage, and twelve patrol planes, and smaller local surface forces at Guantanamo and Trinidad. The latter were virtually undefended; at Trinidad on 7 December 1941 there were only two converted yachts, two "Yippies" (district patrol craft, YP-63 and YP-64) and four of Admiral Hoover's patrol planes.

Needless to say over the next 4 years those forces would be substantially reinforced too. However through the summer of 1942 a comparatively modest number of U-Boats nearly ran rampant. Tanker's were the main prize to be found, and on a handful of occasions merchant crews even refused to sail without more escorts, gravely threatening Allied and especially British supplies of oil from the Caribbean. Through the summer of 1942 over 250 merchant ships were sunk, totaling over 1.2 million tons, with less land 25 German boats even involved, having been redirected after the initial successes found on the US East Coast waned off.

However long transit times, high losses in the Atlantic, and improving Allied counter measures and Hunter-Killer groups meant that success was never repeated. While U-Boats would threaten In 1943-44 the Happy Times of earlier would never be repeated and the needed resources and convoys kept sailing.

As a final note another important role the Caribbean played was as a safe area for work up and training cruises. Virtually every major US warship built on the East Coast would do a cruise down South, and then back to New York or Norfolk to then ether head to Europe, or work back through the Canal and to the Pacific. Aircraft Carriers especially would use the relatively sheltered waters to train up and qualify their air crews, while places like Miami, Key West, and Pensacola in Florida were centers of navy flight training(including the famed Flight 19).

You might be interested in these bits of further reading:

From the USN's official history of its various shore establishments on the Caribbean.

On the Destroyers for Bases Bases by the US Army's Center for Military History.

On the Caribbean U-Boat campaign in a paper from the US Naval War College.

On the Caribbean in Wartime again by the Army's Center for Military History.

All are good authoritative works, and most importantly are free online to read!