The association dates back to the early history of the US, and grew out of the logistics of diplomacy.

Ambassadors had to still get to their destination when appointed, and usually this meant hitching a ride or being given a US Navy ship to carry them. The better to ensure they would arrive safely and with a minimum of delays.

And aboard the ship your only choice for help ashore if you really needed it was either the ship's crew, or the Marine detachment aboard. The Marines had the benefit of at least being somewhat trained as light infantry and being competent in the manual of arms. While at other times like the Barbary War the Marine-DoS connection was more outright, when the tiny US Army was nowhere around to be of use.

While it would also be tradition to send a Navy ship at the request of diplomats to foreign hot spots to protect Americans and American property. Once again bringing their Marines with them. The Maine in Havana harbor being a perfect example. Or the repeated US landings at Vera Cruz and around the Caribbean and Central America. In an age before the Army was in the business of going overseas the Marines were the only trained infantry to be had.

With the post WW2 Security reforms the relationship became more formalized to the program it is today.

The association to foreign diplomacy (well discussed by DBHT14) explains the long standing tradition of Marine Security Guard detachments being the official security force for U.S. Embassies.

The domestic prominence is largely because as a small force initially founded with a narrow scope, we've had to become terrific self-promoters to survive through both peace time and budget cuts. This is discussed in Gen. Krulak's "First to Fight" which shows how we've won the propaganda war by lobbying both Congress and directly to the American people through programs like Toys for Tots.

One of Krulak's main points is that brain trusts like "The Chowder Socieity" decided that to survive, the Marine Corps had to transition from "Infantry of the Sea" to being the most adaptable and rapidly deployed force. They also decided that the USMC would take any role offered, anytime, and do it faster and at a lower cost.

Eventually this led to small but publicly-facing things like Sousa's leadership of "The President's Own" Marine band, the Marine detachment at 8th and I in D.C. and the Evening parades, as well as the fierce marketing as "The Few, the Proud". Over time we've staved off the Department of the Navy and Army as well has Presidents, Congressman, and high-ranking officers who have all tried to eliminate the United States Marine Corps.

We are all taught that the struggle for survival is never over, and so we must continue to strive to be the most adaptable, innovative, and lethal fighting force possible.