Edit: I greatly appreciated the great answers regarding how the SCOTUS system came to being. That was enlightening. I was thinking these questions because in the last two months, I' ve read and listen to interviews from three different Justices, nominated by Democratic and Republican presidents, strongly defending that SCOTUS rulings as apolitical. Yet none of them would be sitting as justices if they aren' t nominated by a politician. These interviews reminded me of a remark said to come from Eishenhower that his chief justice nomination is his "most damn fool mistake". It seem that he expected the Justice Warren to side more with his view and dismay that it did not.
I' m more curious on how the Supreme Court conduct themselves in accordances to the politics of their appointers and their reputation as apolitical branch throughout the 1800s to the Warren Court. I' m not looking for modern controversies in the last 30 years or so, there' s enough if that on the news.
So the short answer is no, this wasn’t a concern at the time of the Constitutional Convention in the summer of 1787 over this issue.
There were many unknowns to the 55 men who met in Philadelphia in the summer of 1787. First, at the time, there were no political parties yet forming in America. Sure, Great Britain had some and there were lose political affiliations in America at that time but they rarely united large groups of people and even more rarely did these affiliations cross state lines.
We first start seeing the makings of political parties immediately after the convention ends. By the end of 1788, staunch supporters of the US Constitution were known as Federalists and those who opposed it were anti-Federalists. Again, these weren’t official parties, at least not yet. It won’t be until into Washington’s administration that we see our first two formal political parties, including the now evolved form of Federalists and the newly established Democratic Republicans who tended to be made up of pro slavery states and prioritized “states rights” over a stronger national government. The first national election where political parties became the focal point was the election of 1796, which saw John Adams (Federalist) prevail over Jefferson by an electoral college vote of 71 to 68. This was extremely different than George Washington’s previous two victories where he ran without a political party affiliation and had both Federalists and Democratic Republicans in his cabinet.
I mention all this because it’s meant to show that at the time the Constitution was drawn up and ratified by each state (1791), political parties were not anything like they would become. John Jay, who was appointed to the Supreme Court by Washington in 1789, and would become the first Chief Justice would later become a Federalist, but again, at the beginning, this wasn’t something they were overly concerned with. That said, by the election of Jefferson over Aaron Burr in 1800, it became apparent the political party was here to stay, even despite warnings from Washington in his farewell address to avoid them. By this point, it would have been too late. The constitution was ratified long ago and Americans seemed to quite comfortably fit in with having political parties at the helm. In order to change it, it would have required a constitutional amendment, but even then, what would an alternative be? The goal was to ensure that once appointed, they would be free from the political whims of the masses or politicians, which guaranteed (in their view) that their opinions would be free from interference. Perhaps some people worried about this concern that you mention, but none ever acted to change it.
As you noted, justices would increasingly connect themselves to political affiliations as time went on, especially later in the 19th century, however I would rather another expert fill in that answer.
Edit: added one more detail