The clearest example of this that comes to mind is the usage of the Bible for swearing an oath on during the inauguration. This is presumably done for someone of Christian faith. I know Kennedy became the first even catholic president, but that’s still the same religion using the same religious document, so that doesn’t really count. What got me thinking was what if we ever got a Jewish, Muslim, atheistic, Hindi, or other religious president that had to be sworn in. Would they still have to swear an oath on the Bible or could they use their native religious documents/none at all? The oath itself is also a reference to Christianity, so would the oath remain the same or would it be changed? Has there ever been historical precedent for something like this on the federal level? There are probably also other aspects of government left over from the beginning of the US that probably assume the person is going to be white, male and Christian, the presidential oath is just the first example that came to mind. But knowledge on this specific example or any other similar example would be very appreciated!
Regarding your specific example, there's actually no legal requirement to swear the oath of office on a Bible, or even to "swear" it at all. Regarding the President, the Constitution says simply:
Before he enter on the Execution of his Office, he shall take the following Oath or Affirmation: -- "I do solemnly swear (or affirm) that I will faithfully execute the Office of President of the United States, and will to the best of my Ability, preserve, protect and defend the Constitution of the United States."
For other officials, it says simply:
The Senators and Representatives before mentioned, and... all executive and judicial Officers, both of the United States and of the several States, shall be bound by Oath or Affirmation, to support this Constitution.
Most Presidents have chosen to swear, to use a Bible, and to add after the oath "so help me God," but not all. Franklin Pierce, a Quaker, "affirmed" rather than swore. John Quincy Adams used a book of law rather than a Bible, viewing it as more appropriate to a civil ceremony; Lyndon Johnson (being sworn in on Air Force One following Kennedy's assassination) was sworn in on Kennedy's Roman Catholic missal since it was quickly available. More recently (within the Twenty-Year Rule), several Congressmen have taken the oath of office on the Quran.
More generally, the Constitution states at the end of Article VI:
No religious Test shall ever be required as a Qualification to any Office or public Trust under the United States.
There was some minor debate in the Constitutional Convention and subsequent ratification debates over whether this should be changed, but in the end it was intentionally left intact. To quote Governor Samuel Johnson of North Carolina from that state's ratification convention:
It is apprehended that Jews, Mahometans, pagans, etc., may be elected to high offices under the government of the United States. Those who are Mahometans, or any others who are not professors of the Christian religion, can never be elected to the office of President, or other high office, but in one of two cases. First, if the people of America lay aside the Christian religion altogether, it may happen. Should this unfortunately take place, the people will choose such men as think as they do themselves. Another case is, if any persons of such descriptions should, notwithstanding their religion, acquire the confidence and esteem of the people of America by their good conduct and practice of virtue, they may be chosen. I leave it to gentlemen's candor to judge...