Quartering soldiers kind of seems like an oddball to me next to things like freedom of speech, freedom from search and seizure, etc.
Yes. This has happened a few times surprisingly enough, but each application was a stretch according to most scholars, and only one application succeeded in a court, Engblom v. Carey.
In Engblom v. Carey, "prison officials in New York organized a strike. They were evicted from their prison facility residences, which were reassigned to members of the National Guard who had temporarily taken their place as prison guards. The United States Court of Appeals for the Second Circuit ruled: (1) that the term owner in the Third Amendment includes tenants (paralleling similar cases regarding the Fourth Amendment, governing search and seizure), (2) National Guard troops count as soldiers for the purposes of the Third Amendment, and (3) that the Third Amendment is incorporated (that is, that it applies to the states) by virtue of the Fourteenth Amendment. The case was remanded to the district court, which dismissed the case on the grounds that state officials could not have been aware of this interpretation." This text was taken from wikipedia's entry on the Third Amendment, but it is thoroughly supported elsewhere. For example, Morton Horwitz discusses this case in his law review article on the Third Amendment, "Is the Third Amendment Obsolete," here. Thomas Bell also discusses this case in his law review article, "The Third Amendment: Forgotten but Not Gone," here.
The Third Amendment was raised most recently in Mitchell v. Henderson, but found not to apply. In this case, the District Court of Nevada ruled that the Third Amendment does not apply to intrusions by municipal police officers as, despite their appearance and equipment, they are not soldiers.
The Third Amendment has also been raised in various court opinions in the past, but not as a directly raised right. For example, the Third Amendment is raised in Griswold v. Connecticut as evidence that the right to privacy lies in the "penumbra" of the Bill of Rights and the Constitution. It was also raised in Youngstown as evidence that the framers intended to constrain the powers of the executive branch during wartime. Even though these cases were from SCOTUS and invoked the Third Amendment, SCOTUS has still not ruled on a direct application of the Third Amendment itself. The highest, and only allowed, direct application occurred at the Second Circuit in the case discussed above, Engblom v. Carey.
The short answer is that there is only one case in which the Third Amendment is seriously considered by the courts, which is the Engblom v. Carey case /u/workingonbeingbetter discusses in this thread.
The medium answer is "like all US legal history, it's complicated," and I'll give the long answer (why it's complicated in general, and why this question in particular has a surprisingly complicated answer) below.
Why is legal history in the US complex?
Common law is complicated! The US has a common law system inherited from England. In common law, custom and judicial interpretation are of roughly equal standing with the letter of the law itself. We call this precedent. A working definition of precedent is "what the courts have decided the law means," and it's customarily very binding to future decisions. While it's more strictly binding to lower courts in a jurisdiction, courts in different jurisdictions do still reference and defer to each other to a lesser extent. In fact, while it's not something you really see today as the US now has a robust legal system, legal scholarship, and body of case law, you'd see US courts citing contemporary English cases through the early-to-mid 1800s to help answer questions of law that were arising. (I touch on this briefly here, where US courts were examining more-or-less contemporary case law in England on water rights.)
This emphasis on precedent has a couple of implications.
One is that the text of laws can be written very loosely on the basis that the courts will understand the assumptions grounding it, which come from custom and precedent. However, this also means that the looseness of such text can leave a lot of wiggle room for differing interpretations and even abuse of the law, which we'll get into with the Third specifically below.
The other is that the study of law and the writing of opinions is itself a study of (a narrow slice of) history. Both precedent and the custom surrounding law are important to writing good law and making good arguments This means that interpreting a particular law (like the Third Amendment) requires a grounding in US legal doctrine and often, especially in the case of early Constitutional text, centuries-old English doctrine to interpret the law itself.
There's a few subquestions and some assumptions in your post that need to be unpacked for a thorough answer, so this is going to be in several sections.
Like many of the Bill of Rights, it's a reiteration of existing (and almost uniquely) English doctrine that was explicitly included due to grievances of the colonists in the leadup to the Revolutionary War.
Quartering in English Law
Protection against quartering (also known as billeting) is actually one of the oldest recognized protections in English common law. Professional soldiers and standing armies weren't so much a thing in England before the centralization under the Norman Conquest, but new policies financing and organizing these forces were implemented in the 1100s. Other legal protections limiting the billeting of these soldiers quickly followed; the 1131 charter of London specified that "within the walls of the city no-one need be billetted, not [members] of my [Henry I's] household nor anyone else, [nor] is any billet to be taken by force." Other charters for towns, cities, and boroughs included similar provisions; some had charters that guaranteed them rights expected of comparable boroughs, which implicitly granted protection from billeting. (Some other familiar rights in the London Charter: the right to appoint their own sheriff and judge, the right to be tried within the city limits)
This wasn't guaranteed protection, and troops often were quartered anyway; while they were supposed to give receipts for reimbursement of food and lodging, these were rarely honored for a variety of reasons ranging from a lack of bureaucratic infrastructure to Parliament keeping a tight leash on military spending for political purposes. Meanwhile, Parliament and various revolutionaries passed a series of acts and presented a series of petitions that steadily expanded protection from quartering.
The 1628 Petition of Right read, in part:
great companies of soldiers and mariners have been dispersed into divers counties of the realm, and the inhabitants, against their wills have been compelled to receive them into their houses... against the laws and customs of the realm[emphasis mine]"
Previous quartering laws had been protections granted to specific polities; the Petition of Right went a step further and made the assertion that quartering was, itself, now a constitutional issue restricting the king across the board. The 1689 Bill of Rights (also known as the English Bill of Rights) went a step further, specifically mentioning quartering as a violation of “the existing rights of Parliament and the subject, which James had outraged.” The 1689 Bill of Rights is a big deal in the development of English legal doctrine; the theory that the individual (the “subject”) has inalienable rights which the government is not permitted to infringe on formed the basis for both American and English legal theory.
The last of the pre-Revolution acts expanding quartering were the Mutiny Acts 1689 and 1692, which forbade quartering without consent and mandated payment for quartering, respectively. However, they notably did not apply to the American colonies.
Quartering in the US Colonies
Just like in England, periods of military deployment and action brought quartering in the colonies. Colonial assemblies responded to these by passing their own acts and declarations; New York’s 1683 Charter of Liberties included a provision that “Noe Freeman shall be compelled to receive any Marriners or Souldiers into his house and there suffer them to Sojourne, against their willes provided Alwayes it be not in time of Actuall Warr within this province.”
Military deployment in the French and Indian War (1756-1763) resulted in attempts by deployed British troops to quarter in colonists’ property; some were successfully denied, some were not, but the attempt was met with complaints by citizens. In response to the expensive war, and to defray the ongoing cost of defending the expansive colonial territory, Parliament passed the Quartering Act of 1765, requiring the colonies to fund barracks and supply soldiers. In the case of insufficient room in the barracks (pretty much guaranteed; it’s not like there were a bunch of empty barracks sitting around), soldiers could be quartered “in such barracks, in inns, livery stables, ale-houses, victualling-houses, and [various drinking establishments]” and then private buildings like barns or other outbuildings if the former were also full. This was quickly followed by the Stamp Act of the same year, which was intended to fund the construction of barracks (which colonial governments were dragging their feet on) and provide revenue for paying rent to landowners quartering soldiers in the meantime; the Stamp Act, of course, is what the famous colonial grievance of “taxation without representation” referred to. Resistance to the 1765 Quartering Act was significant; the New York Assembly refused to comply and was suspended by Parliament until it did, and the governor of Massachusetts (a representative of the king) quartered troops in the state-house as punishment for the legislature failing to comply with the Act.
Parliament reiterated the quartering practice with the Quartering Act of 1774, one of the punitive Coercive Acts (also known as the Intolerable Acts) passed in retaliation for the Boston Tea Party in 1773. Explicitly, it gave governors authority to directly quarter troops as prescribed by law, bypassing the legislatures of the colonies entirely. Dissidents claimed that it gave the executive the power to quarter troops in private homes, which may have turned out to be accurate or not; in any case, it didn’t have much time to be tested before the Revolution started in earnest.
Why was quartering such a big deal to the Framers and to colonists in general? Keep in mind that before the Revolution, most people thought of themselves as English subjects whose rights and privileges as English subjects were being trod on. Even the most ardent separatists believed that they were inheriting an English legal tradition of rights. They identified with the six-hundred-year fight for protection from billeting; their argument was that the 1689 Bill of Rights applied to them, as they’re just as much subjects of the king/have the same God-given rights as someone who happens to live in the borders of England. Having to quarter troops was itself a burden (financially and with concerns of safety, particularly for women), but it was also a violation of rights and outright illegal. There was very much a feeling of being second-class citizens who weren't being afforded the full protection of the law.
When the US Bill of Rights was passed, its conceptual framework was the same as that of the 1689 Bill of Rights – not granting anything, but an assertion and enumeration of already-existing rights. Quartering was included because it had always been an important right in English legal tradition, particularly over the past century, that had been violated repeatedly in recent memory. Its explicit enumeration was so in demand that it was the eighth-most requested addition to the Constitution by ratifying conventions out of ninety proposals. (Dumbauld)