There is such precedent. The authority to do so is right in Article I of the Constitution:

Each House may determine the Rules of its Proceedings, punish its Members for disorderly Behaviour, and, with the Concurrence of two thirds, expel a Member.

Each chamber sets its own rules, over which it is generally held sovereign and would probably dictate the course of things. (But there are ways that a simple majority can get around the rules.) They can include committee hearings or even a sort-of trial before the whole chamber, but by the strictest reading any member can be expelled for any reason at any time, provided the motion can reach a 2/3 majority.

This is overruling the will of the people, so it's not done lightly, but it has been done twenty times in the past. Fifteen of those are Senators, the most recent in 1861. Jesse Bright (D-IN, owned a slave labor camp in KY) got expelled for treason, basically. That's the same as most of the class that went out around that time. The two cases before their class were for the same charge, though one of those couldn't get to 2/3.

Most of the cases in the Senate since then have been for corruption of one form or another. They haven't resulted in successful expulsions. Part of this is that 2/3 is really hard to get and part of it is that people looking down the barrel of an expulsion generally resign before it lands. Because expulsion is so hard, more often Congress makes do with censure. Censure is a resolution voted through the expresses official disapproval of the chamber and has customarily been treated as cause for immediate resignation.

For example, Preston Brooks beat Charles Sumner nearly to death on the floor of the Senate. The House established a committee that looked into things and recommended expulsion. They couldn't get the 2/3. He resigned anyway, but two co-conspirators -they knew, at least generally, what Brooks intended and showed up to watch; at least one of them gave him tips on how to best do it- got censure resolutions. One of those failed, but the other succeeded. Brooks and the censured guy, Lawrence Keitt, both resigned at once and were re-elected by their constituents within a month.

This is also what happened to Joshua Giddings and was threatened to John Quincy Adams; they went after Giddings in part because they couldn't get Adams. This is in the context of their fight against the House rule gagging antislavery petitions in order to prevent slavery debates on the floor. Proslavery people? Not huge fans of open debate. Giddings got censured, resigned, and re-elected in short order. The Congress didn't like what he did, but his voters loved it. He, like Brooks and Keitt, was re-seated without incident.