Heavens, yes. They had the bullets that killed Kennedy. They had the Carcano rifle that fired the bullets, which had been left at the Book Depository with Oswald's fingerprints all over it, and they quickly had proof that he had purchased the rifle- even a photo of him holding it!. Right after the shooting, the police and the managers at the Book Depository gathered the employees for a roll call, and Oswald was the only employee missing. The police soon had the testimony of an employee there who had given Oswald a ride to Depository. He stated Oswald was carrying a long brown-paper wrapped package that he claimed was "curtain rods".
There was also the other murder. Most people forget that after he fled the Book Depository, Oswald stopped by his rented room, got a pistol, ran out and almost immediately shot and killed a policeman- almost certainly because he thought the policeman was looking for the Kennedy gunman. Oswald was found with that pistol when he tried to hide in a movie theater.
It has often been asked, what more evidence does anyone need?
However, it was not precisely the "American" government that had the evidence. As the crime had been committed in Texas, and Oswald had not crossed state lines, it was the State of Texas that had jurisdiction.
Vincent Bugliosi : Reclaiming History