Lynching has been practiced throughout the United States, and perhaps there is another commenter who is in a better position to talk about the practice in northern and western states.

But, mostly, when people talk about lynching they're referring to mob justice practiced in the South (and border states) from the early 19th century to the early 20th. Prior to the end of slavery there were indeed many white people who were lynched, but the practice really came into its own, became an institution, during the post-bellum Reconstruction era.

The purpose of lynching was to terrorize the local black community and enforce white supremacy. Plain and simple. Victims of lynching were certainly guilty of threatening the established racial order, but I think it would be quite a stretch to call them "criminals" in the ordinary sense of the word. Victims' offenses were minor and often barely a pretext for the violence committed against them. Their guilt was, more or less, taken for granted.

I should probably talk a little bit more about exactly how lynching accomplished its goal and the purpose that it served. Lynching enforced white supremacy in the South because it was the purest expression of back people's sub-humanity in Southern society. As a black person you were not subject to the protection of the law. If you were simply perceived to be a threat by a white person in your community--if you were too successful, if you tried to vote, if you failed to get off the sidewalk for a white person, if you had sex with a white woman, if you were accused of assaulting a white person--then you could be summarily tortured and executed by the white people in your community. The police would not protect you, they would often participate. The judge would not protect you, your murderers would hardly ever be charged. The state would not protect you, in the cases where lynch mobs were charged they were often pardoned.

Lynching was a way for white people to violently assert their station above black people. Many of the participants in lynch mobs were poor farmers, hardly better off than the poorest in the black community. But because they were white, they were always, no matter how low they sunk, above the most successful black person. That they would never be lynched--among a thousand other things--underlined this fact for white people. It also made the separation very clear for poor white people and ensured their continued investment in maintaining white supremacy.

Sources:

The Negro Holocaust: Lynching and Race Riots in the United States,1880-1950

Reconstruction, by Eric Foner