According to Paul Merton's Weird and Wonderful World of Early Cinema, which I just recently showed to the students in my Film Literacy course, they were very loud with directors shouting at actors while film was running. There's even a segment in the documentary where they point out times when the actor looks away from where he should be looking and toward the director who has given them some instructions. Additionally, there were larger studios in which several movies would be being shot at the same time since they didn't have to worry about interfering with each other's microphones.

The documentary can be found in parts on Youtube starting here.

They were pretty loud. Beyond the director yelling at people, you would have the sound of the camera rolling and the lights buzzing. This is why when sound came in, camera movement became much more reduced because they had to be locked in big sound proof boxes with the operators. You might also have fans blowing, other weather effects, traffic, crew talking and shifting things about and so on.

EDIT: I should point out that modern film sets aren't that much better. For the most part, audio is added in again later by ADR for the actors and foley sound effects. Usually there's just a guide line for the speaking parts. In some films you can tell when they don't over dub because of an obvious drop in audio quality. The only places where you need perfect sound usually is in television programs or indie movies that don't have access to proper sound editing.

Loudish, depending on the production and the movie set. Silent film covers a very broad range of productions, times, and places, so it's not really possible to generalise. At their loudest, you might have something like the Edison studios in New York, where several movies would be shot simultaneously, in parallel – the noise of the performers, directors giving the performers cues, cameras rolling, lights buzzing, and people going about their business on set might have been a considerable din. But many important silent films were shot on location, and often with small crews; there you might only have the natural background noise of the location.

It is true however that silent movie directors would give actors cues and direction during shooting, often with loud shouting.

Kind of extending the question, I recently watched the 2011 Martin Scorsese film "Hugo", and it depicts silent movie sets as hectic and pretty involved, in terms of number of personnel and cost.

Most films from the silent era are lost now, but is it true that silent films really cost that much money and dedicated cast and set members to make? As in, comparable to modern film sets?