There are a number of ways, depending on time and task, but most of them are based around what gave us the word 'plumb' in the first place.

A professional roman surveyor would use a tool called a 'groma'. This was effectively a wooden cross on a pole, with lead weights suspended by thread from each of the four points of the cross. When all the lead weights fell straight, the groma was level. A surveyor could then aim along the groma, using ropes and other equipment to survey along straight lines. A google picture search should give you plenty of examples of modern recreations of the instrument.

The groma is probably based on an earlier greek instrument, which might well have been based in turn on mesopotamian or persian equipment, but I don't know enough about this to go into detail.

Just to clarify my first statement, 'plumbum' is latin for lead (also providing us with the origin of the word 'plumbing' - carrying water through lead pipes). To plumb something you watched the angle from a suspended lead weight. To level something you compared the angle berween two or more such weights.

Our chief sources on roman surveying are two manuals (or "descriptions"), both of which are from the early empire - Marcus Vitruvius Pollio and Sextus Julius Frontinus. Look them up if you're curious for more.