Introduction

Okey dokey, any question about Ancient Egypt is naturally going to be broad, because Ancient Egypt covers a really long period in human history. Generally, we can divide it into two big periods (and fair warning, I am not an Egyptologist, my knowledge of Egypt is specialized in the Classical Period, so someone with better knowledge may need to correct me on some of the finer points here): the Old Kingdom, which lasted from around 3100 B.C. to about 1500 B.C., and the New Kingdom, from around 1500 B.C. until around either 525 B.C. and the rise of the Achaemenids, or until 332 B.C. and the Ptolemaic conquest, depending on who you ask. Let me stress that this is a hugely broad and glossed over way of looking at Egyptian history (there was a large Middle Kingdom period, for example), but for discussing what we know about the fashions and costumes of the era, it'll serve to keep things simple. I will cover only these two broad periods, as once the Ptolemies came in, Egyptian fashion trends shifted somewhat, although Egyptian culture proved conservative when it came to such trends.

The Old Kingdom

The good news about Ancient Egypt is that even though it is a really old period, the climate was dry and the Egyptians were particularly good at keeping records, both in stone relief and sculpture, as well as written records, of their daily lives. This is much more true, as you might expect, with regard to the nobles and rich people than it is of the commoners and slaves, but nevertheless.

The "skirt thingy" that you refer to is called a "shendyt" or a schenti, and was indeed the typical dress of Egyptian males of this period, from pharaohs to farmers. It appears to have evolved from a kilt-like garment used for hunting, but it was used by people of all walks of life, including soldiers in battle, as it was quite accommodating of movement. While these garments are generally depicted as being extremely fitted and tight in sculpture and relief, this is likely due more to the limitations of the artists' abilities than to their actual tightness -- although they were likely somewhat form fitting and tapered.

People of higher rank would have theirs made out of very fine materials, such as linen, whereas slaves and commoners might have simple ones made of grasses. Sometimes it would be pleated and embroidered for various dignitaries, as you may see here, in this photo of a statue of Egyptian King Men-Kau-Re.

It does, indeed, seem that men did go about bare chested in this period, although they did sometimes wear pendants or other necklaces, beads, and charms of various types. The high priest of Ptah is said to have gone about in an entire leopard skin, but this was probably only during ceremonial duties. Later in the Old Kingdom period, some Pharaohs are depicted wearing a beaded shawl or scarf that wraps around their shoulders, as well.

The New Kingdom

In the later period, men still wore the shendyt, but it had become considerably longer by this time, reaching down to the calves or even ankles. They also wore a long tunic that reached from their shoulders all the way down to their ankles or feet, and seems to have been held up by a strap or ribbon around the neck. The material seems to have been quite thin and translucent, as sculptors and painters usually depict the shendyt as being visible through the tunic itself. So, in this period, at least, the men were less likely to be bare chested, at least if they were high enough in rank to be able to afford these tunics. We also see depictions of Egyptians in the New Kingdom period wearing short cloaks, probably made of papyrus or other durable materials.

Conclusion

The short answer to your question is more or less: yes. Depending on the period and the social class of the individual being portrayed, it is pretty likely that an Ancient Egyptian man would have been wearing a shendyt kilt and not much on top. In later periods, upper-class people would have been wearing more clothes. In both periods, really lower class people and slaves might well have been naked or nearly so (a loincloth or some similar rag). This painting, "Israel in Egypt" by Sir Edward John Poynter uses some artistic license, but gives a fairly accurate and well-researched cross section of some of the typical dress of some variegated classes in Ancient Egypt. It is circa 1867.

Source: Laver, James. "Costume and Fashion, A Concise History" 5 Ed., Thames & Hudson, 2012