There is no direct source for the production of Athenian drama, but scholars have pieced together a general body of knowledge and hypotheses over the years. For Aristophanes, there were typically three main "actors" with speaking parts, with the primary (protagonist) usually taking the biggest part, the secondary taking the next biggest part(s), and the third usually playing the auxiliary characters. Some plays almost certainly required a fourth for very minor parts, usually only a few lines total, and by happenstance you have read one of them: Wasps. A fourth person would have been needed to play the part of the Kydathenaion Dog and the Plaintiff at the end, since there were four speaking characters on stage at those times. Frogs also required a fourth actor, as well as Acharnians and possibly Birds. In addition to these actors was the chorus, which would sing and dance in costume.

All the actors were male, either adults or adolescents, including the members of the chorus. Although we don't have any direct artistic evidence from the time of Aristophanes' plays, we have a lot from a little later, and can be confident that all the actors employed masks with grossly exaggerated features, and wigs. They often wore very large phalluses made of leather that could be raised, lowered, and tucked via a string, like a puppet. Penis-based humor was everywhere in comedy, and in fact a lot of our in-text "stage direction" relates in some way to the phallus, such as in Wasps when Philocleon tells Dardanis to hoist herself up by his member (beginning line 1342).

Actors' garb beyond masks and the phallus was also featured prominently, and was probably styled to represent extremes. Thus Hate-Cleon probably begins Wasps in clothing that marked him as very posh and fancy, while Philocleon begins with rough peon clothing. We are even told specifically that he has on work boots and later changes into nice Spartan shoes. Slave characters would have been easily marked out by their slave-typical clothing, as well, so that no overt references to their status need be made in the voiced lines.

Costumes for the Choruses are another matter, and it is a subject we know frustratingly little about. It is very clear that the Chorus of Wasps begins the play in rather boring garb (just a basic Greek citizen's drapery), but later throw off this drapery to reveal they are in fact buzzing, stinging wasps, complete with actual stingers. We have no idea how this was done, but I personally think we cannot underestimate the level of elaborate detail in which they were willing to indulge. Comedy was meant to be fun, especially as a complement to the stern Tragedies, and the visual identity of the Chorus was a key element in Aristophanes. In fact, the plays are referenced both in ancient times and in modern times by the identity of the Chorus (wasps, frogs, clouds, birds, etc). I think the costuming and revelation of the Chorus was a key element in the whole thing, with a sort of arms race for more and more bizarre and hilarious chorus identities.

If other characters wore costumes, we don't have good evidence. It would make sense. For instance, there are two dog characters in Wasps and a donkey. Other non-costume props were certainly present, but we don't know how extensively. It is lucky that you mention Wasps, because in that play we have a LOT of reference to physical props, which the poet uses to demarcate the various scenes. For the transition from front of private house to lawcourt, we have specific references to objects which must have been brought on stage to transition the scene: a bench of some sort which would have been familiar from the court; a wooden divider/stockade, and the notice-board. In addition, there are numerous other physical objects referenced during the play, including a pot to piss in, a caged rooster, a cheese grater, lamps, ropes, a wreath, incense, a smoke-pot to dispel the wasps, a writing tablet, a full bucket of water, and many many others. Some or all of these must have been physically present on the stage, or else some sort of basic representation of them (maybe just a simple box to represent the caged rooster, for instance).

As for the stage: we know frustratingly little about the theater of Dionysus in the 420s. It had the seating area, of course, and in front of that an orchestra area where the chorus danced. From there, we assume there was a wooden raised platform (the stage) and some means of moving between stage and orchestra, probably via steps. Behind the stage was a wooden two-story façade (the skene) with at least one doorway, which was often the center of Athenian drama, and at least one 2nd-floor window from which an actor could interact with the stage/audience. The roof was also accessible, and Aristophanes' characters sometimes get up there. Though Aristophanes does not seem to use it much, the theater was also equipped with a crane (mechane) for physically lifting actors, and also a sort of conveyer belt thing. It would be obvious that the stage building was decorated, and we have some references to decoration within the dramas, but not very much. I had a sense that we hear how the house in the beginning of Wasps looks, but I'm having trouble finding that now. And just to toss in some confusion: there were multiple venues/occasions for Comedic performance. Four of Aristophanes' plays (that we know of) were performed at the Lenaia festival, and it is possible those were not put on in the theater. In that case, their productions would be different from what we know about the theater space, but (in my opinion) not too terribly different. It is assumed that the basic setup was the same: seating facing an orchestra area, a raised stage, and a backdrop of some sort.

A great source for Athenian drama is Ian Storey and Arlene Allen, A Guide to Ancient Greek Drama (Blackwell 2005).