Brassieres of the 1920s/30s are going to be different than most of what we have today, much closer to what are called "bralettes". The bandeau style very popular in the 1920s doesn't offer much in the way of up-lift and support. It keeps the chest somewhat compressed (often there is a small section of elastic in back), but also serves as another layer under sheer or light weight clothing or protection from heavier fabrics. Rubberized versions were available for fuller chested women who needed more compression. Moving into the 1930s, there was a little more shaping. The Kestos brand was immensely popular, it's style often had triangular cups with elastic bands wrapped around. It offered a bit more support, though still a very natural shape.

As for when these were not worn by women, it's a personal choice. Some might choose to wear slips or step-in chemises. Sportswear, like bathing suits, were often worn without the extra undergarments. Loungewear and pajamas were generally loose fitting and wouldn't require undergarments unless desired. Since brassieres weren't really necessary at any specific time, the fashionable bust being natural or slightly compressed, it's always going to be up to personal choice.

Colevintage answered the basic question, but I just want to tangent a bit about the term "flapper dress". It's very popular, but not a useful term when dicussing women's clothing of the early twentieth century.

The etymology of "flapper" that seems vastly most likely (based on my investigation of primary sources) is that it was used as a dialect word for a young game bird, which was then used as a sort of metaphor for a young girl "of the bread-and-butter age", i.e. in her early to mid teens. In the 1900s and 1910s, it was used this way in fiction and non-fiction, referring generally to girls who are too young for courtship and romance in one context and to girls who are too developed for children's clothes but too young for adult clothes in another.

Around the time of World War I, it began to undergo a shift, referring to a wider range of ages but a narrow range of classes (that is, upper-middle and upper class youngish women). Some of the stereotypical behavior we now associate with the term was associated with it then as well, but it was in way clear-cut. It's similar to "Millennial", which some people today use to mean people born between 1980 and 2000, some people use for people born between 1990 and 2010, and some people use for people who are nebulously "young" and exhibit certain stereotyped behaviors.

The clothing that these flappers wore was really not strikingly different from the clothing their elders wore; by the middle of the 1920s, even elderly women were wearing straight-sided dresses and cloche hats. Adult women were just as likely to be wearing the sleeveless evening dresses that "flapper dress" is usually used to mean, or possibly more likely. It's just what was fashionable, not linked to any social group.

I say this not to be pompous, but to help! When you're looking for information on the subject in future, you might find better sources if you search for "1920s evening dress".