Some of Darwin's scientific contemporaries just hated it. Alfred Russell Wallace for example just thought it was a bad approach — he didn't see why you couldn't just call it all natural selection and be done with it. He noted that Darwin couldn't explain where female choice would have originated. He thought the key question was not why males of species are sometimes highly ornamented, but why females were usually not — the latter of which has a strong natural selection answer (camouflage). (In other words, he thought Darwin was missing the point when Darwin considered the male to be the interesting case.) Wallace himself however had difficulty coping with the cases of bright plumage and suggested that they might in fact just be non-selective in nature. Ultimately Wallace wasn't sure, but he didn't like sexual selection as an answer. He especially disliked female choice for at least one explicit reason (Wallace did have a somewhat mystical view of life at times, and he thought "choice" required a spirit of some kind, and thus was non-animal), and perhaps one implicit one (he was left at the altar in 1864).

This particular debate, and several others which I have completely forgotten given how long it has been since I've read about this, is discussed at length in Helena Cronin, The Ant and the Peacock: Altruism and Sexual Selection from Darwin to Today (Cambridge: Cambridge University Press, 1991). Generally speaking, sexual selection never really caught on quite as much as natural selection did.