I'm not a researcher of the Edo period specifically, but I did recently teach an introduction to gender in modern Japan (Edo - 1940s) (so I think I have a relatively good grasp of the state of the field currently).

So, apart from the obvious problem—namely that our contemporary gender categories can really only find expression in a cultural environment that makes it possible to conceive of them to begin with—the best we can hope to find is something that is similar enough to define it as more-or-less "equivalent."

Unfortunately, historical research on gender and sexuality in Japan that explores the world beyond just the heterosexual male/female dichotomy (e.g., homosexuality, queer identity, etc.) isn't exactly common. Some works on primarily male-male sexuality do exist (which is also the most overt and socially accepted form of sexuality apart from male-female interaction in the premodern era), lesbian studies is still close-to non-extant...there isn't really all that much out there.

That being said, the only examples I can think of in the literature I am familiar with on the subject which would go into the direction of your question are unfortunately located within the Meiji/Taishō period:

In one of her essays on intersexuality, Teresa Algoso (2011, see below for bibliographic data) does mention the concept of “quasi-hermaphrodism”—the term she chose to translate the Japanese term—which described people that had the following features:

Two texts from the late 1890s, for example, both define a masculine pseudohermaphrodite as an individual who “is actually male, but whose outer sexual organs appear female,” while a feminine pseudohermaphrodite is an individual who “is actually female, but whose outer sexual organs appear male. (pg. 244)

Later in the same paper, she gives the following example:

Yet another account from 1910 centers on the prostitution by Gen of his common-law wife, Chiyo, to a farmer. Upon arrest it was discovered that, “despite being a woman,” forty-six-year-old Gen had cut off all her hair and had starting dressing in men’s clothing about thirty years earlier. Though this last story might, by early twenty-first-century standards, be seen as a case of transgenderism, in the early twentieth century it was classified as quasi-hermaphroditism (junhan’in’yö). (pg. 252f)

Likewise, the only other example that springs to mind is noted in Gregory Pflugfelder’s Carthographies of Desire, which on page 166 describes a case from 1881 (the earlier Meiji period). Let me somewhat paraphrase the passage for the sake of omitting the sexually explicit segments:

A certain Inaba Kotoji, 26 years old male restaurant employee was accused of, and sentenced for, illicit sexual intercourse. According to the court proceedings, Inaba had “forfeited the normal condition of males,” dressing in feminine clothing and adopting the name Okoto. He thus passed as a woman, although rumors sometimes circulated that he might be a “hermaphrodite.” One night in February 1881, Inaba crept into the bed of a rickshaw puller also sleeping at the restaurant, and, “using the same methods as a woman,” . . . [I'll leave the rest up to imagination].

It should be noted that in these examples, the explanatory attempts by authorities of what we would likely call transgenderism is approached through the phenomenon of intersexuality ("hermaphrodism") by contemporaries, which shows that the way people thought about gender was quite a bit different from today.

But either way, 1881 is close enough to the late Edo period that I strongly suspect similar cases to have occurred decades earlier, even if there might be no evidence (or rather: no evidence cited in the literature available).

Looking at the Edo period, of course, we are well-aware of professional transvestites (called kagema, or onnakagema if they were women posing as men), since those—both boys and girls—were trained as prostitutes in various establishments to entertain customers. Kagema are trained from a young age to „act“ like members of the other sex. The question this raises is whether (some of) these actually were identifying as belonging to the opposite gender, or if this occupational expression of gender remained at the level of a performance, or whether this, in some cases, resulted in the development of a distinct gender identity. This is impossible to answer without access to ego-documents, where the person in question can give us clues about their own sense of self. I am not aware of studies citing any such documents, so this is a dead end.

You have a similar general issue with the onnagata actors in kabuki, which I assume you might have been referring to in your question. [there is more literature out there on these actors, but I am not familiar with it, so I have to leave it at the mention...]

This might not have answered your question specifically, but this is as close as the current literature on the (English-speaking) market can get you.

​

References / Further reading:

Algoso, Teresa A. "'Thoughts on Hermaphroditism': Miyatake Gaikotsu and the Convergence of the Sexes in Taishō Japan." The Journal of Asian Studies 65, no. 3 (2006): 555–573.

Algoso, Teresa A. "Not Suitable as a Man? Conscription, Masculinity, and Hermaphroditism in Early Twentieth-Century Japan." Chap. 11 In Recreating Japanese Men, edited by Sabine Frühstück and Anne Walthall. Berkeley: University of California Press, 2011, 241–261.

Leupp, Gary P. Male Colors: The Construction of Homosexuality in Tokugawa Japan. Berkeley, Los Angeles, CA, London: University of California Press, 1995.

Mostow, Joshua S. “The Gender of Wakashu and the Grammar of Desire.” In Gender and Power in the Japanese Visual Field, edited by Joshua S. Mostow, Norman Bryson, and Maribeth Graybill. Honolulu: University of Hawai’i Press 2003, 49–70.

Pflugfelder, Gregory M. Carthographies of Desire: Male-Male Sexuality in Japanese Discourse 1600-1950. Berkeley, Los Angeles, London: University of California Press, 1999.

[some edits to rephrase a few sentences, I'm crazy tired]