Generally speaking, they were sent back to Japan.

A documentary, Wansei Back Home was made recently about Japanese born in Taiwan who were repatriated back to Japan.

Here's a very short related footage: https://www.youtube.com/watch?v=V9ZkU9UCDmU

If you are interested in further reason, this book deals specifically with the repatriation of Japanese nationals from Taiwan: https://books.google.com.tw/books/about/%E8%87%BA%E7%81%A3%E7%9C%81%E6%97%A5%E5%83%91%E9%81%A3%E9%80%81%E7%B4%80%E5%AF%A6.html?id=BBlXPQAACAAJ&redir_esc=y

In the book, records showed that Japanese nationals repatriated from Taiwan to Japan over three phases totalled 321,272. Only 260 were allowed to stay as government employees and their family accounted for another 495.

In the first census in Taiwan after the war, in 1956, a total of 1,233 people listed their ancestry as "foreign nation", these possibly included those Japanese who chose to naturalize and were accepted by the ROC government.