Taiwan already had native people who were hostile towards any Chinese fisherman/settlers that would come onto the island.

Hai Nan was within the territorial or influence of Nan Yue a Chinese/Vietnamese Kingdom that arose during the fall of the Qin Dynasty. During Emperor Wu of Han's reign he began to invade hostile nations which Nan Yue unwisely included themselves into by executing the Empress who supported the Han dynasty as well as several Han ambassadors. The Han Dynasty responded by conquering the Nan Yue Kingdom of which HaiNan was a part of thus incorporating it into China fairly early in history.

Taiwan at the time wasn't particularly important and became settled after the Ming remnants of Koxinga escaped there defeating the Dutch East Indaia Trade company