Tokyo Metropolitan Archives in fact uploaded the administrative document published in 1878 that annotated some historical background of the integration of the Izu island chain.
Almost contrary to its name as well as geographical (volcanic as well) traits, Izu Islands had been directly ruled by the Tokugawa Shogunate during Edo Period. Not only used as the penal colony, but also some exotic product of the isles like purple root (Shikon: purple dye can be extracted from this plant root) and dried as well as fresh bonito fish (skipjack tuna), abalone, and further, carmellia seed in Ohshima (linked to the picture of the wood in the municipality's official site) (for excellent seed-oil, not only edible, but also good for making up hairs) attracted attention from citizens of Edo, capital of the Shogunate. Shogunate decided to abolish the normal tax from the isles, but instead to imposed the fourth (for bonito fish) or the tenth of other sea and land products of the isles as its substitute.
Thus, the connection with Edo's commercial economy had already become fundamental to the inhabitants of these isles. Since the late 18th century, several travelogue (more precisely stating, experience notes of the exile) have been published in Edo, and they also incited the interest of Edo's citizens to these southern Isles.
After the Meiji Restoration, Izu isle chains had still maintained its distinct judicial system, well-connected with New Tokyo Prefecture before its administrative integration. The annotation in the site above also states that Minister of Interior Toshimichi Ohkubo (who appeared and was assassinated in famous manga/anime series Samurai X - Rurouni Kenshin) himself explained to his colleague on the pros of this administrative measures, based on the economical as well as judicial dependence of the isles from historical point of view, as I briefly summarized above.
On the other hand, Ogasawara in the late 19th century had been just acknowledged as a part of Japan, and was almost directly governed by the Ministry of Interior. If Japan transferred these new isles from the direct rule to the more normal local government, it would be no other than Tokyo Pref. that could conduct such a delicate business, in addition to its geographical location.
References:
(Edited): fixes typos.