It seems no book was used at all. The U.S. Constitution does not require any book, religious or otherwise, to be used when taking the oath.

Theodore Roosevelt became President of the United States following the assassination of President William McKinley in September 1901. He took the oath of office at the home of his friend Ansley Wilcox in an improvised ceremony attended by several members of McKinley's (now Roosevelt's) cabinet. Wilcox wrote a firsthand account, which provides one of the most detailed descriptions of the ceremony. He describes the oath as "taken with uplifted hand" and does not mention any book being used as part of the ceremony. [1]

Wilcox also mentions that press was allowed in the room, but that they were not allowed to take photographs, so there's no hard evidence to prove or disprove whether Roosevelt was sworn in on a book. I found one newspaper sketch that appeared shortly after the fact which depicts Roosevelt taking the oath with no book at all, though its hard to say how accurate something like this is. [2]

If you're really curious to know for absolutely certain, I suggest calling the Theodore Roosevelt Inaugural Site and asking someone there. (trsite.org) The building is now a National Historic Site and a museum. If anyone knows the answer for sure, it's them. (And if you do decide to call them, please let me know what they say!)

_____________

[1] Ansley Wilcox describes the Inauguration of Theodore Roosevelt, https://www.nps.gov/thri/upload/awilcoxoninauguration.pdf

[2] Sketch of Roosevelt Inauguration, originally appearing in Nashville, Tennessee News on October 13, 1901: https://commons.wikimedia.org/wiki/File:TR_Inaugurationsketch.jpg