According to Sir Michael Dummett's "The Game of Tarot", the first mention of Tarot came from the upper classes of Italy in the 15th century for a card game still played in parts of Italy today. The Roma traditionally practiced palmistry. The first reference to the Roma and Tarot came in the highly fanciful "Doctrine and Ritual of High Magic", published between 1854 and 1856.

So yes, it is just an assumption.