Dice were made of ivory or bone, yes, and they were indeed required to be stamped. Here's an example of 18th century dice stamped with the crown and GR. (That's George Rex.)

The Stamp Act of 1765 wasn't a new tax on dice (or playing cards), it was an increase of a tax that had existed since 1710 (9 Anne c.23). The stamping was proof that the tax had been paid, and forgery of said stamp was illegal. It seems to have been done by the makers themselves in the beginning; the law prevented dice or cards from being removed from the place where they were made until the stamp was affixed. An act the following year (10 Anne c.12) prevented any dice or cards that were not stamped from being used in any public gaming place. Further regulation (6 George I c.25) was enacted to prevent the removal or falsification of the seal by marking or otherwise altering the cards or dice in a fraudulent way. Repackaging, and affixing the seal to altered/fraudulent cards or dice was also made illegal. This last one was aimed at preventing people from defrauding his majesty of his rightful taxes, but in practice it regulated the cards and dice themselves. Such acts of forgery were made felony crimes in 1756 (29 George II c.13), which meant you could be executed if you were caught trying to pass off marked cards or weighted dice as new. Therefore, the card and dice stamps became a fee to ensure fair play as much as they represented a tax on gaming.

The taxes on dice and cards (and other stamped goods) were increased a couple of times during the 1700s, but the Stamp Act of 1765 wasn't specifically targeting gambling. The dice were included because they were already being regulated.