Novelty items go a long way back. Egyptian rings turn up in places where its really rather unlikely that anyone could read the inscriptions. Philip de Jersey in Celtic Coinage in Britain mentions Carthaginian coins from the third century (might be a bit earlier dating is tricky) being found in small numbers. Since they were bronze bullion value is unlikely and Britain wouldn't be using coins for another century suggesting they could only have been seen as novelty items.

Stone wrist guards were probably generally non functional and are about 4000 years old (well in Ireland anyway). A open access paper that talks about them in an Irish context can be found at:

http://intarch.ac.uk/journal/issue26/31/toc.html

This is going to be rather difficult from an archaeological perspective. Apparently non-functional items may have a religious purpose (consider the various models found in Egyptian tombs) or be a demonstration of a craftsman's skill (consider inside out barrels).

Defining your question more narrowly probably not because that kind of kitsch is the product of mass production. Early examples that still exist in large numbers would be items created for the tourist trade (in a UK context that would mean the various things that turned up once the middle classes started doing the grand tour or items made for places like Aberystwyth).

Depending on what exactly you mean Bonsai would also be a fairly early example. Sator Squares (a latin word square) probably weren't jokes but they are widely reproduced and don't have any obvious function