They actually were, for a time. Starting on Stockholm 1912 the Olympics included architecture, sculpture, literature, music and painting. The requisite was that all artforms had to be inspired on sports. These continued untill London 1948, and ended because of several reasons: the need for separate spaces for these competitions, frequent disagreements amongst the judges on which piece was best (to the point that sometimes no medals were awarded), and conflicts on which artforms shoud be part of the competition. The final blow was given by Avery Brundage ( OIC president starting 1952) who argued that the Olympics where supposed to be for amateurs, and that the competitors for the art categories were pros using the Olympics to boost their sales. Even after the amateur rule changed the arts did not oficialy return to the Olympics. I know that in Mexico 1968 there were the Cultural Olimpics in which all participating countries were invited to send participants for cinema, child painting, national dance, art, poetry, sculpture and music. These were no competitions, but rather showcases