As a quick answer, early bayonets were limited in their utility because, once affixed, the bayonet prevented the musketeer from firing. These were called "plug bayonets" and were inserted into the barrel. You can see what they looked like here and here.

So they had extremely limited usefulness when they were first introduced. The ubiquity of pikemen, and the infrastructural reinforcement of them, made the use of pikemen still an extremely reliable method of conducting warfare for a time.

Ring and socket bayonets, which were designed to fit around the barrel of the musket and so leave firing unobstructed, were introduced a little later (the plug bayonet is dated to the early 1670s as a military weapon, the ring style in the late 1670s), but these also took a while to become fully adopted in armies for a number of reasons, with expense, tradition and logistics problematizing their adoption.

It also takes a significantly different attitude to use the bayonet as opposed to shooting at enemies. Charging is significantly more stressful to a would-be combatant, and engaging in hand-to-hand combat depicted in a lot of films was so rare as to be exceptional, even well after socket bayonets had become ubiquitous.

One of my favorite little blurbs to that end comes from Winfield Scott, when talking about the Battle of the Chippewa in the War of 1812, which ended with a successful bayonet charge:

That clash of bayonets, at each extremity, instantly followed, when the wings of the enemy being outflanked, and to some extent doubled upon, were mouldered away like a rope of sand. It is not in human nature that a conflict like that should last many seconds. The enemy's whole force broke in quick succession and fled...

It is, quite simply, something that people would rather avoid.

---

Tactics-wise, one of the best books on the use of the bayonet in an 18th century context is With Zeal and With Bayonets Only which relates the bayonet-heavy focus of the British army in the War for Independence in a wider context, explaining that bayonet use was a tactical doctrine adopted on the ground for that war, and was an unusual approach for the British army to take.

And you can find Winfield Scott's memoir online here, and it should take you right to the appropriate section.

u/PartyMoses has covered the difficulties involved with developing and fielding an effective bayonet, however it might also be worth re-examining what advantages the bayonet actually provided.

A bayonet certainly provides more reach than a sword does and many musketeers found the bayonet less cumbersome than carrying a three foot sword around everywhere, however the musket was still rather heavy for a pole-arm and some infantry continued to prefer carrying swords well after the bayonet was introduced. In addition, even with bayonets musketeers still stood no chance against an actual push of pike unless they could repulse the enemy with firepower alone. Sweden would continue to use aggressive pike charges to good effect even into the 18th century and during the American revolution, a number of founding fathers including Washington thought it would be a good idea to arm colonial troops with pikes or half-pikes to counter British superiority with the bayonet.

Instead the bayonet is just one of many developments which lead to the pike being widely abandoned by European armies. The 17th century saw muskets become lighter and more ergonomic, gunpowder improved, paper cartridges were introduced, cavalry became lighter and abandoned the lance in western Europe, the flintlock mechanism improved the reliability and rate of fire of muskets, drills improved, and formations became thinner to better emphasize firepower. At the end of the 16th century observers were already noting that it had become uncommon for infantry to come into physical contact before one side fled, and as u/PartyMoses points out by the 18th century actual melees with bayonets had become very rare and sporadic, typically responsible for perhaps only 1% of battlefield casualties. The 17th century had already seen the ratio of infantry with pikes to muskets in most armies gradually increase from 1:1 to 1 pikeman for every 4 musketeers. It just took the bayonet to finally convince conservative military men that armies could survive without any pikes at all.

For more reading:

European Warfare, 1350-1750 Frank Tallett

Military Experience in the Age of Reason Christopher Duffy

Fighting Techniques of the Early Modern World Christer Jorgensen