I asked myself this question when I began to research early modern military history. The simple answer is that it became more of a pushing match than a tactical engagement. The common name for this event is called "push of pike," and can be visualized with this image. Whichever pike formation broke first would undoubtably suffer the worst casualties, and this was often the defining moment of many infantry engagements.

In order to counteract the tendency for these engagements to become large pushing matches, the Spanish employed rodeleros - men armed with a small shield or 'buckler' and a short arming sword. These men would be very effective in the close quarters combat resulting from push of pike, and could often provide a decisive element in such an engagement.

Many german commanders utilized Doppelsoldners - men who would be offered a higher rate of pay in exchange for volunteering for the front line. They would carry more specialised weaponry like ranged crossbows or arquebuses, or even the massive zweihander - a two handed sword used to break up enemy pike formations. Thus most armies had troops specifically detailed to deal with the congested conditions created by push of pike.

A good resource (if somewhat dated) is Charles Oman's A History of the Art of War in the Sixteenth Century if you are interested in reading more about early modern warfare.

A follow up question if I may; how accurate is this representation of renaissance pike warfare from the Spanish movie Alatriste? Is it an reasonably accurate depiction of this type of warfare? Renaissance warfare is not really my terrain but on first sight it seems quite accurate. https://www.youtube.com/watch?v=CTYuYxmICGo