Baggage carts.

It would be very unusual to wear armour or weapons as often as is seen in films/TV shows during the medieval period.

A knight would travel with a personal retinue of lesser knights/men-at-arms, and staff to set up camp, cook meals, clean his armour, look after the horses etc. A knights armour and weapons would be kept in oiled bags to prevent rust, and put in the baggage carts with everything else. The same is true for a lance or shield, they would only be carried when needed.

If it was an area where attack was likely, then they would simply carry them (leaning up against the shoulder, or over the shoulder to take the weight off the arms and hands) or have it in easy reach on a saddle.

If they were carrying a lance and shield into battle however, the sidearm could be stuck through the belt at the back so that it hung horizontally, keeping it out of the way of your legs, it wouldn't interfere with the horse, and it would keep it more secure than the axe rings that are popular on reenactor belts, even though there's not really any historical evidence for them.

Edit: Sorry, didn't read the 'into battle's aspect of the title, so you can ignore the top two and half paragraphs.