So, I imagine the largest and most travelled roads were paved stone. How common were these? Who made them / paid for their upkeep?
Furthermore, were many more places connected by, say, gravel or simple dirt paths, or did you just have to strike out across the countryside to reach most places?
Depends on when you're talking about, and where. But bad to non-existent. Roman roads continued to exist in some level of repair/disrepair. Bridges were generally keep in good repair either by monasteries which owned the land or the local lord. It's in a local landholders best interests to keep river crossing points accessible, during peace, for trade and the opportunity for tolls and/or taxation. In large roman founded cities like London and Paris there would have been cobbled streets but less so in the 'sprawl' of the growing city. Gravel is a terrible medium for horses to travel over! Most major roads would have been delineated in some way; such as, ditches but many if not most would have been packed earth in the countryside. Cutting 'cross-country' for a short distance in a place you knew would have been common, but for long distances very dangerous. Thieves, outlaws and highway men were very real threats to travelers. Plus there were no accurate maps so getting lost is a real possibility. Pilgrims, the most common medieval travelers, would often travel in groups, like in Chaucer's Canterbury Tales, along well known routes like the Camino de Santiago (St. James's Way) in Spain.