Your guesses are pretty good. Litters were good because they provided anonymity of a sort, but that also made them a liability. They were also considered a bit distasteful (Juvenal and Martial and even Cicero view them as the height of supercilious ostentation). Many people like Caesar would have walked around Rome. It was good for their public image to be seen around Rome with their friends and clients instead of hiding in a litter. It also showed them as vigorous and humble.

I have been marathoning Roman poets and epistlists and have not seen a reference to traveling in the city on horse. If you have ever been to Rome, you will realize that the ancient city was actually quite small and easy to travel on foot. The powerbrokers of Rome all had houses on the hills, making them quite close to the center. Others may have rode to the city and stabled their horses but that is a speculation.

As for your speculated mildly wealthy person, walking would not be seen as disgraceful if he could not get a litter, but he may have been able to borrow one from a friend or generous patron (instead of picturing that textbook hierarchy, it would be more like a business partnership).

As to long distance travel, it depended on the money you had available. The army almost always entered Gaul from Masillia as you propose, but a small group could go by horse through the Alps, weather permitting. Then again the Alps were not totally subdued until Augustus's time. Cicero sometimes claims that he could not "travel openly" in which case he could use a carriage or more likely a litter. Chariots were almost alien to Romans. I wager the only time any grown Roman would ride in a chariot would be in his own triumph. Check out Orbis, a Google Maps for the ancient world to get a better idea of long distance travel.

I know rich people would have litters and slaves to carry them around. And poor people had to walk everywhere. But what if I were just mildly wealthy? Could I rent a litter for a day or a single ride, like a taxi? Or was there any other means of transportation I could hire? Was it possible to ride in the city on horseback?

The streets of ancient Rome were not large and were very crowded. I can't personally see too many people trying to ride a horse through the city, at least outside the major thoroughfares. I have also never read anything mentioning the renting of a litter.

Also what would be the best way for me to get to, say, Lugdunum (or any other city far away from the coast). My gues would be by ship to Massillia and then to Lugdunum in a rented chariot. If one could rent a chariot, that is.

The word you are looking for is carriage, not a chariot. And, to get to Lugdunum, you would have taken a ship to Massilia most likely, and from there either taken the Via Agrippa or simply secured a ship passage up the river Rhone.