Did the average Roman, for example, believe that Jupiter would bless him no matter where he stood; or did they believe that as the Romans advanced upon a land, so did the domain of their gods? Would he believe that while in Egypt, his gods held no sway? ESPECIALLY curious about how this idea would've changed under the Ptolemies.
I understand that often connections were made for the sake of assimilating cultures, like making Thor into Zeus. But at times there don't seem to be any connections at all; the only similarities between Ra and Apollo Helios being the sun itself.
I'm also curious about Zoroastrian relations with foreign faiths, what with the monotheism and such.
This is actually several different questions and therefore a bit tough to answer. I'll do my best.
In short: no, the Romans didn't believe that their gods advanced with them when they conquered new lands or that 'their' gods were powerless in other nations. The Romans employed something know as the interpretatio romana when dealing with foreign gods. What that means is that they interpreted foreign gods to be the Roman gods, even if they were worshipped under different names or had different stories attached.
Thus, Tacitus writes that the head of the Gemanic pantheon is Mercury. In fact it was probably Woden, but both Woden and Mercury play roles as psychopomps, hence the association. Or, during the Romano-British period, you get tablets inscribed to 'Mars Nodens', conflating the Roman god (Mars) with the native Celtic deity (Nodens).
Sometimes if the Romans met a new they'd simply incorporate it into the pantheon. At some point, for example (the timeline is disputed), the Romans encountered the Persian god Mitha, and near the end of the first century CE temples to Mithras (his new Roman form) begin appearing in Roman territory.
Some entities were of course believed to govern specific places; hence e.g. the genius loci, a spirit which protected a particular locale.
But that's just the Romans. Asking what 'early pagan cultures' believed is an unanswerable question because there were a lot of them and they all believed different things. I don't know how the Egyptians or Zoroastrians saw foreign deities; each of those is a separate question.
For more, see Ando, Clifford. Interpretatio Romana. Classical Philology. Vol. 100, No. 1 (January 2005), pp. 41-51. A sketch and transcription of the Nodens curse tablet can be found here