While we often speak of "old/middle/transitional" languages of other European languages like English or French, the case for the language we would call "Spanish" (and the Spanish of the 1600s) differs from the rest of Western Europe. Instead, we should picture Iberia as a pool of multiple languages that rise, fall, and split throughout the peninsula.

Before the 16th century, various states of the Iberian peninsula spoke a number of tongues derived from Latin, Arabic, Basque, Semitic, and Visigothic language roots--many of which live on as languages or dialects today. Of these, the principle language families were Catalan (northeast Iberia), Aragonese (northeast/central Iberia), Basque (north central Atlantic coast), Castilian (central), Leonese (central western), Arabic/Mozarabic (south), and Galician/Portuguese (western coast).

The language we would call "Spanish" stems from castellano, the language of Castile. It was a vulgar language used within the Kingdom of Castile, derived from Romance Castellano , the famous language of the Cantar de Mio Cid (El Cid) of the 13th century.

Castilian became a big fish in the Iberian pond with the joining of the thrones of Castile and Leon, and later Castile/Leon and Aragon in the 15th century. By the 16th century, sources used "español" to refer to the Castilian language.

Portuguese was a separate language, derived from gallego-portugues, a language famously preserved in medieval Iberian songs, poems, and religious texts. Gallego-portugues wasn't always confined to modern-day Portugal; Aragonese and Castilian kings spoke and wrote in the language, and it was popular in art and trade. By the 1600s, the language had morphed to two separate languages: portugués and gallego (the latter seen controversially as a "Spanish dialect", whatever that means).

Could someone speaking Castilian Spanish understand Portuguese in the 1600s? Most likely. The two languages have separate literatures, but they do share linguistic roots. There were (and are) differences; even by El Cid, Castilian had started the process of diphthongization (puerta and tiempo instead of the Portuguese porta and tempo) and different pronunciations of palatal and velar fricatives (/x/ in Castilian like hijo and its hard h sound versus the Portuguese /ʒ/). If your subjects are literate, they may have an easier time reading and writing to each other than speaking--but, they could manage.

Additional Sources: Paul M. Loyd. From Latin to Spanish: Historical phonology and morphology of the Spanish language. Nicholas Ostler. Empires of the Word: A Language History of the World

Before anybody can begin to answer this question you are going to have to specify exactly which languages you are talking about.What do you mean by "Portuguese" and "Spanish" languages? It is important to clarify this as there were not just two languages in use on the Iberian peninsula during the 1600s and it's unlikely that people native to the area would have called their languages by these terms. Just to name a few of the languages that are in use in this are today there is Castillian, Catalan, Galician, Valencian and Basque.