No, not really. It was rather the catch-all term for the inhabitants of the Middle East, Northern Africa and Asia Minor, especially those following Islam. It appeared in late Antiquity (possibly around 3rd century CE) as was most likely derived from the Greek word sarakenoi, used to denote nomadic Arabic peoples living in the North-West Arabia, Sinai Peninsula and Nabatea. The word itself is a rendering of the Arabic sharkiyun (also Syriac sarkaye) simply meaning 'Easterners' (cf. Arabic term al-Mashrik or 'the East' meaning the territory roughly corresponding to what is called 'Near East/Middle East' in other languages). In early Middle Ages, this term was usually applied to all Arabic-speaking people in the Mediterranean region and Middle East.
On a side note, 20th century Arabist Günter Lüling proposed alternative etymology, suggesting that the word sarakenoi could have been derived from the biblical Sarah, Abraham's wife and konein (to fight) and originally meant 'Sarah's fighters'. This proposal is based on the the ancient folk etymology presented by Hieronymus of Stridon and Salamanes Sozomenos who attributed that name to the 'Hagarenes' or descendants of Hagar, Abraham's slave, who wanted to elevate themselves in the eyes of the others by adopting name 'Sarazenes' i.e. descendants of Sarah, free wife of Abraham.
Inhabitants of Western Europe adopted this word around the time of Crusades, when their contacts with Greek-speaking Byzantine Latins significantly increased. Given that virtually all Arabs met by Crusaders were Muslim, this term has been later extended to include other Muslims, e.g. Berbers inhabiting northwestern Africa and the Iberian Peninsula. Additionally, this word has been also applied to the Gypsies who, due to their exotic culture and foreign appearance (in popular literature and imagery, dark skin was a common attribute of Muslims, possibly due to the extensive contacts with predominantly Berber inhabitants of Spanish peninsula prior to Crusades), although in 16th century it has been replaced by the terms derived from Greek atsiganoi (ger. Zigeuner, pol. Cyganie, hun. Ciganyok) or from the name of Egypt (eng. Gypsies, fr. Gitanes, it. Gitani).
In general, the terms 'muslim' and 'islam' we use commonly today were not used in the Middle Ages and Early Modern period, and thus 'Saracens' was a neutral term describing the followers of Islam. In late 15th century it started to be accompanied by 'Mahomentans' and in late 16th century by its variant 'Mohammedans' (akin to 'Christians', although Muslims shun this word pointing out that Islam is centered around God, not Mohammad). Both 'Muslim' and 'Islam' are dated in English only to mid-18th century. Only after the popularization of the term 'Muslim' and its equivalents, the obsolete historical term 'Saracens' started to be used in Europe as a slur.
Som to sum it up, during the Crusades, the word 'Saracens' was a neutral, common word denoting Muslims in general, or Arab Muslims specifically.