Yes, at least on a few occasions. Most notable is the fight for Cypress Valley(Baigu) in 621, where they aligned with Li Shimin’s army in a war against Wang Shichong. The monks were caught between factions, and the conflicting interests of those factions. Their land, property rights, and wages were in serious trouble. The other notable instance was a bandit raid on the monastery in the last years of the Sui Dynasty, just a decade before the fight for Cypress Valley began. This 20 page pdf document by Meir Shahar gives an excellent and concise history of the Shaolin in battle.
EDIT: As far as their effectiveness is concerned, it is a bit foggy. Given the limited occurrences, and arguably more limited transcriptions of occurrences, we have very little to go on. However, according to Shahar, Li Shimin felt that the monks did indeed contribute to the war effort. Li Shimin went on to repay the monks. Even still, Li Shimin was not a fan of Buddhism and closed several monasteries. Even the Shaolin monastery was closed under the pretext that they obtained the land illegally, yet was allowed to reopen due to the military support lent by the monks in the previous years. This is most interesting, as the monks had sided with a leader that had antipathetic tendencies toward their religious beliefs. Still, he seemed to truly appreciate their effort.
The Shaolin Monastery, like many remote temples in the countryside, had to depend on themselves against bandits and other criminal elements, leading to kungfu being developed for self-defense. Over the course of Chinese history, military leaders found the "warrior monks" useful in battle when they happened to be available and nearby, but they were only called upon to battle in special circumstances. The Shaolin monks were recorded to have been involved in at least two important periods of Chinese history, namely the Sui-Tang transition (613-628) that created the Tang dynasty, and Ming dynasty's campaign against the "Japanese" pirates (1547-1567).
In the former case, Shaolin monks helped the eventual Tang emperor Li Shimin eliminate a rival warlord on his march to the capital Luoyang. The Shaolin Monastery was nearby, since the Sui emperor had granted that piece of land to the Shaolin. However, when popular rebellions flared up all over China against the Sui, the warlord Wang Shichong took over the mountain estate that was granted to Shaolin and set up camp there. The Shaolin monks teamed up with Wang's rival Li Shimin and took back the mountain, even capturing Wang's nephew. Li Shimin went on to take the throne for his father, beginning the Tang dynasty of China, and took the Shaolin Monastery under his patronage. Despite this record, nothing shows that the monks here fought any differently than other militia troops at the time.
However, by the mid-16th century, during the Ming dynasty, Shaolin somehow gained a reputation as a centre of martial arts, drawing students from all over the country. The Chinese coast was undergoing a military emergency at this time, when "Japanese" pirates known as the wokou raided up and down the coast. (They weren't actually Japanese, but "wokou" 倭寇 literally means Japanese pirates. Chinese smugglers were hiding behind this name to shift the blame.) The Ming military had its hands tied up on another military emergency in the north, since a Mongol raid broke through the Great Wall and burned the suburbs of Beijing in 1550. Military leaders along the coast had to raise troops of their own since Beijing couldn't grant them too much, so various mercenary and militia forces were enlisted to fight the pirates. The Shaolin monks were among these irregular forces.
There were at least four known battles involving the Shaolin monks at this time, but they seemed to have been mostly small skirmishes (then again, most engagements against the pirates at the time were small skirmishes): their gravest defeat at Taozhai (1555) had them sustain 4 casualties while the biggest victory at Wengjiagang (1553) involved 120 warrior monks with a 100-to-4 kill-death ratio. In this victory, the monks were described to have chased the fleeing pirates over 10 days and 20 miles, even mercilessly bashing a pirate's wife to death with an iron staff. So much for the Buddhist prohibition on killing. The other side of this fierceness in battle is the problem of ill discipline, like with other mercenary and militia forces. In one battle, 80 monks broke the formation of the wokou, but instead of finishing the job, the monks fought amongst themselves for enemy heads. The wokou fought back and scored a victory, and the leader of the monks left the battlefield in disgust.
Because of the reliance on irregular forces in the anti-wokou campaign, military leaders mostly appreciated the martial arts of Shaolin and duly noted their prowess in military texts. However, some criticized the Shaolin for being too focused on the staff, to the detriment of the blade and spear. Political officials also point to their lack of discipline and viewed them with an eye of suspicion, lest these monks become bandits themselves.
Sources:
Shahar, Meir. (2001) "Ming-Period Evidence of Shaolin Martial Practice"
Shahar, Meir. (2008) The Shaolin Monastery: History, Religion, and the Chinese Martial Arts.
Lim, Ivy Maria. (2010) Lineage Society on the Southeastern Coast of China