It seems to be an article of common knowledge that samurai from the Nara period up until the 1700s or so were very into Zen Buddhism. Many practices from Zen, like meditation, are useful to warriors in combat.
However, Buddhism in general (to the best of my knowledge at least), doesn't exactly have a high view of feudal clan violence. Were samurai aware of this contradiction? Did they just take the practice and reject the teachings? Are there records of samurai who got too into religion and stopped being samurai?
They were aware of the contradiction. The Japanese generally follow both Shinto and Buddhism (I can look up the Japenese word for this concept, it's somewhere in a book I'm reading right now), and Shinto was very, very big on ritual purity. (http://en.m.wikipedia.org/wiki/Harae) So much so that if you were someone who handled unclean things regularly (manure, dead bodies, etc.) you were an eta, the lowest class of individual. Gardeners, torturers, morticians, etc.
So the problem facing the samurai class (who were mainly in the next to the highest class) was that blood was considered inherently unclean, and shedding it and getting it all over you as you would in battle was, well, bad. And so it presented a paradox. They were supposed to be the upper classes of society, but they engaged in more unclean things than, say, merchants, who were just one step up from eta.
I don't know if my books cover all the ways they dealt with the paradox, but some examples I've read include: 1) Conducting regular and sometimes extreme purification rituals, 2) retiring to a monastery to spend the rest of your days trying to spiritually cleanse yourself, 3) not really worrying about it particularly.
Sources: a bunch of works by Turnbull, as well as In Search of Japan's Hidden Christians which discusses the many differences between the Christian and Shinto/Buddhist mindsets (and how this caused friction).