Recently I noticed that a reply I left to a decently upvoted post with only a couple of answers wasn't getting any responses, so I checked logged out and it was removed. The removal itself doesn't bother me. What bothers me is that if I hadn't thought to check, I wouldn't have even known it was removed, and even knowing it was I don't know why. This is a problem, because if people who leave insufficient replies don't know that their responses are being removed, then they will probably keep posting similar responses.
I get that its unreasonable to ask that every single removed comment get a whole paragraph about why its being deleted, but I feel like even just a single sentence with a link to the rules would be better than nothing, which is what a lot of people get right now.
The ideal solution would be for us to give a warning for every removed comment, yes. However, you have to bear in mind that some threads receive many inadequate comments, and that leaving a warning for all of them would result in the comment count visible from the AH main page and, more crucially, /r/all being doubled. When we have 5-10 removed comments on a highly-voted post, people start to get tetchy and post "[removed]"/"where are all the comments?" Imagine how much more frustrated users would get if they saw on /r/all that a thread had 16 comments and clicked on it to find 8 removed comments and 8 identical "we require that answers be in-depth and comprehensive" responses!
We try to keep a balance between letting people know they need to post better answers and keeping the clutter down. Occasionally this means that someone slips through the cracks and is never told that their answers are bad, but most of the time people who repeatedly leave bad comments will get warned. And I have to tell you that many people don't actually stop posting inadequate responses after being warned one or more times, and end up being banned.