I have looked through the past history of meta posts and found excellent answers on how best to write top-level responses here, here and here.
The sources cited in the answers I have read have generally been sound, so my question is more about what disqualifies a source from being relevant. I admit, I don't have much knowledge of answers that have been deleted for this reason (because they have been deleted) so this is based on one recent post.
There was a long top level answer that cited several books and quoted relevant paragraphs from each of them to paint a broad and seemingly accurate picture. However, there were complaints about his sources and soon after the entire thread was deleted. The complaints were:
"The [first] book is written by an economist, not a historian. Hence it is not relevant." This despite the fact that the author had received a Nobel Prize for somewhat related work and had first-hand experience of the events being spoken about.
"The [second] book is written by a journalist, not a historian. Hence it is not relevant." The person complaining did not cite any other reasons why the book was bad. He even got the name of the author completely wrong. FWIW, BBC's review of the book was extremely positive called it a "scholarly investigation" that relied on primary sources.
"The [third] book is written by an author whose other works are political. Hence he is probably biased."
Let me be clear that I have no interest in discussing the topic of the thread itself. I only ask if these valid criticisms of the sources? I think I speak for many when I say that clarity on this would be great.
It is not the case that the submission was removed simply because of the sources used, and the comments you reference are from users, not moderators. The topic in question is a highly controversial one, and in our original estimation, we felt that the response, while pushing a narrow, and contentious viewpoint, should be allowed to stand and be contested. However, we soon after received credible reports, confirmed by the Admins, that the submission was part of a coordinated upvote brigade intended to push this controversial stance on the topic of the Famine. While we are not averse to controversial topics, or even posts that entertain views which go counter to the mainstream historiography, we do not play host to astroturfing like this. A response in the future that references those sources would be allowed to stand, if it was part of a larger, balanced explanation of both sides of the issue, or at the very least a less disingenuous treatment of the topic.
I think it would be useful to first say that all sources are biased and that this extends to modern academics too. There is no such thing as an authoritative, definitive and unbiased source, though historians should at least attempt to analyse the evidence in an unbiased way. The job of a historian, and by extension the same goes for top-level commenters here as well, is to look at the available evidence and propose a likely interpretation. Moreover, these interpretations should always be open to criticism. If the answer was built on solid knowledge and good analysis, the historian should be able to convince others of his position, so it isn't that big a deal. But the point is that from an outsider's perspective, civil critiques of a seemingly well-written answer, even if they are ultimately based on the wrong assumptions, are a good thing - we are all here to learn after all.
Like you, I know nothing about the stuff mentioned in the deleted thread, but I think the various complaints are reasonable, though obviously someone who is an expert on that topic would be able to write a more nuanced counter-argument. It is also worth noting that we shouldn't dismiss all flawed sources and deem them irrelevant. I've read many terrible books, but even the worst will have some new insights that might be relevant. Likewise, it is also sensible to have doubts about the 'best' of modern historiography, as no historian can hope to explain every single nuance of their field of study or entirely eliminate their personal bias.
I imagine the books cited in the top answer are all relevant to the topic, but their quality as reliable sources is probably debatable. So my problem wouldn't be that the sources were 'irrelevant' or not, but whether the books cited actually represent how modern historians understand that topic. Some economists can write good history, others can't, so it makes sense to ask if this particularly economist actually knew his stuff. The same goes for the journalist and the political author. If I recall correctly, another complaint was that one of the sources is a Marxist historian, which to me doesn't seem that important, since the Marxist approach to history is a very common mode of analysis and has produced some excellent scholarship even now. But if someone is new here and doesn't know about what Marxist means in historiographical terms, then of course it seems a bit dodgy and worth questioning. I'd say that it is entirely valid to question the credentials of the sources, the key difference between a good answer and a bad answer here however is that someone who knows how to do good history will respond to these complaints and explain why they preferred certain interpretations over others.
Besides Zhukov's very fair mod post explaining about that post, let's discuss the reasons you list as complaints against sources - none of those are in and of themselves good reasons to discredit sources.
Just because a historian didn't write something doesn't mean it's not a valid source. All sorts of people write history. While a PhD in History can mean a seat at the academic conversation table, it is not itself essential. Even worse is when people think that Mrs. X Z is not a valuable source because they are not employed at a university as a 'historian'.
No, rather, anyone who is prepared to engage in serious work with secondary literature must in fact engage on the level of the work itself - is the source good. There usually is a correlation between an author's background and their output, but people not-discipline-trained in history can indeed produce good sources, and historians often produce some very bad ones. Learning to evaluate sources for their own merit is itself part of the discipline.
A good (secondary) source shows (a) broad awareness of the field of study as a whole, (b) the major relevant primary sources, (c) all specific primary sources relevant to the arguments being made, (d) an integrated and thoughtful argument that makes something out of those sources, (e) awareness and critical engagement with other points of view.
If we're just talking about secondary sources cited here in this subreddit, generally I myself am trying to post works that (a) provide the basis for the position I have outlined in my own comment, (b) come from reputable sources because most of the readers are not experts in the field hence they are not likely to be evaluating sources on their own merits but at least somewhat on 'authority', (c) introduce a broader reading in the field of the inquiry so the inquirer can gain a greater understanding than my post can possibly provide.
If an economist or journalist (or anyone else whose primary occupation is not ‘historian’) is doing fairly rigorous historical research, doesn't that make them a historian in that context? Are works that touch upon history by such people as Jared Diamond and Charles Mann pretty much worthless?
Long time lurker, making an account to post this. I'm still extremely confused why that thread was deleted. A previous thread with a similar question on that topic had an answer from an actual fansite of one of the individuals involved yet that answer was allowed to stay whereas the extremely detailed one yesterday was deleted. A lot of the questions & answers on here are heavily Euro-centric too, it would be nice if the mods didn't delete any answers which didn't support their existing views. Maybe we need an /r/AskAsianHistory or /r/AskAfricanHistory subreddit for questions about colonialism.