I'm curious to see what everyone's thoughts are on this.
First of all - speaking personally, I want to acknowledge that this is one of the better META questions I've ever come across. I was a little tempted to change the flair, in fact...
I hope someone else from the mod team can supply some quantitative insight into the impact of the GQ flair. However, I can shed a little light on the logic behind it, and how it's implemented.
The main motivation behind the flair is acknowledging that we're operating in a public history space where - somewhat unusually - creative or analytical input is not the sole preserve of the historians involved. People asking questions here play an absolutely vital role in shaping what the subreddit is, quite literally setting the terms of the conversations we all have. Asking good questions is in itself an art form (an art form which, I would hasten to add, plenty of academic historians suck at). So we do want to have mechanisms to acknowledge and reward users who contribute by asking questions, and this is one of the ways we do that (along with, for example, our Interesting Inquirer flair that we give to people with a track record of asking good questions).
The other key rationale for the flair is increasing the visibility and frequency of good questions that touch on subjects or perspectives that are underrepresented on the forum. It's a long-running joke that sometimes AskHistorians is really just AskHistoriansAboutHitler, which quite aside from getting a bit dull after a while, means that we sometimes find it hard to recruit excellent historians who work in less Hitler-y fields to join us. We've found that having people writing answers about other topics actually tends to be self-perpetuating (reading an answer, after all, provides you with the knowledge to ask more questions!), so by bolstering and encouraging questions that break our usual mould, we hope to expand the histories (and therefore historians) we showcase here. Per my comment above, I don't have the numbers to say how far it has worked, but my unscientific impression is that the GQ flair helps with these issues, but hasn't solved them by itself, which would be a pretty tall order.
In terms of the perceived slight on other questions, I think the relative rareness of the flair mitigates against this - it's rare to have more than one or two such questions near the top of the subreddit. If, say, half of all questions did, then I could see this concern much more clearly.
I've always interpreted the 'Great Question' marker as a question that's by and large different/unique from the normal stream of questions that get asked, as well as one that asks the responder to go in-depth and give us the comprehensive answer we all want to read.
From my point of view at least, after reading questions that come in for the past 2 years, you definitely see a lot of "What did Hitler think about [Insert literally anything]", "Why are Italians so bad at war?", and "Did Christianity cause the Fall of Rome?" and the like, where great versions of these same questions would be more like "What were the consequences of Hitler's views/policies towards the Latvians?", "What were the logistical and supply challenges involved with Italy's fighting on the North African front?", and "What were the social and political effects of Emperor Theodosius' Edict of Thessalonica on the Roman Empire?". Essentially, to me the Great Questions are the ones that ask the responder to dig deeper and give us the comprehensive answers that we all love.
As for the mods method of picking and choosing, that's for them to say, but that's how I've always seen the rationale in what question gets the marker.
Checking the last fifty posts so flaired, I count twenty-four of these as having been answered. A 50% answer rate feels to be above the usual; I'll leave it to the statistically-inclined to see if that's over our usual answer rate. It may itself be a matter of how the question is asked, as some questions are more answerable than others. (Number edited after correction to search method. Remember kids, sanitise your database inputs!)
If anything, the real below-average questions are the ones I deal with that I don't put the "please don't let this post stop you from making your own answer" boilerplate on. (It's boilerplate, but dangit, it's sincere boilerplate.) The recurring "are we benefiting from Mengele/Unit 731" grotesquerie, for instance, or 90% of all Hitler-related questions. The sub does believe that there's no such thing as a stupid question, but fuck me is it blighted soul-destroying to not only have to touch on these matters but do so every time they pop up again. (Now imagine how the people feel who had to write the answers I link to, and consider that Genocide Studies is an actual thing. The soul weeps.)
One of the first things new teachers are taught is that if you're going to say "good job!" to one kid, be prepared for other kids to notice. In fact, it's even used as a way to encourage students who may be having a hard time doing what's been asked of them. They're able to successfully do something and a teacher might give them a public, "good job!" as a way to reinforce that behavior. (There are lots of different schools of opinions on this approach, to be sure.)
Which is to say, yes, it's very likely it makes people get uncomfortable and it's very human for us to wonder why someone else gets praise and we don't as we see ourselves as doing the same thing. However, the benefits of using it outweigh that as what we're doing is reinforcing the behaviors we want to see. I'm in the same headspace as /u/jschooltiger and others as I will GQ questions that reflect a different perspective, demographic, era, or community than is normally seen. I also GQ questions that make me personally go, "huh. I wonder about that, too."
The marking is rare enough that it feels to me like “there are great questions out there, including this one” rather than “these are the only great questions”.
We haven't pulled the stats on that but...yeah, part of the reason to flag an entry with the "Great Question!" flair is to call attention to it and hopefully attract a good answer. Those are the kind of questions we like to see on r/AskHistorians, rather than someone posting their homework, asking for help with their novel, wanting an exact accounting for Hitler's testicles, or creepily specific questions about historical underage marriage or homosexual relationships in ancient Greece.
Each individual mod has their own standards for what constitutes a "Great Question!" flair, but generally they're something that shows a spirit of genuine inquiry, is somehow out of the normal run of questions we get on how great Sparta is or the historicity of Jesus, and maybe is a question that we want to see an answer to too. As for the rest of it:
Does it make people feel uncomfortable because a question not marked can be implied as a average/below average question?
We hope it doesn't make anybody feel bad about their question. It really is very idiosyncratic as far as which questions get flaired - and that way, there's no system to game.
From the perspective of being one of the people answering the questions, the "Great Question" tag is basically a way of highlighting what an actually good question looks like so that our readers learn how think more carefully and imaginatively about history. Questions along the line of 'What did Hitler think of X' are, quite frankly, boring to write about. In my own area of expertise there are a handful of questions about the Crusades that people ask over and over. Although we believe there's no such thing as a dumb question, there is such a thing as a relatively boring question that could be much more interesting with just a little bit more thought behind it.
For example, a question that crops up from time to time is whether medieval people had a concept of democracy. The question usually stops right there. The answer to that question is just 'yes, here are some examples', and that's not interesting to write. It's also not very interesting to read. It would probably take about an hour to gather the documentation needed to give a detailed answer, for a question only a few dozen people will ever read. This is reflected in how such questions rarely reach double digit upvotes even though they're posted to a subreddit with thousands of readers at any given moment.
The kind of questions that get marked as "great" are typically ones where the moderators look at it and go 'that way of looking at X never occurred to me' or 'that's a pertinent question, but not one people generally think about' (incidentally, this meta question is itself a pretty great question). Returning to the medieval democracy question, a better way of putting that might be "I've read that medieval London had mayors, what did a political campaign in 1400 look like?" at which point I could go into how we have extremely limited evidence but all the evidence we do have tells us that political campaigns could be very high strung, and how the city repeatedly reorganised its voting system in various attempts to reduce polarisation and promote the idea of shared community. That's way more interesting, and the "Great Question" tag is used to promote that kind of curiosity.
Others have already said some good things on this, but I would also just point to the META we made announcing it originally, as it lays out some of our reasoning as well.
This really is a great question and one I wish we had more information on. I know from my brief investigations that several of my question asking buddies do try and aim for GQ flair, but it doesn't stop them asking whatever question they have in their minds. Instead I'm told getting the flair is a bonus victory.
A bunch of us went through their post history to see if GQ made any difference and it doesn't seem to make a make difference. Questions that get highly upvoted generally already were getting upvoted before flair, or were on topics that "traditionally" already get upvoted. GQ flair does seem to attract a little bit more attention, but that seems to depend just as much on topic.
So I'm kinda echoing what others have said already, but: on the Digest every Sunday I compile a list of my "own" great questions (the real™ questions), my goal being to highlight the more unusual questions that get asked around here. My standards are different than the mods', but the intent behind my digest and the flair is I think very similar: that while there are a lot questions on here about the same broad set of fields—major wars, the parts of the ancient world that became the bedrock of Western society, Shakespeare's plays and society, literally anything about Hitler, etc.—that seem to largely be building on the topics that we learned about in high school history classes in the US, while largely there are plenty of other topics that aren't nearly as popular. I think this meme from the top posts of all time on /r/HistoryMemes sums it up pretty well, actually: a lot of subsets of history get routinely ignored or drowned out by these more popular topics, which makes it harder to appreciate the other aspects of history.
What the GQ flair does well that Real Questions doesn't do (since I'm looking more broadly at just amusing or unusual questions) is it highlights questions about marginalized groups really well. A lot of questions are about the experiences of white men, it's easy to miss the questions about women's studies or queer studies or non-western countries beyond how colonialism affected them. (As an aside, I do worry that my collection has a pretty western bias, but I'm not sure how much of that is because of my unintentional bias, and how much that's the result of the bias of questions being asked.) For a bit of data:
And honestly, I thought those numbers would be higher, but they're both just under 10% of the questions I've collected since June. There are also a decent handful of questions by other flaired users, and a number of questions that much the theme of that given week, but I didn't think to track those while I was counting. Of course, my collection process isn't at all scientific, and there are certainly plenty of questions that should've made the list and probably some that are less deserving. I didn't really take a look at which ones did and didn't get answered, though, which I guess is technically the point of this thread. But I know that at least a few people have appreciated getting the shoutout, whether their question was flaired or not.
And I think that's what it boils down to. The list—and more importantly, the flair—aren't so much about denigrating other questions as "lesser", but as raising up questions that deserve attention but might otherwise not receive it, questions that the average user might not think is that interesting at first but is actually something worthwhile.
Thanks for weighing in, everyone! I appreciate the time and energy put into several thoughtful responses.
Side note... It is quite ironic that my post got the GQ tag, LOL. I do enjoy a mod's good sense of humor!
Research Proposal
Abstract We^1 propose an A/B test (open label randomly controlled trial) quantitatively assessing the engagement response of "Great Question" (GQ) flaired vs non-flaired posts, including good answer rate, upvote balance, number of comments. Assuming flair has a significant effect on engagement response, we can then assess whether controlled decrease in flair across the cohort causes a lagging increased response rate to non-flaired posts in said cohort and/or overall.
Background A/B testing is a commonly used technique for determining the effect of an experimental vs controlled aspect of an online experience. Assessing engagement response online without the use of controlled A/B testing (i.e. purely correlative reasoning) can have difficulty deriving causal relationships between factors. In the case of /r/askhistorians, it is unclear whether great questions are answered more because they are flared, or whether they are flaired because they are great questions which were already predisposed to quality answers. Similarly, non-flaired posts may receive lower answer rate due to perceived lower quality question, as lower visibility implies lower affirmation and karma gain, reducing overall subreddit engagement and engagement with non-flaired posts.
Methodology A subset of question topics will be selected as the study cohort in order to limit experimental complexity and preserve the integrity of the remainder of the subreddit for future studies. Moderators across /r/askhistorians will be instructed to continue flairing GQ as normal. For a period of 60 days, half of cohort posts with GQ flair will have flair immediately removed by automoderator in order to eliminate experimental burden on moderators and to reduce the possibility of bias introduced by non-blind moderators aware of the experiment. Therefore, the cohort will be divided into three groups: flaired great questions, non-flaired great questions, and non-flaired non-great questions. Cohort post URLs will be recorded for future analysis. Response data will be collected using the official Reddit API.
Analysis After 60 days we will quantify the following:
Ethical Considerations For a period of time some great questions will not receive GQ flair. To mitigate the disappointment effects and avoid confusion, we should configure automoderator to private message posters of such messages explaining that they have received flair but it has been hidden as part of a controlled experiment, and that their sacrifice has contributed to the glory of the subreddit.
Funding & Support We^1 are capable of necessary programming and statistical analysis but additional collaborators are welcome. We will require support from the /r/askhistorians moderator team including access to automoderator configuration and briefing of moderators on flair removal. No funding will be required.
Conflicts of Interest /u/imanauthority is looking for a data science job and will probably put this in his portfolio if the results look pretty
Researcher Bio /u/imanauthority likes math and helped design the current flair notification system used on the back end, credited here.
[1] /u/imanauthority. Interested collaborators can PM.
I do not believe the GQ flair generally increases the probability of being answered. Threads marked as "Great Questions" tend to be very complex and difficult to answer, which is by design. It's a rare redditor who has the simultaneous expertise and patience to write an answer to any given Great Question. Great Questions chosen as the monthly "Best Question" are usually unanswered by the time the contest is complete.
Personally i really like it, it seems to give useful questions a (hopefully) boost to their popularity. A lot of questions get asked again and again here (both good and bad ones), so i assume it's useful for the moderators (at least i appreciate it) to be able to reward and highlight questions that are deemed needed or useful on the subreddit, to encourage niche subjects, new perspectives etc.
That said, i'm not personally more liable to answer a question that has a "great question" mark. I skim through most questions with a certain tag-words related to my field, and answer those i feel i have time and qualifications for. If they are asked in a less than ideal way i'll just clarify some points in the start and frame the question the way i see as a useful basis for an answer as a help to myself.
On the other hand, i am more liable to upvote great question flaired posts, i almost always do no matter the subject. And that is useful - and makes it more likely the question is seen, and thus answered. So in that way i think it influences it, at least i hope so, as it would be a good thing.
I can only speak to how I use the flair as a non-historian who just likes reading the sub.
I always read the question before I notice the flair. If I find the question interesting, I’m going to read the answers regardless of what flair it has.
However, if I’m interested in an unanswered question and I notice that it has the “Great Question” flair, I’m more likely to save it and return later to see if it’s been answered.
Would a simple named change to something like 'Spotlight' instead of 'Great' be enough to remove the negative inflection to those not chosen for the title? Something that says the question is worthy of particular focus reguarless to the 'quality' of the question itself. A boring, frequently asked question that just happens to spawns ten or twenty in-depth. nuanced reponces that get to really focus on overlooked/underrepresented details is certainly worth of highlight, even if the original question wasn't extraordinary.
Also, if the goal of marking the title is to encourgae more original/creative questions then maybe a "Fresh" title would work just as well.
What is this question even asking? Can someone explain what this is about?
