Last night it came to our attention that an ad with the title, "Use r/AskHistorians? Win $100 for taking our 5 minute survey on how to improve the subreddit" was being promoted in our community. We want to clarify that we are not affiliated with the survey.
While we have confirmed with the researchers that the survey is part of research study and does not appear to be a scam or phishing attempt, and that it received ethics approval from its institution’s IRB, we have requested they remove our community from the study immediately as we have several concerns.
First, is that while asking for usernames and email addresses is standard operating procedure when offering incentives, there is no indication that the username it asks you to provide will be stored separately from your responses. There is no indication of who is anticipated to access the data. This is a security risk and thus not standard operating procedure in survey research.
Second, there is no indication in the informed consent form that states that you can withdraw consent, or how you would be able to do so. This is also not standard operating procedure in research involving humans.
Third, while the goals stated to us by the researchers when we reached out to them sound interesting and valuable to the reddit community as a whole, we believe the wording of the title, which suggests that results will help improve r/AskHistorians specifically and directly, is misleading. It is not clear to us how the results will improve our community, or indeed, even make their way back to us when the study is complete.
We are highly supportive of research on r/AskHistorians. We are regularly contacted with research requests and often agree to participate after we’ve discussed the project with the researchers and have confirmed IRB approval. Researchers interested in working with us should contact us first, so that we can ensure that research goals align with our interests, those of our community, and that our community will be safe from harm. Not only did that not happen here, but because the survey was posted as an ad we couldn’t remove it until we could confirm that it was indeed an academic survey and not a scam.
We aren’t sure if the ad is still running. If it is, we’d like to be clear that this is not a study the r/AskHistorians mod team has approved or endorsed.
I'm really glad that you've posted this with so much detail, its generally important to educate people both in what human subjects research shouldn't look like and also what it should look like. When I encountered and completed it, on top of these points, I also saw some additional human subjects protections red flags:
I’m not going to lie but I didn’t even see that post, however the dedication to actually track them down and figure out everything about them like this is pretty badass, good on the mods who dig deeper
Hi there, I’m /u/cyclistNerd, the researcher behind the study and ad mentioned here.
It’s clear that my behavior in this case made people feel uncomfortable, and for that, I sincerely apologize. As a longtime reddit moderator myself, my relationships with other subreddits are hugely important to me, and I feel guilty that I’ve damaged them, so with the permission of the moderators or /r/AskHistorians, I wanted to explain some of the context here, as well as detail the steps we (myself and my collaborators) are taking to ensure this doesn’t happen in the future. Of course, I very much value your feedback as well, and am happy to answer any questions you may have in subsequent comments.
First and foremost, I want to make it clear that our project is not specifically targeting /r/AskHistorians or any other individual subreddits. The advertisement that was shown to some members of this community has been retracted at the request of the moderators of /r/AskHistorians and will not appear again.
Some background on this project: We are hoping to understand what redditors’ values are for the subreddits they participate in, and how these values vary from community member to community member, as well as across entire communities as a whole. Because one of the beautiful things about reddit is how diverse it is, we have sought out responses from members of as many subreddits as possible. We’ve recruited participants using a number of different methods, including advertisements shown to members of specific subreddits. We selected /r/AskHistorians because we believe it’s an important part of the broader reddit community, but we’ve also gathered responses from almost 2,000 other subreddits. Part of what we feel makes this work valuable is the number of subreddits that we’re able to include in our survey, but this also makes it infeasible to communicate directly with the moderators of every subreddit whose members chose to participate in our study.
Transparency in Study Personnel and Affiliations
It was never our intention to suggest that we were collaborating with, or endorsed by, any of the moderators of subreddits that our study participants were answering questions about. For this reason, we used the logo of our university on the advertisements’ thumbnails, and we have always disclosed our university affiliations on the first page of the study.
However, it’s clear that the advertisement language was still confusing to some participants. To ensure this misunderstanding doesn’t happen going forward, we’ve changed the language of the advertisements (across all subreddits we’re recruiting from) to remove the implication of direct connection between the results of our study and changes to subreddits. We’ve also added additional details to the first page of the survey, clarifying that our survey is neither run by, nor endorsed by, Reddit Inc. or any moderators of any subreddits.
Informed Consent and Research Ethics
Informed consent is critical to all research with human participants. As mentioned above, our study has been reviewed and approved by the IRB at our institution (the University of Washington), and all researchers involved in the study have taken human subjects research training. We explicitly ask for the consent of participants before starting our survey, after explaining the purpose and goals of the study. As /u/EnclavedMicrostate pointed out, an important element of ethical research involving human subjects is the ability to retroactively revoke consent. We have added additional details to the first page of our study explaining that participants can remove their data from the study at any time, with details on how to do so.
Confidentiality and Data Security
The privacy and security of our participants’ responses is hugely important to us. The first page of our survey informs participants that their responses are kept confidential, and we take this commitment seriously.
Across the board, our data security practices exceed those required by our IRB’s requirements. We follow best practices including keeping computer and storage servers in locked datacenter facilities with controlled access limited only to authorized technical staff, ensuring that these computers are behind a firewall and hence not publicly accessible on the Internet, and limiting access to data to only researchers directly working on the project.
The separation of identifiers (e.g., usernames) from the remainder of the data is generally limited to highly sensitive data whose erroneous disclosure could result in material harm to individuals.
Going Forward
We’re very excited about the potential impact from this research, which we think addresses a very important and understudied topic. I sincerely believe that a better understanding of community values on reddit will help improve future community moderation, for example through moderation policies that incorporate the concerns expressed by community members.
My personal goal as a scientist is to be transparent, accountable, and helpful while conducting high quality research. It’s clear that in this case I haven’t met my own standards, and for that I sincerely apologize. I appreciate your constructive feedback and continued dialogue.