Hello everyone! A few months ago, a now departed mod shared some statistical work that he did. While interesting, as a few commenters noted, the methodology was somewhat weak, leading to a likely over estimation of the overall response rates in the subreddit - although likely fairly accurate in its more narrow breakdowns. It was a very interesting project all the same though, and one that I felt needed further exploration, so for awhile now, in my spare time I've been working on what I hope to be a much more accurate look at the /r/AskHistorians subreddit from a statistical perspective.
To start with, I'll cut right to the chase. Popular threads, that is to say, threads which hit the top of the subreddit, consistently receive a substantive response over 90 percent of the time. Overall, looking at all threads in the subreddit, the response rate for the past year has been 39 percent (compared to the roughly 50 percent estimate of the earlier stat job).
Finally, a few general notes.
When I started this project, I didn't know what I was doing, and I was terrible about record keeping. I'm not kidding when I say it was me putting tally-marks on sticky-notes. It is quite possible I made errant marks here and there, but I don't believe there are likely to be any substantive mistakes large enough to significantly misrepresent any of the data here. I am... not a statistics major, although I did have to take a class in college on it. All the numbers are just plugged into Excel, and show whatever Excel spits back out. I rounded where it seemed appropriate, and I apologize if/where I screwed up the 'significant digits' or whatever other things like that...
When checking threads, the decision on the state of the thread was very much a snap judgement - "Is there a response or not?" I looked close enough to make sure it was an actual response, and not an unanswered follow-up, or a shitty joke that we just didn't see the first time around, but beyond that, there is no qualitative evaluation here. A just sufficiently good enough answer to avoid removal gets the same tally-mark that a 5 post magnum opus does. There were a few cases where the answer was deleted by the user, but it was clear that a) the answer had been approved by a mod (the check mark still remains) and b) it was originally a substantive response, as other users had responded to say "Thanks" or ask a follow up, etc. In these cases I did choose to count it as "Answered" as it was at the time, even if the user later chose to delete their account. That said, I don't believe there were more than a dozen of these cases that I recall.
Likewise, there is no qualitative evaluation of why a question went unanswered. A deep, thought-out, highly upvoted question which never got a response is no different in this study then the most incomprehensible, downvoted, or obvious query. Having sifted through quite literally thousands upon thousands of questions over the past month of compiling these stats I can say confidently that there is certainly correlation in (my subjective judgement of) question quality and how likely a response was, but I did not make any notations to that effect. Questions either have a response or they don't, and the why is not pondered.
As you will note, I used two core statistics when judging a thread, the "Response Rate" and the "Answer Rate". The first includes threads which receive a link to a relevant FAQ page, or a previous answer to the same question. There likely can be some debate over which is a more 'honest' stat to use, but I personally believe that the Response Rate is a better representation, as having already existent material does provide the Asker with what they wanted to know. When the linked answer was being linked by the author themselves though, I tallied that as an "Answer" rather than a "Response", as I believe that their presence, which allows for engagement, such as follow-ups or critiques, encapsulates one of the core aspects of getting an answer on the subreddit, so those posts rightfully fit under the "Answered" rubric.
I also calculated the "Ignored Rate", which is threads with NO comments, period, removed or otherwise, and the "Insufficient" rate, which is threads with comments, but neither an answer or a response. This is perhaps the least precise statistic though, since as in other cases there is no qualitative evaluation of what those comment(s) were, so it might be a removed joke, or it might be an unanswered follow up question, or any other number of non-answering possibilities.
Finally, as I said, I have stared at alot of threads to do this. Roughly 10,000 or so (and more to come as I do want to go back further eventually, as well as keep the numbers current going forward). The statistics only represent one aspect of how to quantify what my takeaways were from doing so. I'm more than happy to answer any questions, best that I can, about other thoughts and takeaways I have gained from the insight of doing so.
So now, without further ado, let us get on to the statistics themselves.
The first group of statistics is a study of the Top Posts for a given month. This evaluates the likelihood of responses to the 50 most upvoted threads of a given month, which roughly approximates the threads most likely to have hit the top spot in the sub for that month, and thus be visible on /r/All, or /r/Frontpage. It also evaluates the time in which it took answers to arrive.
TABLE I: Monthly Top Thread Statistics
| Month | Response Rate^1 | Answer Rate^2 | Average Time^3 | Median Time^3 | Max Time^3 | Min Time^3 |
|---|---|---|---|---|---|---|
| 2016-01 | 98% | 94% | 4:41 | 3:41 | 20:32 | 0:19 |
| 2016-02 | 98% | 96% | 6:59 | 5:50 | 21:40 | 1:07 |
| 2016-03 | 94% | 92% | 5:45 | 4:40 | 19:14 | 1:21 |
| 2016-04 | 98% | 90% | 5:35 | 4:55 | 19:09 | 0:42 |
| 2016-05 | 94% | 92% | 6:10 | 5:21 | 15:08 | 0:15 |
| 2016-06 | 98% | 96% | 6:12 | 5:37 | 19:13 | 0:46 |
| 2016-07 | 96% | 90% | 7:46 | 5:53 | 22:04 | 0:50 |
| 2016-08 | 96% | 96% | 6:14 | 4:47 | 2:01:19 | 1:18 |
| 2016-09 | 96% | 92% | 6:44 | 5:39 | 18:16 | 1:34 |
| 2016-10 | 94% | 86% | 7:24 | 6:17 | 23:11 | 0:18 |
| 2016-11 | 92% | 88% | 6:29 | 5:49 | 21:45 | 0:33 |
| 2016-12 | 96% | 88% | 7:19 | 6:05 | 20:54 | 0:31 |
| 2016 AVERAGE | 96% | 92% | 6:26 | 5:22 | 20:06 | 0:47 |
| 2016 MEDIAN | 96% | 92% | 6:21 | 5:38 | 20:43 | 0:44 |
| Month | Response Rate^1 | Answer Rate^2 | Average Time^3 | Median Time^3 | Max Time^3 | Min Time^3 |
|---|---|---|---|---|---|---|
| 2017-01 | 94% | 92% | 7:27 | 6:23 | 1:06:58 | 1:31 |
| 2017-02 | 98% | 94% | 10:51 | 8:10 | 6:07:22 | 1:32 |
| 2017-03 | 92% | 90% | 6:58 | 6:06 | 14:57 | 0:35 |
| 2017-04 | 94% | 90% | 7:19 | 6:48 | 1:00:01 | 0:44 |
| 2017 AVERAGE | 94.5% | 91.5% | 8:08 | 6:53 | 2:05:36 | 1:05 |
| 2017 MEDIAN | 94% | 91% | 7:23 | 6:36 | 1:03:29 | 1:07 |
1. Response Rate is the percentage of questions which receive a response of either an answer, or a link to a previous thread or FAQ section. Other visible responses such as follow up questions are not counted here. 2. Answer Rate is the percentage of questions which receive an answer, excluding responses which link to previous threads or the FAQ, except in cases where it is the original author linking. 3. Time is for the first visible answer that appeared. This excludes comments which are links, and does not factor questions which remained unanswered. When averaging, I excludes outlier threads where the answer was >48 hours after posting. Minimum and maximum only note cases where there was an answer, not a link.
As you can see, the response rate has always remained over 90 percent, and the answer rate has dipped slightly below a few times, but generally stays in the 90s as well. 2017 is slightly lower than things were in 2016, but keep in mind that 2 percentages points represent only a single thread, so it is minor. Interestingly though, the time has gone up somewhat over the past year, although February being a big outlier definitely is screwing up those 2017 numbers!
One interesting thing to note is that generally, the small number which did go without any response were the ones near the lower end of the list here. It almost never happened in the Top 10, and quite rarely even in the Top 20, which helps to further reinforce that popular questions almost always get answered. It just sometimes can take over a day.
As for the questions which recieved no response at all, I did not do any qualitative analysis as to why, but I would note that there are trends in what leads to a question going unanswered despite being very popular. The topic as there are definitely some fields which are just poorly covered by contributors on reddit. And in a few cases, the question struck me as neigh unanswerable for various reasons.
The Second Group of stats is intended to provide a larger snapshot of the subreddit as a whole, highlighting for each month seven days, chosen semi-randomly, to ensure that there is one Monday, Tuesday, Wednesday, etc. for every month. This is a total of 84 days evaluated, or 23 percent of the year if you prefer. I've broken it into two parts, one is raw numbers and one is percentages.
TABLE II: Monthly Snapshot by Numbers
Month | Total Resp.^4 | Total Answer | Total Insufficient^5 | Total Ignored^6 | Total Threads | ---|---|----|----|----|----|----| 2016-05 | 351 | 336 | 132 | 335 | 818 | 2016-06 | 329 | 309 | 119 | 278 | 726 | 2016-07 | 317 | 297 | 136 | 297 | 750 | 2016-08 | 310 | 286 | 127 | 351 | 788 | 2016-09 | 303 | 278 | 119 | 346 | 768 | 2016-10 | 284 | 270 | 121 | 337 | 742 | 2016-11 | 303 | 283 | 138 | 419 | 860 | 2016-12 | 333 | 302 | 128 | 360 | 821 | 2017-01 | 352 | 333 | 120 | 411 | 883 | 2017-02 | 319 | 295 | 143 | 442 | 904 | 2017-03 | 301 | 273 | 143 | 440 | 884 | 2017-04 | 333 | 293 | 147 | 376 | 856 | TOTAL Checked | 3835 | 3555 | 1573 | 4392 | 9800 | 365 Projection^7 | 16664 | 15447 | 6835 | 19084 | 42583 | AVERAGE/Week | 319.58 | 296.25 | 131.08 | 366 | 816.67 | MEDIAN/Week | 318 | 2934 | 130 | 355.5 | 819.5 | AVERAGE/Day | 45.65 | 42.32 | 18.77 | 52.29 | 116.67 |
And the same stats as percentages, rather than the raw numbers:
TABLE III: Monthly Snapshot by Percent
Month | Average Threads Per Day | Response Rate | Answer Rate | Insufficient Rate | Ignored Rate | ---|---|----|----|----|----|----| 2016-05 | 116.86 | 0.43 | 0.41 | 0.16 | 0.41 | 2016-06 | 103.71 | 0.45 | 0.43 | 0.16 | 0.38 | 2016-07 | 107.14 | 0.42 | 0.4 | 0.18 | 0.4 | 2016-08 | 112.57 | 0.39 | 0.36 | 0.16 | 0.45 | 2016-09 | 109.71 | 0.39 | 0.36 | 0.15 | 0.45 | 2016-10 | 106 | 0.38 | 0.36 | 0.16 | 0.45 | 2016-11 | 122.86 | 0.35 | 0.33 | 0.16 | 0.49 | 2016-12 | 117.29 | 0.41 | 0.37 | 0.16 | 0.44 | 2017-01 | 126.14 | 0.4 | 0.38 | 0.14 | 0.47 | 2017-02 | 129.14 | 0.35 | 0.33 | 0.16 | 0.49 | 2017-03 | 126.29 | 0.34 | 0.31 | 0.16 | 0.5 | 2017-04 | 122.29 | 0.39 | 0.34 | 0.17 | 0.44 | Average Year | 116.67 | 0.39 | 0.37 | 0.16 | 0.45 | Median | 117.08 | 0.39 | 0.36 | 0.16 | 0.45 |
4. Total excludes META and Feature threads from the count.
5. Insufficient: This is the questions which did receive replies, but either none remain visible, or else what is visible is not an attempt to answer the question, such as mod warnings, or unanswered follow-ups.
6. Ignored: This covers questions which received no comments at all, visible or otherwise. It also does not make any judgement on whether the question was answerable, or well phrased.
7. 365 Projection extrapolates these numbers to estimate the stats over the entire year period, assuming that it remains consistent with these numbers of course.
As you can see, things are pretty steady here! The number of responses has remained, overall, incredibly steady over the past year. As a rate, it has gone down slightly in that time, which is in large part a reflection of the increase in the number of threads the subreddit gets per day. What is interesting also is that the rate of threads in the "insufficient" category remained very steady, and the increase in the number of threads means more threads just don't get any comments at all. This likely reflects, to some degree at least, the nature of reddit, and only so many threads will get noticed one way or the other.
Finally, here are the stats for each day!
TABLE IV: Monthly Snapshot by Day
Month | Days^8 | Daily Response Rate | Daily Answer Rate | Daily Ignored Rate | Daily Total Threads | ---|---|----|----|----|----|----| 2016-04 | 8th, 9th, 11th, 14th, 17th, 20th, 26th | 44%, 46%, 39%, 45%, 36%, 41%, 47% | 40%, 41%, 39%, 43%, 35%, 38%, 47% | 37%, 29%, 47%, 43%, 47, 38%, 39% | 111, 78, 94, 101, 88, 111, 104 | 2016-05 | 5th, 11th, 15th, 20th, 23rd, 28th, 31st | 38%, 40%, 39%, 52%, 45%, 43%, 44% | 37%, 38%, 38%, 52%, 40%, 39%, 43% | 39%, 49%, 41%, 38%, 41%, 37%, 36% | 141, 125, 107, 115, 114, 98, 118 | 2016-06 | 3rd, 6th, 11th, 15th, 19th, 21st, 30th | 45%, 39%, 40%, 50%, 52%, 53%, 40% | 40%, 37%, 40%, 47%, 46%, 50%, 38% | 39%, 45%, 49%, 34%, 33%, 30%, 39% | 114, 98, 103, 100, 92, 104, 115 | 2016-07 | 1st, 5th, 11th, 17th, 21st, 27th, 30th | 47%, 47%, 45%, 46%, 45%, 33%, 39% | 45%, 42%, 44%, 43%, 39%, 31%, 36% | 29%, 34%, 38%, 38%, 41%, 49%, 44% | 97, 86, 101, 92, 128, 140, 107 | 2016-08 | 2nd, 3rd, 13th, 18th, 21st,26th, 29th | 42%, 36%, 38%, 38%, 48%, 41%, 34% | 40%, 31%, 33%, 36%, 43%, 40%, 31% | 37%, 54%, 48%, 46%, 38%, 44%, 45% | 114, 123, 97, 118, 107, 110, 119 | 2016-09 | 2nd, 4th, 6th, 10th, 14th, 22nd, 26th | 42%, 40%, 46%, 35%, 34%, 41%, 35% | 39%, 40%, 46%, 29%, 32%, 37%, 33% | 44%, 45%, 48%, 42%, 50%, 48%, 46% | 109, 99, 85, 99, 119, 147, 110 | 2016-10 | 4th, 8th, 10th, 14th, 20th, 26th, 30th | 43%, 42%, 30%, 44%, 35%, 39%, 36% | 35%, 40%, 27%, 44%, 31%, 39%, 33% | 45%, 40%, 53%, 37%, 54%, 48%, 45% | 91, 89, 104, 100, 136, 113, 109 | 2016-11 | 2nd, 4th, 6th, 8th, 12th, 17th, 28th | 36%, 43%, 34%, 33%, 25%, 36%, 40% | 34%, 42%, 30%, 27%, 25%, 36%, 37% | 49%, 40%, 45%, 54%, 57%, 53%, 44% | 123, 110, 127, 107, 127, 132, 134 | 2016-12 | 2nd, 4th, 6th, 10th, 12th, 21st, 29th | 45%, 43%, 37%, 41%, 36%, 43%, 40% | 41%, 39%, 33%, 38%, 32%, 37%, 36% | 43%, 38%, 45%, 47%, 44%, 45%, 45% | 126, 124, 120 102, 112, 108, 129 | 2017-01 | 2nd, 8th, 12th, 14th, 18th, 24th, 27th | 36%, 42%, 46%, 32%, 48%, 32%, 35% | 35%, 40%, 43%, 28%, 48%, 29%, 34% | 48%, 42%, 37%, 57%, 35%, 52%, 48% | 140, 129, 123, 127, 126, 133, 125 | 2017-02 | 1st, 7th, 10th, 13th, 19th, 23rd, 25th | 43%, 30%, 36%, 30%, 36%, 34%, 41% | 39%, 29%, 31%, 28%, 34%, 30%, 38% | 43%, 55%, 47%, 51%, 47%, 50%, 47% | 129, 135, 121, 140, 116, 151, 112 | 2017-03 | 3rd, 9th, 12th, 13th, 18th, 22nd, 28th | 31%, 37%, 31%, 38%, 29%, 29%, 41% | 28%, 33%, 28%, 35%, 25%, 27%, 38% | 55%, 48%, 47%, 44%, 58%, 55%, 43% | 142, 140, 109, 127, 102, 131, 133 | 2017-04 | 4th, 8th, 12th, 20th, 24th, 28th, 30th | 40%, 37%, 38%, 36%, 49%, 39%, 34% | 35%, 30%, 33%, 34%, 42%, 37%, 28% | 46%, 41%, 47%, 53%, 33%, 41%, 46% | 126, 113, 120, 126, 118, 119, 134 |
8. Days: These are chosen with a random number generator, with discretion to exclude US Federal Holidays, as these are likely to reflect abnormal traffic and usage patterns, and other days which generally result in 'wonkery' (April Fools for instance). The process is only semi-random, as it represents one of each day for the month (Monday, Tuesday, etc.) and I did my best to avoid consecutive days, although due to poor attention, it happened once or twice. Weekend days are in italics.
I don't really have much to say on this, aside from the fact I find the wide divergence in the same month to be interesting, as I feel it helps to demonstrate how heavily chance plays into things. Some days people are really active answering, some days people are really active asking, and sometimes those overlap well, and sometimes they really don't.
I will, however, apologize that they are percents instead of numbers... As I noted at the beginning, I did a lot of this as tally-marks on sticky notes. And I tossed the sticky notes once I put the numbers in my Excel sheet. It was only after I had done several months when I realized I really ought to have kept these numbers as raw numbers as opposed to percents, but too late by that point, and given the percent of the total, it isn't like there are more than 2 options anyways...
So that is the sum of my studies - up to this point. As I said, I plan to do more number crunching, so would love to hear suggestions on other possible ways to improve this (although I will note that I've considered a number of ideas I threw out due to the hurdles they present vs. my free time). At the very least I want to explore how to look into topic frequency, and have some ideas on how to do that. I'm also happy to chat about the various observations one gains from trawling through 10,000 threads on AskHistorians in quick succession.
I am so excited that you shared this! Like, giddy even! Data like these will help provide context for my qualitative findings, so not only is this just really interesting to me, but also highly valuable.
I also might be able to help with further analysis. In addition to the dissertation work that I've been in contact with you guys about, I'm also part of a team that's using reddit as a way to explore informal learning. We've scraped all the threads from 2015 and 2016 from a few subreddits, AskHistorians included. It looks like there might be some overlap between the kinds of things that we want to know and that you want to know; for example, are there any aspects of questions that make them more likely to get answers (or in our case, lead to dialogue that suggests evidence of learning)? Would you be interested in me bringing this up to my team to see if there's anything we can do?
Okay, can you tell me the platform, software and technique how you collected this data? I wanted to do some similar studies too, but would be great if there was a technology stack already ready for using! It would really save me tons of time and focus on the data analysis more than the technology behind it. Would really appreciate your reply here or message me privately! I am really looking forward to do some data analytics, love this sphere of work!
How long before we get a historian of askhistorian flair?
You inspired me. I just looked at three days of posts that have been at least one day old. At the time I began "Does the new edition of Renfrew and Bahm..." was the first 1d old post, "Foucault and history..." was the first 2d post, and "9th century Wales and 2 handed axes..." was the first 3d post.
One category I counted was "no response". No comments count as "no response". Deleted comments, OP comments alone, follow-up questions alone, mod comments about deleted comments, etc., count as "no response".
Another category is "answered". Almost every post that does not qualify as "ne response" counts as being answered. If OP is answered in part, it counts. If OP is directed to other references/sources, it counts.
The third category is "other". Meta posts are other. Schoolwork questions that are flagged as such count unless someone answers the question in part or in full. Corner cases like the only response is the suggestion to crosspost elsewhere counts as other.
1d old: 39 answered, 80 no response, 7 other.
2d old: 44 answered, 76 no response, 9 other.
3d old: 44 answered, 62 no response, 6 other.
Thanks, this is really cool! I'm curious about the extreme ends of the response times: what was the question that took 6 days to answer in February? What was the question that took 15 minutes to answer last May?
A few months ago, a now departed mod shared some statistical work that he did. While interesting, as a few commenters noted, the methodology was somewhat weak
Perhaps I'm reading this wrong, but you kicked out a mod for weak methodology? You guys are harsh...
Any chance you could dump these stats into a public google doc? Then people can mess around with them and graph them up.
As a methodological observation: holy crap, did you have to review 10,000 questions yourself? Surely there are some suckers volunteers who could be found around here to help!
Seriously though, this sort of analysis lends itself to collaboration. Split 10,000 questions across 20 people and that's only 500 each; few enough that you could do some slightly more time-consuming stuff. For example, you could count how many threads have a response from a topic expert, or precode and count the most popular periods or topic areas (ie. WW2, PTSD, etc.). (I'm sure someone with proper data analysis skills could also do the latter automatically.)
Another thing you could do in the process is focus in on some particular areas. For example, say you're interested in what kind of questions don't get answered. Well you should end up with a list of, say 5,000 threads with no answers. You can then use that as a sample frame and pick a sample of one or two thousand to look at in more detail - eg. coding all the topics, maybe question "type" (if you can figure out a useable categorisation).
This is amazing work!
It would be interesting to know the absolute figures. A question that popped into my mind whilst reading this was: What is the ratio between answers given and questions asked?
This might provide a clue as to why the % of answered questions has dipped. It could simply be that a higher total number of questions get asked, whilst the absolute number of answers (read users answering) stays stable.
I just want to say that, as a constant lurker, the amount of work you did on this is appreciated.
I realize I'm a little late to the party, but man, this is really cool! I had some fun messing around with a few quick graphs in excel, which are here if you're interested.
And here's the results of fitting various parts of the data [Note: I included April 2016 in this, using Table IV, even though it wasn't in Table II or III]:
You also made me curious about the top threads of the past year. Turns out it's quite striking how much the number of really highly upvoted threads has increased in the past few months (not that the mods need me to tell them that, I'm sure - and thanks for keeping up the good work with all the increased attention!). I'd had a general sense that AH was hitting the front page more often lately, and indeed over half of the top 50 threads of the year are three months old or less, and about a third just a month or less.
One question: did you happen to keep the raw data for the times-to-first-answer for the monthly top 50 threads? It would be interesting to see the distributions.
The reason some good answers go unanswered while 'meh' ones get attention is simply a matter of who's online at the time and who feels like they're up to answering. Really good questions might also be the ones that take the most work to answer and some may not feel like it at the moment, and the other questions seem simple enough to answer.
And of course if no one is online to see said 'good question', they're not going to answer.
The best way to remedy this is to place multiple 'flair' categories on a question (similar to Quora's category system) that contain both the region, time period, and subject matter if applicable; all three at once may be optional but recommended. A question must have at least one flair. Then, someone can simply browse questions by a certain flair (e.g '1600s', 'Ming Dynasty', and/or 'Cuisine').
With the current way, intriguing questions are quite simply becoming functionally lost. Flairs for question have been suggested before, but the prevailing argument seems to be having to choose a flair for questions that naturally are going to touch multiple subjects. For that I'm not sure why multiple flairs from a selection of approved categories are a bad idea.
After reading so much AskHistorians, can you tell me what deep insights into history or human race you have gotten, if any?
You mentioned a lot of this is automated, is there a reason you haven't automated this with some kind of script? Seems like data collection could be faster with something else doing the collection.
As my stats work got some attention in another thread, and I don't want to go side-tracking that thread more, I've responded to a few questions brought up here:
Would your stats be a little skewed? This sub locks down any thread that doesn't have someone with a background or enough credible sources. I feel like that create a false positive. /u/dread_pirate_roberto
It really sucks when someone asks a really interesting, but niche, question and the comments are all [deleted]. Is it possible to mark the comments as not having enough sources without removing them all together?
Most threads get upvoted to the top of the sub well before an answer shows up, so the presence of an answer definitely isn't influencing the fact that the question itself was upvoted, if that is what you mean to say, but I confess I'm not totally certain that was your point. In any case, the 'Daily Snapshot' stats, which look at all threads in a 24 hour period, would definitely not be affected by that in any case.
As for marking comments, not really possible with reddit even if we were inclined to do so. That would require custom CSS to be edited for every marked comment, and also not work on Mobile.
What percentage of the total submitted questions are those "top 50 threads";?Because looking at the sub right now, besides an announcement thread, I see only one post with more than three comments; and the vast majority with none at all. /u/Paciphae
Those stats are only intended to deal with threads which trend, and it is pretty much 100 percent of threads which hit the top position of the subreddit. We average 117 threads per day though, so it is only one percent of all threads. I keep a separate statistical measure though culled from 7 semi-randomly chosen days per month, which looks at every thread posted in the 24 hour period, so if you are interested in the larger statistical matter, then jump to Table II and Table III above.
Hello Herr Captain-General Zhukov the Great!
I'm glad you linked to this thread from elsewhere. I didn't realize that there was an active process for archiving/statistical interpretation of AskHistorians. I love to see that there is already a process.
Contextually, my wife and I are huge fans of the subreddit. I am a statistics person by education, and she is a software engineer. We had been casually working on a scraping system for AH with the goal of both preserving the full extent of the subreddit and providing a database for statistical programs/machine learning activities. Clearly, this is something that is in parallel to what you (and I presume other mods) do. To that end, if you have the time/patience I would love to have a chat with you and the moderation team about what features they would appreciate from such an exhaustive compilation of the subreddit and how we could best serve the subreddit's needs vis-a-vis archiving/data analysis. Specifically, "topic frequency" is something we can definitely extract from such a database.
As always, thank you very much for the work you do. I look forwards to maybe making this casual hobby useful.
