W E L C O M E   T O
                                 .___     .__
_____ _______   ____ _____     __| _/____ |  | ___.__.
\__  \\_  __ \_/ ___\\__  \   / __ |/ __ \|  |<   |  |
 / __ \|  | \/\  \___ / __ \_/ /_/ \  ___/|  |_\___  |
(____  /__|    \___  >____  /\____ |\___  >____/ ____|
     \/            \/     \/      \/    \/     \/
-- Writing, games & side quests - Bart's small website --
            *      .         .
    .
                      *
        *        *
                    .
             _\/_
       _\/_  /o\\
'-,    //o\   |
   ,-_,-|-""'"|'-,_
        |       "  "-,_
                  "    "--.
          *         .
  .            .-.
              ( (       *
        *      '-'
                  *
          _\/_
          //o\  _\/_
            |   /o\\    ,-'
        _,-'|"'""-|-,_-,
    _,-"  "       |
.--"    "

Not many people are mourning for Stack Overflow

The other day the topic of LLMs killing Stack Overflow came up again on Hacker News. Not for the first time, and not the only place it’s been discussed - it’s been talked about a lot in different forums and channels.

And what’s notable about these discussions - given how beloved Stack Overflow used to be amongst software engineers - is the ambivalence, and sometimes the outright hostility of the conversations.

And this obviously got me thinking about my own use and experience of Stack Overflow over the roughly 18 years of its existence, and how that changed over the period.

So let me start at the beginning.

It’s hard to overstate what a breath of fresh air Stack Overflow was when it arrived on the scene in late 2008. Prior to this support and knowledge sharing was spread across a variety of channels of varying quality and effectiveness:

Other sources of help included standard debugging and profiling tools, the documentation (the quality of MSDN and Java documentation 20+ years ago was such that very rarely did I need the online sources), bloody minded trial and error, and your colleagues and friends.

Stack Overflow was different. I remember the launch marketing being very effective because, almost immediately it felt like, there was a strong community of smart, knowledgeable, and engaged developers on the site. It meant that even posts within the days after launch were getting plenty of answers and engagement.

And then there was the usability: for a Q&A site, Stack Overflow trounced the other options. No scrolling through posts and flipping pages in a long running support forum discussion to get the nugget of information you needed. No trawling through endless newsgroup or email discussions. The answer would be just right there at the top, immediately below the question, marked as the answer by the person who asked it.

And if you were submitting a question or a response it had a nice interface with simple to use formatting options.

Overall the presentation was clean and uncluttered, and the gamification through points and voting kept quality and relevancy high.

It felt like overnight it killed the support forum, particularly the phpBB/vBulletin style support forum, at least for widely used technologies. I barely used anything other than Stack Overflow from the moment it was launched, and I remember that sinking feeling when occasionally I’d Google something and the results would force me towards MSDN - or wherever - rather than Stack Overflow.

For the first few years it was genuinely great.

I guess I noticed things starting to change around 2013 or 2014, and the darker and more dysfunctional side of the gamification, as well as its more prosaic limitations, started to show.

I first noticed what was basically karma farming activity by a portion of the sites users. I don’t know how widespread this was but one factor that stoked this was some employers started looking at Stack Overflow scores as part of assessing job applicants for interview. I mean, great, if you want to employ someone who’s going to spend more time on garnering internet points than they’re going to spend on working for you, I suppose. But, also (and bluntly), screw you for amplifying the negative effects of gamification on Stack Overflow and helping to make it an overall less attractive place to contribute, and a less useful site to visit.

But then I also increasingly ran across questions where the accepted answer either wasn’t the best answer amongst those that had been submitted (e.g., didn’t generalise well, not future proof, fragile/hacky, etc.), or had been at one time correct but the evolution of technology had rendered it out of date and no longer best practice, or sometimes just flat wrong.

As time wore on the concentration of mod powers amongst a relatively small cadre of users, who’d amassed tens or hundreds of thousands of points, led to an elitism that made the site incredibly unwelcoming for newcomers. It engendered something that felt like a throwback to the RTFM culture of software engineering and IT from the 1980s and 1990s.

You got issues like:

Blogs like this - a relatively early piece from Stack Overflow themselves - whilst well intentioned and meant to mitigate the excesses of low effort posting, certainly contributed to the situation.

Here, for example, is a key quote from that post:

We believe asking questions on our site is a privilege, not a right. If, after a few fair attempts, you haven’t been able to prove that your contributions to Stack Overflow make it at least … not-worse … then we reserve the right to refuse your questions. If we don’t do our part to cull the bad questions, then we risk alienating the true experts who provide what really matters: the answers!

(Emphasis theirs, not mine!)

So the site became increasingly frustrating to use. It felt like it existed more to curate content than to help people, which is weird for a Q&A site. It was still useful: I found myself happy enough to refer to it, despite having to do more digging and vetting of answers. But I also found myself not wanting to actively contribute because it wasn’t worth the hassle and negativity.

Here’s a story from 2015 - I think the last time I posted - that shows how toxic and frustrating the site had become to contribute to…

I’d always been someone who found more answers on Stack Overflow than I ever posted. Even more rarely I’d ask a question. But, when I did, I’d always try and follow the spirit of the old netiquette rules of yore, do my research first, and think in terms of, if I was a support engineer trying to help with this, what would I need to know?

Because I have supported products, and we’ve all seen enough bug reports and questions come in where we can straight away tell when someone is going to be easy to help - even if they have a particularly tricky problem - versus someone who’s going to prove more challenging to help.

So I had this question about Elasticsearch. I forget exactly what it was: I think it was something to do with sort order and pagination, and we had some fairly gnarly requirements on how to prioritise matches and fields in our search results. This was at a time when the Elasticsearch documentation wasn’t great, and in fact you were actively encouraged to post on Stack Overflow and tag your question to get help if you couldn’t find an answer to your question.

So I wrote out in detail what I’d done, where I’d got stuck, what I’d read and tried to get unstuck, etc. I put in code samples, and queries - all this sort of stuff. It probably took me 30 minutes or more to pull all the information I had together into a coherent post. Then I spent some time checking it over pretty thoroughly and editing for clarity before I submitted.

So what do you think happened?

The very first comment I received was from some insufferable prick saying words to the effect of, “Oh, so you want us to do your work for you, are you going to pay us too?”

Literally, WTF dude? Why even post that? You might not want to help, and that’s fine, but if that’s the case the option to simply move on and go about your day without getting involved is always available. Especially because I had done exactly what I’d been instructed to do by Elasticsearch, and I’d followed the guidance for posting questions from Stack Overflow, and this kind of snippy smartarse no value response is what I get?

As you can probably tell, it still irritates me today (11 years later!), although I think at the time I managed to respond with relatively good grace.

Anyway, I don’t remember what the specific outcome was, other than I didn’t get a sufficiently satisfactory answer from Stack Overflow. As a result, lacking other channels or forums to find help, nor having anyone else in the organisation I was working for - or friends, former colleagues, or associates - with any knowledge of Elasticsearch, and lacking a clear picture from the documentation… I spent roughly a day layering some godawful and somewhat baroque hack on top of Elasticsearch to get the results into the shape we needed. It wasn’t the “right” way to solve the problem: I should have been able to get what I wanted using the Elasticsearch’s query syntax. But I knew it would work (and it did), and I had other high priority work I needed to move onto. The business concerned has continued to be very successful since then so, for all I know, that hack is still the working, in production solution for that search function (because I don’t think anyone would be particularly keen to roll up their sleeves and get any more deeply involved unless they really had to).

So, as I say, I think that might have been the last question I posted on Stack Overflow, and it might be my last contribution to Stack Overflow full stop.

Over the years, because of the issues above, I simply started finding Stack Overflow less and less useful. I’d find I was getting information from other sources: GitHub issues particularly, but even occasionally sites like Reddit. And then of course more and more projects started using Discord for support and discussion. I’ve never really been a big fan of Discord for this because I don’t like walled gardens and, whilst Stack Overflow is a proprietary platform, all of the information on that platform is public, indexed by search engines, and now (of course) in the training datasets of every frontier LLM provider.

Stack Overflow launched so well, and worked so well for its first few years. As I said, it was absolutely a breath of fresh air. But as time went on it felt like Stack Overflow thought their model was the last word in online Q&A and they didn’t want to evolve to address the flaws in their model. Even just making the site a bit more friendly and positive, particularly towards newcomers, would have gone a long way.

Would that have changed the outcome in the face of AI?

Perhaps. Perhaps not. But I wonder if, without all that frustration and resentment that had built up over the years in the community that were at least nominally Stack Overflow’s target audience, whether the collapse would have been so complete. If you look at the graph of new questions per day on Stack Overflow you can see that from the beginning of 2023 AI simply accelerated a downward trend that had begun maybe even as early as 2016 or 2017. And, even without that AI acceleration, if you draw a trendline along the prior decline, it looks like Stack Overflow would have been dead by 2030 or maybe 2031. That’s what makes me think there’s more than just AI at play: the rot, in terms of the attitude and cultural issues, had become entrenched 10 years ago to a point where it started to hurt that site.

In fact the site peaked in 2014, which is weird when you consider that between 2014 and now, there’s been huge growth in the number of software engineers globally (something like 2.5x growth between 2014 and 2026 according to Google), huge growth in the variety and sheer number of technologies we use (some maturation of course), including lots of new technologies, and then also massive evolution of even mature technologies. So there’s lots of questions still to be asked.

Stack Overflow is now - and the more I look at that graph the more I think it perhaps has been for a long time - a site that, like the dinosaur in A Sound of Thunder, is dead but maybe hasn’t realised it yet. It does still get a few hundred questions rolling in each day, but that’s less than 1% of its peak between 2014 and 2017.

So do I feel about Stack Overflow?

Well, as much as I’m in some ways sad to say this, there’s a part of me that sort of feels like good riddance, actually. They kind of did it to themselves. It feels a bit like just deserts.

OK. So that’s true, but it’s not the whole truth - it’s not the way I feel in the round. It’s me thumbing my nose at the vanity of the haughtily arrogant and high-handed mods who abused their fleeting powers for a time.

On the more positive side, Stack Overflow had a good run, they were very successful for a good long while (which is something many technology businesses never achieve at all, never mind sustain for year after year), and I know that the people who founded the site deservedly did very well from it - and I hope (although I don’t know) that the early employees similarly benefitted. Critically, they also - objectively - help a lot of people, probably millions of engineers all told have benefitted from what Stack Overflow built and what Stack Overflow was. And Stack Overflow continued to help them for years and years. I used it almost every day at work for probably 10 years, and was still a pretty regular user for another 3 or 4 years on top of that.

All good things come to an end. Everything we build as human beings will one day disappear. Most of it will disappear within our own lifetimes. Relatively few of us will create anything that outlasts us, except for our children. And, honestly, we need to stop feeling bad about that because it is the very nature of our existence. More than that: it is the very nature of the universe within which we exist.

Filed Under

Category:Musings

Tags:llms · future · ai · web

I'd love to hear your views.

You can drop me a line, or write your own post and send me a link. Anything interesting I receive I'll add here.