Stack Underflow
How are we supposed to feel when we compete with AI in performing a task, and are completely vanquished?
We might reflexively throw up all the usual objections - "it makes mistakes, it lacks personal touch, it doesn't do it as well in some specific situations..."
Or, we might celebrate - "isn't amazing that I never have to do that again!"
In this case, the human being it vanquished was me. I have mixed emotions.
In 2020 I founded Artificial Frost and immediately became both the best and worst developer on the team. I would often turn to Stack Overflow (SO) when poorly written or non-existent documentation for some particular library proved impenetrable. SO was one of the most popular sites in which to ask specific technical questions about software languages, libraries or techniques.
Stack Overflow was incredibly popular. On 2 June 2021, it was sold for $1.8 billion.
I preferred SO over Reddit because answers tended to be more detailed, and because of this accidental 'hub answer' feature. I and many other people found Stack Overflow answers via Google Search, so we would find more popular answer pages first, which would often have a rich discussion and back and forth about how to do some particular thing.
Typically, Reddit threads dry up within days of the original question. The original questions on these SO pages could be years old, but it didn't matter because Google Search kept on sending visitors to the page. Those visitors would often provide updated answers and comments.
These hub question pages had thousands of views and a plethora of reasonable/partial answers.
The answers I contributed tended to be "end to end" solutions, which I posted to these hub pages, showing how to do something (relevant to the original question) from end to end. I provided these answers to thank the SO community for the fragmentary answers that I was able to piece together into a solution to my problem.
These thoughtfully rendered answers took a long time to write. They usually had a human touch, a bit of humour, and most important of all, accountability. Users would comment if something in my answer was wrong or didn't work and I would respond.
My reputation grew at a relatively steady pace until early 2025 when it completely stalled.
Around this time, I finally asked my first question. I had no inherent reluctance about asking questions, it's just that every time I had gone to ask a question in the past, I had to first make sure that I had tried everything, and carefully checked my work, and most important of all in the eyes of the SO community, was not asking a question that had already been asked and answered.
The SO community is notoriously critical, so I had to take care that every detail was correct. In the process, each time I would end up answering my own question before hitting "Post".
In any event, in January of 2025 I asked my question and... got no answers or comments.
Ever.
As of this writing my question has 48 views. The chance that anyone is going to be directed to this question by Google Search is microscopic, especially now that Google Search is being replaced by AI answers.
I wondered if the chatbots had eaten Stack Overflow's lunch.
I grabbed my question from SO and plopped it into Claude and ChatGPT. Perfectly reasonable answers were spit out, with sample code and explanatory text, just as in those well-crafted SO answers I used to write.
Once I had this chatbot experience, I knew I was never going to go back to SO.
Ever.
It made sense - if you could just train a model on a combination of documentation, tutorials and maybe some SO or Reddit exchanges, you would be able to generate answers that were much better written than typical documentation, and just as complete and correct.
There's also only so much Python to learn, and crucially, writing code is a task at which AI's shortcomings are uniquely irrelevant.
When we ask about the ability of AI to replace something human, the defeat of Stack Overflow and its human community (including me) gives us some useful information about how and why that happens.
Why the task was so accessible to AI replacement:
Lots of Q&A format training data with voting (as with Reddit)
Technical subject matter with few disputed facts
Online documentation that has the answers providing a Wikipedia-like training task (meaning, if you just trained on the documentation, you'd have the answers)
Minimal need for opinion, point of view, personality
But that is only half the story - what shortcomings in SO made it vulnerable?
Answers often fragmentary, out of date, wrong versions - you had to piece together your solution
Answers often anecdotal, not backed by the documentation ("I just rebooted and it solved the problem")
new questions had little pickup - you were less and less likely to get an answer by asking a new question
oddly insulting and/or confrontational tone of many comments
How was AI victorious
Synthesized online reference material and GitHub code
Used a neutral tone, eliminating the negative elements of SO
Provided end to end answers with multiple examples
This is an example where the absence of humanity works in its favour. AI lacks seven key human traits: Backstory, Friends and Colleagues, Humility and Self-doubt, Accountability and Shame, Humour, Tactile senses and a rooted Point of View. None of these are required or even desirable in answering technical questions about how to write code.
More fundamentally, we must accept that most people prefer chatbot experiences now, over any other means of interacting with information. It's like chatting with a knowledgeable friend who never fails to respond.
I have a huge library of technical notes and was wondering if I should post them somewhere - maybe to my website. But then I realized I would just be spending time on a task that has been, not just rendered obsolete, but obliterated completely.
In the human competition with AI, we have to pick our battles.
Refs:
https://blog.pragmaticengineer.com/stack-overflow-is-almost-dead/
https://web.archive.org/web/20211029175938/https://www.wsj.com/articles/software-developer-community-stack-overflow-sold-to-tech-giant-prosus-for-1-8-billion-11622648400
https://stackoverflow.com/questions/79359131/consistent-datetime-datatype-when-using-both-pymongo-and-flask-mongoengine