Are reports of StackOverflow’s fall greatly exaggerated?
👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only The Pulse issue. To get full issues twice a week, subscribe here.
Before we start: plenty of people who subscribe to the newsletter utilize a learning and development budget their company provides. In what is neat: The Pragmatic Engineer is now the first-ever newsletter listed on Learnerbly. Learnerbly is an L&D platform hundreds of tech companies use, as it makes administering these budgets much simpler — and it’s also a lot easier for people to request books, courses, or newsletters. If your company already has Learnerbly, you can now request the full newsletter with just one click.
A week and a half ago, machine learning engineer Ayhan Fuat Çelik wrote an interesting analysis titled The fall of Stack Overflow. Stack Overflow shares website data with some of its most active members with reputations higher than 25,000. I’d like to call out how neat this approach is and how it contributes to transparency, even when the data isn’t flattering.
Ayhan visualized this data and observed a definite fall in all metrics: page views, visits, questions asked, votes. Visualized:
The post stated Stack Overflow has lost about 50% of its traffic. However, the traffic data turned out to not account for a Google Analytics change. Allowing for this, the drop would be 35%. Still, the most worrying part of the statistics is not traffic, but the drop in questions asked and upvotes.
I pinged engineers at Stack Overflow to get their thoughts about what’s happening. What they said is that they are not seeing so dramatic a drop, internally, and that data shared with the most active contributors is inaccurate. I also reached out via official channels to Stack Overflow, and here’s what the company told me (the company later published a blog post with some of the below data included):
- 5%: the company wrote “overall, we're seeing an average of ~5% less traffic compared to 2022.“
- 14%: the sharp decrease in traffic in April 2023. The company said: “we can likely attribute this to developers trying GPT-4 after it was released in March.”
- 14%: this is by how much search engine traffic is down, year-on-year.
- A predictable rise and fall, as with any sudden change. When global lockdowns started in 2020, Stack Overflow saw a spike and then a decrease in cloud migration questions and security-related ones. I sense the company is not surprised that AI had an impact on traffic and the types of questions.
- Q&A activity is definitely down: the company is aware of this metric taking a dive, and said they’re actively working to address it.
While the pandemic and its associated lockdowns didn’t pose a threat to Stack Overflow’s business model, coding assistants like Copilot, Cody, ChatGPT and the other alternatives are different. I find myself reaching less for Google and StackOverflow as a first step when building a new project, and instead using an assistant to get started. More complicated problems – and when I can’t get unstuck with the GenAI helper – are reasons why I head for the browser and open up new tabs, which tend to lead to StackOverflow.
Still, GenAI tools are only as good as their training data. And StackOverflow has the world’s richest programming Q&A training data. So the obvious question is will StackOverflow be content with being scraped for this data – as they have been, until now – or will the company turn it into AI products?
Two weeks ago, CEO Prashanth Chandrasekar announced OverflowAI, a set of AI-related products StackOverflow is working on. The company also announced an upcoming improvement: better search, and also 4 AI tools coming to Stack Overflow for Teams.
Stack Overflow for Teams could be the product which benefits most from the GenAI wave. Until now, our attention has been on the “original” StackOverflow public site we all know and use – especially when fixing an annoying bug. However, the increasingly important revenue-maker for StackOverflow is StackOverflow for Teams, a “private” StackOverflow instance.
When it comes to GenAI, Stack Overflow for Teams is getting a lot more love. Here are features the company is building:
- A new Slack integration to give answers to technical questions, using the Stack Overflow for Teams knowledge base
- Stack Overflow for Visual Studio Code: a pair programmer to bring Stack Overflow content right into the IDE
- Enterprise knowledge ingestion: ingesting content from Confluence, Google Drive, Github, and ServiceNow
- Improved search: similar to the makeover of the public StackOverflow product
Stack Overflow will struggle to compete with AI tools scraping tools which ingest all public StackOverflow questions and answers. However, it can compete by offering tools for companies to utilize their own internal knowledge bases, and by ensuring that internal company data doesn’t leave the premises.
Over time, I won’t be surprised if Stack Overflow’s main business becomes Stack Overflow for Teams, and the public Stack Overflow serves as a lead magnet for Stack Overflow for Teams, while generating some – although increasingly less important – income via ads.
Could we see the fall of public Q&A sites as AI tools rise? A striking statistic is just how much the volume of questions asked has dropped. It’s not as if people have fewer questions, it’s just that developers are typing these questions into AI tools, instead.
For public Q&A sites like StackOverflow to thrive they’ll surely need to be where these questions are asked, which will be integrations in Slack that the company is working on, and programming AI agents, which is another thing Stack Overflow is building.
But the use case that will surely thrive is private, company-only knowledge bases. I would be surprised if Stack Overflow can do much about developers turning to AI coding tools first, before visiting it, but integrating private Stack Overflow for Teams into these chatbots seems like a smart strategy for being where customers ask questions.
The one thing left to answer is how will StackOverflow incentivize people to ask more answers? Looking at this broadly, when AI tools can answer most questions, who will write the answers for complex questions which AI tools don’t have answers for? And what will be their incentive to do so?
This was one out of the five topics covered in this week’s The Pulse. The full edition additionally covers:
- What kind of migration is causing a payout outage at Booking.com? Small business hosts on the travel booking platform are waiting more than a month to be paid. Booking.com says a systems migration is the reason for the delay. I talked with engineers at the company and discovered an SAP migration is to blame. Exclusive.
- Amazon gets stricter about enforcing return to the office (RTO.) The online retail giant sent a warning email to employees “not meeting the expectation of joining colleagues in the office at least three days a week.” I talked with engineers about the response to this ominous, unfriendly email. Exclusive.
- Zoom to end remote work: an RTO turning point? Remote work tool Zoom is having most staff return to the office for two days a week. It’s a symbolic turning point which may signal how many companies will operate in a similarly hybrid way, going forward. Analysis.
- A senior engineer/EM job search story. Davidson Fellipe, a software engineer with 15 years’ experience, based in New York, was recently let go. After 350 applications and 85 first-round interviews in 4 months, he secured 3 offers, and has now started his new job. He shares first-hand learnings about navigating the jobs market. Exclusive.
Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 tech newsletter on Substack.