Stack Overflow: Not Convinced
Tomorrow, Stack Overflow is scheduled to go public. Being a long time reader of Jeff Atwood’s and Joel Spolsky’s blogs and essays, I was sceptical, yet interested in how this project would turn out. After all, although I have a strong tendency of favoring the usenet over Q&A web sites and forums, there was the possibility that given their audience, Stack Overflow could become a Q&A site that does not suck like just about all the other forums on the web.
So for the last two or three weeks, I have been a beta tester of the site. I have posted answers to more than a dozen questions and gained around 500 to 600 reputation points, along with a set of badges. My experiences in using the site, however, were mixed:
Others already have written about the fact that it is not the wisest decision to trust the crowd in selecting and upvoting the best answer to a particular question. I fully agree to this decision being flawed and have experienced similar situations on the site by myself, but I will not get into that any further.
You gain reputation for asking questions. This is stupid – asking questions is trivial, why should anyone be rewarded for that? The users posting answers are the only ones that deserve being rewarded.
The signal to noise ratio is pretty weak. There are just too many stupid questions being asked – both RTFM-type questions as well as pointless questions such as “What is your favorite Eclipse shortcut”. Not only that these questions do not add any value to the site, they also undermine the significance of the reputation system: By posting and answering to such questions, you gain credit for nothing – credit that would have been a lot harder to obtain if focussing on real questions only. As a conseqence, the reputation count must be expected to not truly reflect the user’s competence – it only reflects the user’s willingness to post.
Given this experience, I am not so sure about the benefits of the reputation system any more. Sure, it does motivate users to participate. But the fact the you gain credit for just about everything you do motivates users to favor quantity over quality. Odds are that you gain more reputation by posting lots of mediocre or even useless stuff to popular questions than by posting elaborate, good answers to niche questions. Needless to say, this is probably the exact opposite of what Stack Overflow aims at achieving.
But there is a another thing that made me think. Stack Overflow relies on OpenID for authentication, which is a fair choice. With Opera, however, which is the browser of choice for me, the authentication system fails completely. No matter which cookie settings I tried, and whether I used Wordpress, claimid or myOpenId as ID provider, there has been no way to get authentication to work.
So I posted a question asking whether the current login situation is bearable – how it could be possible that the thing failed so badly with Opera and whether OpenID was, after all, a wise choice given that it does not add much to the site intending to be frictionless to use. With around 40 answers, a rather interesing and valuable discussion emerged and as it turned out, I was by far not the only one criticizing the current state of the authentication system. Anyway, one week later, the question is gone – deleted, obviously. Although I do not care much, I tend to find it a bit questionable that this seems to be the way the team deals with criticism.
Finally, I tend to dislike is the organization of the site. Rather than dividing the site into sub-forums, Stack Overflow entirely relies on question tagging for categorization. Although this is a nice idea in theory, it turns out to be insufficient. With something around 700 tags currently being in use, the site has become a mess. In particular, there currently is no good way to filter out all the topics that I am not interested in. Rather, I have to wade through hundreds of posts just to find a handful of questions that I would consider reading in detail.
All things considered, I have pretty much lost interest in Stack Overflow. Although technically being nicely done (except the login), it is, after all, just another forum that fails to catch up with the Usenet. Neither is the website better or more frictionless to use than a decent news reader, nor is the quality of content anywhere near the level of most technical newsgroups.
For the record: Referring to Dave Winer’s theory, I am among the one out of ten who do not seek any advantage in criticizing stackoverflow.