Solving the problem that the topmost comments get all upvotes

This is unequivocally the most interesting comment in the world, and yet here it lies at the bottom...hmmm, call me a skeptic.

That can be fixed, and is already fixed on this website :-)

Details: I think the approach suggested on this page should be combined with algorithms that take into account that one doesn't know how interesting a comment is, after only a few votes. Only after many votes, it's possible to make a good estimate.

One way to do this, should be to sort by the lower bound of a confidence interval of the true score of the comment. — This is actually how the discussion system on this website already works :-) So the problem you're referring to, should have been largely solved already, on sites that use this discussion system.

Also Reddit has solved the problem you're mentioning (we've chosen the same solution, independently of each other).

Sometimes "highly" disagreed with comments can be interesting to read as well (to see what kind of opinion many people find untrue), so perhaps both could be pushed to the top -- comments with a high upvote to view ratio, and comments with a high downvote to view ratio.

In your case, your comment might remain high with only a single down vote, until either (1) time passes and more people view it without voting on it, or (2) up and down votes even out.

aawww sad story. better luck for next time

Like it

That assumes people read all comments but they don't. Imagine a page with 1000 comments — most people read the topmost comments, but I'd guess that less than 1 in 1000 read all comments. Taking only elapsed time (or page views) into account, would unfairly favor the topmost comment with a factor of > 1000, and would not work well.

One needs to take both page views and position on screen into account.

It might, however, be a good idea to let a vote decay somewhat with time. Comments might get obsolete after some years (for example). So it might be a good idea to favor newly posted and popular comments, even if they're not quite as popular as a many years old comment. — I'd guess, however, that letting comments and articles be wikis, is good enough. (Then one can fix links that get broken, etcetera)

I'm not sure. Long posts do take longer to read, so fewer people tend to upvote them. I was thinking that, on the whole, the effect you're mentioning and the effect I'm mentioning cancel each other out.

However, my actual implementation might not work very well. Perhaps measuring which comment a visitor is reading, works well mainly on mobile phones (where only 1 comments is shown at a time).

I read an article by Paul Graham that discussed "good" and "bad" comments (the "Comments" section in the article), and he wrote that: "There is a strong correlation between comment quality and length", and "Whatever the cause, stupid comments tend to be short."

Re 1: One can use mathematics, to take into account how many people have read a comment. If only a few people have read it and upvoted it, then we really don't know for sure if it is interesting, and we won't give it a score of 100% interesting (1up/1view).

The mathematics stuff is called "using the lower bound of a binomial proportion confidence interval" — here's an article about it: How not to sort by average rating.

(This discussion system already works in that way actually :-))

Re 2: With problem 1 solved, I think this won't be that much of a problem.

Re btw: Fairly many people seem to prefer vertical layout rather than horizontal. I'm thinking about making this configurable per website.

It's a DAG graph actually I think :-) And a directed graph with cycles if one can edit one's comment and refer to what people said, later on.

I've been thinking about rendering a graph instead of a tree, but that feels... a little bit too crazy (and hard to implement)

Actually that's not how I'd implement it. Instead, one can use mathematics, to take into account how many people have read the comment: if only 1 person has read it, then we really don't know for sure if it is interesting, and we won't give it a score of 100%.

The mathematics stuff is called "using the lower bound of a binomial proportion confidence interval" — here's an article about it: How not to sort by average rating.

Initially I was thinking about considering all visitors, like you suggested. Instead of considering only visitors who voted on something. But I'm afraid this would use up too much disk storage space (since the computer would have to store information on each and every visitor).

(The read-and-blue-boxes stuff in example 2 already uses client side Javascript.)

It's Utterscroll, here.

Okay, I noticed it's only 88 pages and entitled "Bandit Algorithms for Website Optimization" — then I'd consider reading it soon :-)

Thanks for the suggestion

I think there ought to be — many people apparently like that format. But it'd probably take a while before I get/take the time to implement it. Perhaps someone else might want to do it, if/when Debiki becomes open source. (Under the AGPL license probably.)

That's really cool :-) I googled your name and read about Direct Hit Technologies. I noticed you've studied engineering, law and finance — this seems almost a bit crazy :-) (in a good sense of course).

I found this article from 1999. I noticed you were using information about the user to decide which search results to show. Something interesting would be to use info about the user, to decide which comments to prioritize or highlight. For example, if the user is a rich middle aged man that votes on this or that party, then show highly upvoted comments that dissents with people from that group.

The intention would be to contribute to a more tolerant world, where people better understand others with different opinions.

(Perhaps this might even be doable, if one could gather information via Facebook or something. Hmm but there'd be lots of privacy issues of course. — Or people could contribute info about their own anonymous profiles, voluntarily, if they wanted to participate in "the-understand-others-better" project)

Just want to add, I usually only upvote exceptional comments, or comments I really agree with.

That means there are many good comments that I don't upvote. But on average, it should still work, I would imagine.

What if there are 100 comments, or a forum topic with 100 pages filled with comments? Then people won't read them all and the interesting information is lost forever, somewhere in the middle...

Reddit, SlashDot, StackOverflow (!) + 100 StackExchange sites, and HackerNews are some sites that uses up/downvotes and work well. People do try to get more reputation points, and I think this tends to make them behave well and be more constructive and respectful towards each other.

Measuring the screen time is a really interesting idea. Or rather, measuring how many people read a comment to the end, versus how many people only start reading it. But really hard to implement reliably? Except for mobile phones with a small viewport. Or if there was eye-tracking available :-)

Oops, my intention was that they fit inside the viewport. I'll have to test on my mobile phone again — I made them wider a while ago, because people were complaining that they became too narrow, when they were deeply indented.

If you'd like to think about it for many more days:

At Hacker News, whathappenedto posted links to long articles about how search engines handle a similar problem. — Perhaps many search engine techniques are actually relevant when it comes to sorting comments on a discussion site — if one views the comment sorting problem as a search for the most interesting comments.

Oh, lucky choice of title then :-) I was wondering if it was too long

The problem discussed on this page might actually be a fairly recurring problem: People at Hacker News mentioned that search engines have a similar problem (that they've solved): they use link clicks (instead of upvotes) to estimate which search results are useful, but only the topmost search results tend to be clicked.

Edit: Also see Gary Culliss' comment to the left.