— for the fruitful conversations Logout
Please consider upgrading to a newer version of Internet Explorer (or install Google Chrome, it's even faster).
By KajMagnus, 2013-02-23 14:32:31Z, improved , 2013-06-10 21:52:14Z.
The first comment gets much attention, many votes.

This assumes a discussion system that sorts comments by number of upvotes.

A fairly prevalent problem in discussions online: Some early posted comments are fairly useful, and get fairly many upvotes, but no downvotes (since they are fairly useful). Later on, some really interesting comments are posted — but at that time, the early comments have already accumulated many upvotes.

The early comments are therefore shown first, and continue to gather more and more upvotes, because they receive all attention. The really interesting comments, however, remain forgotten somewhere below, because too few people take the time to scroll down, find them and read them.

A Solution?

This should solve the above-mentioned problem:

The computer counts how many people have read each comment, and takes this into account, when it sorts all comments. — So, we don't sort by upvotes only, but by upvotes divided by attention (that is, how many people have read a comment).

The fairly useful comments mentioned above got many upvotes, but much more attention than upvotes. Therefore the sorting algorithm would move them downwards, and the really interesting comments would take their place — because the really interesting comments have been upvoted very much, compared to how many people have read them.

The result should be that really interesting comments surface to the top of the page, even if they are posted much later than the early comments.

Broken Solutions

Please note that dividing by downvotes won't work, because early posted fairly useful comments get virtually no downvotes, only upvotes.

Promoting new comments, in some manner, until they have received a few ratings, won't work well, I think. The result would be top of the page covered with lots of new mediocre comments?

Two Examples

How does the computer know which comments you've read? Here follows two examples of how the computer could deduce which comments you've read.

First example. In the picture below, if you upvote the comment outlined in orange, the computer would assume you've read all comments leading up to that comment, plus any siblings placed before it. Those comments are outlined in blue.

The computer thinks you've read the blue comments, if you upvote the orange comment.

Second example. The computer could assume that you read comments that are shown on screen. Have a look at this demo page. In the demo, a red square indicates that the computer thinks you have not yet read the comment. A blue square indicates that the computer thinks you've read it. Look at the demo page for a while, and watch the squares change color to blue.

In the two examples above: When you upvote a comment, the computer thinks that the other comments you have read (the blue ones) but did not upvote, are not terribly interesting.

This problem elsewhere, and solved?

Search engines have a similar problem. They estimate the usefulness of a search result link, by counting how many people click the link. However people tend to click the topmost links only. Here are some articles about how search engines approach this problem (I have not yet read them in whole):

  • Cascade Model (one of the original click models)
  • Dynamic bayesian network model (a more generalizable Cascade Model). As far as I can tell, example 1 above reminds of the simplified model described in section 5. A SIMPLIFIED MODEL.
  • DBN model with scroll and hover interactions (reminds of example 2 above)

(Thanks to whathappenedto at Hacker News for posting these links.)

Your Thoughts?

Is it a good idea to take into account how many people have read a comment? And not consider only upvotes and downvotes. Are there problems I've overlooked?

(This article is being discussed at Hacker News.)

63 comments Logout
  1. Reply
  2. #6 By Anonymous??, 2013-03-24 05:50:48Z, improved , 2013-03-24 10:39:18Z. like this comment.

    I was an early poster on a political discussion and got a single down vote. That pushed my comment down to the lowest imbedded level. So unless you were showing level 7 comments it didn't even show up. Totally unfair that one disagreeing thumbs down buried my comments.

    1. #8 By Anonymous??, 2013-03-24 05:54:29Z. likes this comment. 1 person thinks it is off-topic.

      This is unequivocally the most interesting comment in the world, and yet here it lies at the bottom...hmmm, call me a skeptic.

    2. #194957 By Anonymous??, 2013-03-24 07:44:32Z. likes this comment.

      Sometimes "highly" disagreed with comments can be interesting to read as well (to see what kind of opinion many people find untrue), so perhaps both could be pushed to the top -- comments with a high upvote to view ratio, and comments with a high downvote to view ratio.

      In your case, your comment might remain high with only a single down vote, until either (1) time passes and more people view it without voting on it, or (2) up and down votes even out.

    3. #258097 By Sweet Anonymous??, 2013-03-24 06:17:37Z.

      aawww sad story. better luck for next time

  3. #220253 By Gary Culliss?, 2013-03-24 07:04:09Z. like this comment.

    I developed a similar technology for search results back in the late 1990's that considered how many times a result was selected by users relative to how many times a result was shown to users. We used similar methods of counting the links above a selected link as being shown to the user, as well pagination breaks, time and other metrics. You can then use ratios of the selections over the views for ranking, but the resulting number will be too volatile at low counts, so you need to build in some padding to account for that fact. Also, in search where results are more persistent, we took account of time by keeping track of the age of clicks and views, and expiring them after certain rolling periods of time. Anyhow, it's neat to see you come up with the idea for news. We applied it everywhere, including things like category listings and even to determine if something might contain inappropriate content.

  4. #860092 By Matt?, 2013-03-24 09:49:06Z. like this comment.

    Very smart solution. In fact, I hadn't recognized it was a problem until I read the title of your post... then it clicked immediately. Great idea & would definitely surface more interesting comments.

    1. Click to show this thread
  5. #7 By Anonymous??, 2013-03-24 05:51:02Z. likes this comment. 1 thinks it is wrong. 1 person thinks it is off-topic.

    So True

    1. #232216 By Anonymous??, 2013-03-24 06:43:06Z.

      Like it

  6. #133754 By funflow?, 2013-03-24 11:32:44Z. likes this comment.

    Make reading comments a 2D navigation problem? Could be interesting

  7. #154057 By Anonymous??, 2013-03-24 11:55:33Z. likes this comment.

    Cool idea. I'm curious whether the comments that are further below would tend to naturally have a higher "vote / see" ratio because users who go that deep tend to be more engaged anyways.

  8. #165015 By Anonymous??, 2013-03-24 06:11:13Z. likes this comment. 1 person thinks it is off-topic.

    Wow this page is really broken for people who like to highlight text with this mouse...

  9. #388660 By Anonymous??, 2013-03-24 13:48:30Z. likes this comment.

    I'm guessing you could also play with javascript to detect comments thst have had screentime. That way you might even get rid of voting - just prefer comments that have more screentime.

    And as another commentor said, you also need to solve the problem with one malicious downvote.

  10. #684957 By Anonymous??, 2013-03-25 17:58:14Z. likes this comment. 1 person thinks it is off-topic.

    i didn't even know there was a horizontal scrollbar! lolz

  11. #3 By Anonymous?, 2013-03-24 05:08:58Z.

    good thoughts, thanks for this!

  12. #4 By Anonymous?, 2013-03-24 05:23:06Z.

    I think it's a good idea, and I don't see any problems with it, no obvious ones at least.

    One improvement I could see is giving less weight to a comment as more people see and upvote it. But that would require experimentation.

  13. #5 By Anonymous??, 2013-03-24 05:32:29Z. 1 person thinks this comment is off-topic.

    you're a genius.

  14. #9 By InternetUser??, 2013-03-24 05:57:48Z.

    Let's see if it works. Actually reddit sort by "best" for instance: the score is proportional to the number of upvotes and the date it was posted.

  15. #11 By Anonymous??, 2013-03-25 20:00:48Z.

    I think it's good, but I think that the replies to comments should be shown to the right of the original comment, rather than scrolling down to see them.

    The reason for that is because it is still more convenient to scroll down than to scroll to the side, even with your fancy system in place. This is because of the page down and page up keys.

    When a lot of low-content comments are posted, as is currently the case, it's important to have a great way to sift through them quickly. Threads, on the other hand, tend to need less space, and therefore can occupy the less searchable horizontal axis.

  16. #12 By Hopefully helpful??, 2013-03-26 06:20:18Z.

    The solution is simple - the weight of a vote should decay over time. It should affect up votes and down votes identically. This is more difficult to implement, but it is essentially what you're trying to approximate with your approach (number of reads being an approximation of the passage of time) and doesn't require trying to guess what's been read.

  17. #102670 By anonymous?, 2013-03-24 10:42:15Z.

    Doesn't this confuse long post with interesting post? Now a short, earth-shattering response will get no visibility vs long-winded replies...

  18. #116980 By Bastian?, 2013-03-24 18:29:24Z.

    well - i see two problems here:

    1) if you have some collusion of people who upvote their comments in groups it would be very easy to dominate the comments section with new comments ... you have to factor in absolute values somehow because 1000ups/1000views is far better than 1up/1view

    2) sorting this way will result in a very fluctuating comment section which probably isn't very usable - also most people dont take the time to equally consider all comments (not scrolling down e.g.) when giving upvotes ... maybe the solution should be to increase / decrease the font size of good / bad comments to make good stories more visible while still maintaining some order.

    btw) going from left to right like on this side is equally unsuitable -- but the reddit-style comment system with threading and hiding bad comments is the best i know

  19. #134243 By Anonymous??, 2013-03-24 06:22:47Z.

    only 1 person sees your thread and upvotes = 100% so top comment for next person.

  20. #140887 By sdoering?, 2013-03-24 14:00:27Z.

    Well, interesting idea. But I would propose that even people reading a comment, but doing nothing, would count as divisor for "upvoterate".

    I know, would be difficult to achive and would have probably use some form of client-side-js to determine, if a comment had been visible in the browser. But seeing a comment and doing nothing, is a strong signal in itself. So you would get a picture more rooted in reality that way.

    And yes, what you wrote is quite a problem for online-discussions, as the first-come first-win situation, does not push the best content to the top.

  21. #174014 By Anonymous??, 2013-03-24 07:00:52Z.

    Can you post your code for this the screen viewport trick?

  22. #217338 By Paelidore??, 2013-03-24 10:51:23Z.

    This seems like an interesting solution, but what do you say to those who prefer a more "forum-based" format of reading? Would there be a more vertical version of this model?

  23. #236855 By Dmitri??, 2013-03-24 07:26:21Z.

    I think it's a great idea. I've had similar thoughts before too, but I'm happy someone actually implemented (a nicely working, at first glance) solution.

    1. #982901 By Dmitri??, 2013-03-24 07:27:47Z.

      Just want to add, I usually only upvote exceptional comments, or comments I really agree with.

      That means there are many good comments that I don't upvote. But on average, it should still work, I would imagine.

  24. #252271 By Justin?, 2013-03-24 07:17:36Z.

    This is a great idea. I think factoring in the number of people that have read a comment does solve the problem: it equalizes the field.

  25. #253257 By Anonymous??, 2013-03-24 07:11:36Z.

    People don't tend to read sideways though. They follow the big F shape.

  26. #267667 By vonskippy?, 2013-03-24 06:54:35Z. 1 person thinks this comment is wrong.

    Upvotes/Downvotes are the bane of all discussion threads. What is this, Junior High? I can read and decide for myself if the comment is brilliant, valid, or just another jackass on the web. Just post the comments in the order they were made, and let the reader decide what to do with them. There's not a single website that uses upvotes/downvotes that hasn't turned into a huge circle jerk session of mutual back patting and popularity contests.

  27. #354014 By Anonymous?, 2013-03-24 11:00:54Z, improved by KajMagnus , 2013-03-25 14:25:55Z. 1 person thinks this comment is wrong.

    F_ck you, it is not the solution.

  28. #610740 By Anonymous??, 2013-03-24 06:17:50Z.

    I think adding a random factor would allow early comments to jump to the top occasionally to give them the opportunity get upvoted.

  29. #781601 By Anonymous??, 2013-03-24 12:12:39Z.

    Too wide representation of comments. Super bad on mobile.

You have been logged in, welcome Anonymous!

Login failed: unknown error

Be notified via email of replies to your comments?

Submitting ...

Comments rated interesting, funny
Your post has not yet been voted on, and was therefore placed below.
Comments rated boring, stupid
(Click this box to dismiss)