New post indicator (replaced by new s/w)

steve_sa · Nov 23, 2005

slvr,

I thought about this some more... the idea of only relying on cookies for what is read/not read, or even saving that to a db is a good one. There is only one problem with that scneario. One can only save a finite number of cookies (4k is max. cookie size on FireF).. so you cannot cookie everything you have read before.. things drop off. The same with saving things to a db.. you just cannot save everything you have read, too much processing.

So the designers of forums came up with this (I think) brillliant compromise to show posts since last visit. That dramatically reduces the things that have to be tracked and makes it managable. They figured that if you did not view something on your last visit, it was not too important, and on the next visit it will be white. I am heavy user of some forums, and I have never seen this as being a major problem.

There is a perception problem here, since the last visit time has been messed up. But if/when that is fixed, I think most people will find it a reasonable compromise.

You are a s/w enginner, you see a better solution?

steve

steve_sa · Nov 23, 2005

Ned,

Your points are correct, but there is a 'cheat' that works:

Database server side stores all new posts since you last logged out of the forum

don't need to do that at all. You simply mark everything since last visit red, that is how the s/w works (unless last visit time is messed up, and that is what I am fixing right now).

slvr_phoenix · Nov 23, 2005

Database server side stores all new posts since you last logged out of the forum (Logout being either a manual log out or a session time out (which for this to work session time out needs to be reduced to like 3 minutes of inactivity to keep information more accurate)) - That information is then used to correctly mark threads as read/unread from ANY PC as you log back on.

Wow! Bad idea! Bad idea!

What you're saying here is that any time you leave the forum you've just marked all unread posts as read. This is very very bad as these posts haven't been read yet.

Local Cookie data should be used for keeping track of new posts in a session - For example, I read a thread (Cookie data stores visit time (to the second)). Then as I return to the thread index of a sub forum the last visit of each thread should be checked to the last post time. If there has been a new post since my last visit (note I haven't logged out / session timed out) orange light should be shown.

This is especially useless if you have a three minute timeout as you suggest above.

The reason for needing session cookie data is that the forum database cannot be expected to track all visit times of all threads for every user.

Cannot be expected to? Why not? That's the only way you're ever going to get this right without cookies.

And you shouldn't be storing a time stamp. That'd be a waste of space. It should be as simple as each thread has an associated list of user visits stored in a user visit table. Each user visit is really just a pair of integers. The first integer is the user_id key, the second is that user's last viewed message number for that thread. (I suppose you could also store the thread_id in the table and just let SQL queries of the table find the right items instead of linking from the thread table to the visit table.) The coding from that point should be pretty obvious, I'd think.

Each time a read/unread state is to be determined the thread_id and user_id keys are used to query the last message number for that thread. All posts on or before that number are read. All posts after are unread. And from a higher view, any thread with unread posts is marked as unread.

I don't even use SQL much at all and I could code that one in my sleep. It's very simple, and the database only has to store data when it's being used. (As opposed to each thread always having a message number for each user, or worse, each post having a list of users that have viewed that particular post.)

I hope I made myself clear - It's hard to explain :?

And I hope that I've made it clear that any time-based mechanism is counterintuitive and annoying because it'll always be marking non-viewed posts as read when they should be unread just because something timed out and started a new session. If you get rid of the timeout and work purely on actually logging out, that might almost be bearable ... as users like me would then just never log out.

steve_sa · Nov 23, 2005

And you shouldn't be storing a time stamp. That'd be a waste of space. It should be as simple as each thread has an associated list of user visits. Each user visit is really just a pair of integers. The first integer is the user_id key, the second is that user's last viewed message number for that thread. The coding from that point should be pretty obvious, I'd think.

Good idea but does not work. Why, because there are options like 'mark all topics read' which is impossible to write to 10k threads (some forums have that many). Also you sometimes say 'mark all forums read' (let's not even go there). Also we show status lights at forum level (very top level) impossible to query the entire topics tables just to show red/white.

IMHO people who designed forums (phpbb, etc.) probably went thru these scenarios, and looked at the compromises, and the one they designed is the best one.... anything you don't look at in your current session (and you have an hour to fiddle around in the current session), you really don't care much about, and is marked read. Perfect, no, good cheat, yes.

slvr_phoenix · Nov 23, 2005

One can only save a finite number of cookies (4k is max. cookie size on FireF).. so you cannot cookie everything you have read before.. things drop off.

While I agree, having to occasionally wipe my cookies is still a heck of a lot better than a timeout. Obviously, server-side is really the best solution.

The same with saving things to a db.. you just cannot save everything you have read, too much processing.

I disagree there. That's what databases are designed for, is to store and query for data. If the database can't do it efficiently then it's the wrong database engine to use for the scenario.

So the designers of forums came up with this (I think) brillliant compromise to show posts since last visit.

You think it's brilliant. I hate it. I detest it. If there were no timeout involved, it might not be so bad then. But the timeouts make the whole thing generally useless.

Plus, there's still the one basic precept that a time-based system like that will always be counter-intuitive to: that a message is 'unread' until it is actually 'read'. It's both very bad English and logic to call a thread 'read' when the user hasn't even so much as seen in.

That dramatically reduces the things that have to be tracked and makes it managable.

Oh, I don't doubt that. Storing one timestamp for every user is a very small amount of space to take up in the database. However it has it's serious detractions from usability.

They figured that if you did not view something on your last visit, it was not too important

Which, again, is a horrible line of reasoning. How do they know it wasn't important to me? Maybe I only had five minutes to check one highly essential thread. Maybe I got hit by a power surge. Maybe this is a huge freaking forum with tons and tons of sections and even if I spent every waking minute here I wouldn't have the time to go through them all. And again, this is especially nasty with timeouts added to the logouts.

I am heavy user of some forums, and I have never seen this as being a major problem.

I see it as a major annoyance, especially as ToF didn't have this problem.

There is a perception problem here, since the last visit time has been messed up. But if/when that is fixed, I think most people will find it a reasonable compromise.

I think, in general, you've got us by the balls. Either we put up with it or we leave. That however is not 'reasonable compromise'.

You are a s/w enginner, you see a better solution?

Yep. Store it on the server side. I just listed how I'd do it in an above post. This is exactly what SQL is written to do. If it were a flat file system or something awful then I'd see the reason to cut corners. But this is a database. Databases are designed to store and retrieve data, and though it might accumulate to an amount worthy of note, it's still hardly an overflow of data. Worst case scenario, use a versioning tool like CVS to tag before and after the implementation and if it turns out to incite a serious lag (which it shouldn't unless you've got some seriously bad hardware) then wind back to the old code. It should be pretty easy to implement unless you've got some really weird code to deal with.

steve_sa · Nov 23, 2005

slvr, for the reasons I stated in my previous post, it cannot be saved to threads (IMHO 8) ).

read/unread is really not the correct terminology (my bad). It is officially "new posts" --if you mouse hover over orange indicators it says 'new posts'.

There is really no other solution. If the info is saved to user's record in db (vs. threads//threads won't work for prev. reasons), then the LIFETIME activity has to be saved/retrieved. Our servers would not be able to handle that... in fact the user table is only refreshed today very minute and not on every access (to save disk access).

Imagine having 100 users online, and on every access we have to pull in 10-20-30k worth of user info and save it right back... someone selling a cray supercomputer around here :roll:

Jake_Barnes · Nov 23, 2005

Just to test, I logged out at:

You last visited on Wed Nov 23, 2005 12:54 pm (a few minutes behind)

And just logged back in:

You last visited on Wed Nov 23, 2005 2:15 pm (accurate as to log in)

Everything was white ... there must have been some new posts over that hour or so. Just a test.

steve_sa · Nov 23, 2005

It is not fixed. I am in this big debate with slvr and ned

Ok, guys.. I am timing out. I will fix the forumz to the way phpbb works (and all phpbb forums run that way). Then we can have this offline discussion about doing it another way, if we can figure out a better mousetrap

slvr_phoenix · Nov 23, 2005

Good idea but does not work. Why, because there are options like 'mark all topics read' which is impossible to write to 10k threads (some forums have that many). Also you sometimes say 'mark all forums read' (let's not even go there).

Why is this impossible? Combine it with a time stamp for the last time that the user marked all as read. You'd need time stamps for each section, and a global function that iterates through all sections, but that's hardly an impossible complexity to overcome. It only adds a small extra logic step into the determination of (un)read status and a very small amount of extra storage space per user.

Also we show status lights at forum level (very top level) impossible to query the entire topics tables just to show red/white.

This one gets trickier, I admit. But it is hardly insurmountable. I could write a query to do that fairly quickly. If a section_id is added to the view table (so now it's a table of section_id, thread_id, user_id, last_message_num) then a query on section_id and user_id would quickly bring up all that user's views in that section. Search interatively from newest thread in that section for a thread_id match in the view. Filter out any threads in a section with a timestamp past the user's 'clear all' timestamp for that section. Any thread_id not in the list of the user's viewed thread_ids means the thread is unread and you can break out at that point. Any matches means that you have to actually compare the view's last message_id against the thread's last message_id, again, breaking out if there's something new. And as a precaution against a very long search in case everything actually is read, you could probably also limit the search to only the threads in the fiviewable page for that section.

There might be a bug in that logic, but if there is, I can't imagine that bug being worse than the over-simplified timestamp scene we have right now.

IMHO people who designed forums (phpbb, etc.) probably went thru these scenarios, and looked at the compromises, and the one they designed is the best one.... anything you don't look at in your current session (and you have an hour to fiddle around in the current session), you really don't care much about, and is marked read. Perfect, no, good cheat, yes.

IMHO one person's compramises don't/shouldn't apply to all scenarios, especially when a real solution doesn't look all that complicated to me. Maybe a day to threes worth of work (and testing) and a little extra processing time to overcome a cheat that's annoying as all heck.

Jake_Barnes · Nov 23, 2005

Everytime I read a post, my "your last visit" time is reset?

steve_sa · Nov 23, 2005

Everytime I read a post, my "your last visit" time is reset?

Quick question, Jake. As this seems to be an intermittent problem (I cannot reliably replicate) what browser are you using, and also any cookie blockers (e.g. ZoneAlarm Pro)?

Jake_Barnes · Nov 23, 2005

Everytime I read a post, my "your last visit" time is reset?

Quick question, Jake. As this seems to be an intermittent problem (I cannot reliably replicate) what browser are you using, and also any cookie blockers (e.g. ZoneAlarm Pro)?

Firefox and Zone Alarm Pro - but cookie control is set to off, but both internet and trusted zones are set to high ... here's a screen shot:

Jake_Barnes · Nov 23, 2005

As this seems to be an intermittent problem (I cannot reliably replicate)

This may seem intermittent ... but it's been chronic/always since the forum problems yesterday.

steve_sa · Nov 23, 2005

The image did not make it. Can you post it again.

Jake_Barnes · Nov 23, 2005

The image did not make it. Can you post it again.

I'll repost, but it looks fine at my end. Wait a sec ...

If it doesn't show correctly, here's the direct url link;
http://img261.imageshack.us/img261/8171/zacookiessettings5fc.jpg

steve_sa · Nov 23, 2005

Can you pls test without zonealarm, and tell me how it goes... first thing in debugging is eliminate any variables. I just turned off my ZA Pro as well.

image is good.

steve_sa · Nov 23, 2005

Never mind.. I see the problem without ZA as well.

Jake_Barnes · Nov 23, 2005

Can you pls test without zonealarm, and tell me how it goes... first thing in debugging is eliminate any variables. I just turned off my ZA Pro as well.

image is good.

Sure ... make a couple of test posts somewhere ...

btw, the " You last visited on ..." issue is still the same, ie. everytime I read or write a post, the time is adjusted to my current time!!!

Now, please do some test posts ... I have ZA off.

Jake_Barnes · Nov 23, 2005

btw, the " You last visited on ..." issue is still the same, ie. everytime I read or write a post, the time is adjusted to my current time!!!

Same re-adjust time for the above post too.

Ned_Flanders · Nov 23, 2005

Then we can have this offline discussion about doing it another way

Oh,... cheers :?

Ned_Flanders · Nov 24, 2005

Wow! Bad idea! Bad idea!

What you're saying here is that any time you leave the forum you've just marked all unread posts as read. This is very very bad as these posts haven't been read yet.

Ok, oversight on my part.

To solve this the database would need to delete the reference to a new post if you view it. That way, if you don't view the post the post reference is still kept and when you later log on to check the forum the correct information can be shown.

This is especially useless if you have a three minute timeout as you suggest above. Surprised

The idea of the 3 min time out is to reduce the delay from where local cookie data is keeping track of things to when the server DB is. 15Mins time out currently is a long time to not be keeping track of anything - 3mins isn't.

Jake_Barnes · Nov 25, 2005

Any chance of getting these "new post indicators" issues resolved?

Edit: An explaination. When I booted my machine after being shut down and logged off for 12 hrs or so, every main Category page was white (nothing). Went to hardware, main pg was white, but a few sub-forums (CPU's, GPU's, etc) showed new post by red color - while nothing was indicated on the main page. Only new post after I logged in this am began showing color - and I tried to view using this acct and a dummy acct.

steve_sa · Nov 25, 2005

To the best of my judgement, the problem has been fixed. I logged in this morning, and my last visit is shown as 11 PM of yersterday which is right. Everything posted after that date is orange, which is how it should work.

Jake, I cannot replicate what you are saying... works fine for me. When reporting on this bug, always check the last visit time, and report that too. Everything posted since last visit time should be orange.

There may be a better solution in the horizon as suggested by slvr and others. The currenet solutions how phpbb works today, and does not require extensive code change.

Jake_Barnes · Nov 25, 2005

Funny ... not even this post reply of yours was colored. And I was logged off for an hour.

steve_sa · Nov 25, 2005

Jake, PLEASE (!!) tell me your last visit time next time you post about this 8O

New post indicator (replaced by new s/w)

Distinguished

Distinguished

Splendid

Distinguished

Splendid

Distinguished

Splendid

Distinguished

Splendid

Splendid

Distinguished

Splendid

Splendid

Distinguished

Splendid

Distinguished

Distinguished

Splendid

Splendid

Splendid

Splendid

Splendid

Distinguished

Splendid

Distinguished

Share this page