Stardust@home

Sorry to cross-post, but having flagged it up here, think the issue belongs in this topic:

[... ...]

1: My original skill-score proposal was that skill be shown on a percentage scale and, although didn’t specify, envisaged at least one or two decimal points (e.g. 0.00 - 100.00).
The implemented 0.00 – 1.00 scale reduces that sensitivity by 100-fold, meaning that one can remain seemingly for ever within a certain narrow band without feedback as to gradual performance improvement or otherwise.
[I did privately ask the team to let me see their precise implementation of the formula before imposing it, but heard nothing until it suddenly appeared in current form.
And could that, if they added a hidden rounding-up operation, and/or by clicking refresh rather than ‘no track’, explain how both enigmatic leaders of the pack (id & Noname Yet) spookily maintain their perfect & static 1.00 skill levels??? I’ m mostly jealous, but certainly curious].

2: Because of the cumulative nature of the formula, personal experience and those of others such as ERSTRS suggests that the minimum 50 PMs viewed, during which one is implicitly expected to learn all the subtleties of PM track (& hopefully new real) track-detection before being fairly scored, is probably too few. I’d now recommend several hundred, if not 1000!
Essentially, too little credit is being given to gaining in experience over time; too many errors early on and you’re faced with a slow up-hill struggle.

3: Can I now therefore suggest that ‘skill’ should in fact be reassessed and updated at regular intervals; it’s something that hopefully grows, after all. How many Gold-medal Olympic winners would have got that medal if all their previous failures over the years were also taken into account?

I stand by the formula (if the results are presented in more detail – point 1), but would like to propose it be recalculated on a ? monthly / ? 3-monthly basis – either for everyone simultaneously, or based on each individual’s start date.

Thoughts, anyone?
John

Regarding item one, John, yes, I've been calculating my own score in order to track progress or regress. When I do claw my way back to 99%, I leapfrog Patrick and Tritone for some reason, perhaps alphabetical? That can't be correct, so more significant figures would be nice.

Regarding item two, half my "misses" are for PMs on which I indeed clicked, just clicked in the "wrong" spot. With experience, we can learn which are the problem PMs, but worrying about those too much detracts from the "fun" - as we learned too well in Phase IV. However ---

--- regarding item three, I happen to know (and have met in person) a key figure on the Berkeley team who had been kicking around an idea that I supported in the beta testing and that I think you might like even better than your own proposal. I don't know if they decided implementation would be too difficult, but perhaps Dan could check into it to see if it's still viable.

Mike

caprarom wrote:--- regarding item three, I happen to know (and have met in person) a key figure on the Berkeley team who had been kicking around an idea that I supported in the beta testing and that I think you might like even better than your own proposal. I don't know if they decided implementation would be too difficult, but perhaps Dan could check into it to see if it's still viable.

A welcome but teasing response Mike! So just what may that idea be exactly?
John

Well, John, to quote my April 29th reply, "I don't want to usurp the Berkeley Team, so I'll leave any more specific comments to Dan." With so few of us dusting now, I don't know how much energy the Berkeley team would want to divert to tweaking what is in pretty good shape already. Of course, there's always the possibility of further enhancements for Phase VI, assuming the pattern recognition software doesn't improve enough to render us obsolete.

My sincere apologies everyone for being away for so long! I'm saddened to learn John that we miss interpreted how you meant the new Skill Score to be displayed. The team has been away for weeks but I'll try and meet with them this week on these items and get back to you.

As always, thanks for your patience!

Dan

Changes are in the works - but there are some bugs. Will keep trying!

DZ

DanZ wrote:Changes are in the works - but there are some bugs.

Yep, looks like it, so what was the point in adding a double decimal zero to everyone’s skill score before said ‘bugs’ were sorted, and how difficult can it be to program basic sum/subtraction/division?

Can I also reiterate that a percentage scale (0.00 – 100.00) would be more understandable to most people than the current clumsy-looking 0.wxyz format.

And what are the team’s present thoughts about regular updating of skill levels (my point 3 above and as referred to by Mike C)?
John

Not to be picky - OK, maybe a bit picky - but I just want to point out that the skill score should not be confused with a "percentage." Due to the weighting factor, it can be more or less severe than the straight percentage. In my case, I've missed 14 out of 1583 power movies. On a straight percentage basis, my skill score would thus be .991156 (99.1%), but my actual score (not percentage) under the current skill protocol is .987526 - close, but not the same. If you miss a relatively "easy" PM, then you are treated more severely than you would be on a straight percentage basis. Vice-versa if you miss a higher-rated PM. Ya, I guess that is a bit picky. Sorry.

Well Mike, of course I agree about the maths of the formula, since it was my proposal!
The point I hoped to make, though perhaps a trivial one, was simply that the implemented presentation (0.000 - 1.000, as used by statisticians) might instead be more readily understandable (& aesthetic?) if shown as a percentage scale (00.00-100.00). Simply a thought!

John (looking forward to the last two decimal places, then arguing about the issue of reassessing skill as dusters progress!)

[addition] Re Dr Westphal’s latest update, it’s great to hear that there could soon be some more scientific journal papers published specifically to do with the SD@H project and its findings.
Will these papers be freely available on the web or, if not, might they somehow be made available to those amongst us interested in the scientific detail?

You may have noticed that the Skill Score is now four decimal places long (correctly I might add). We are also currently working on a system that will still show the old ("all time") Skill Score (what's currently being shown), but show first and foremost everyone's Skill Score based on their last 30 days of dusting (as long as they have clicked on at least 50 Power Movies in those last 30 days). Before we go live with this system though, we will preview it for a few days and may ask some of you to chime in as well. More on that soon.

Dan