Specificity calculation

Post here if you are having any kind of problem with the Stardust@home website.

Moderators: Stardust@home Team, DustMods

Post Reply
rgraf
Posts: 2
Joined: Fri Nov 03, 2006 3:49 pm

Specificity calculation

Post by rgraf »

My Specificity value remains at 100% while the Sensitivity bounces around. I think something is amiss.

ergophobic
Posts: 3
Joined: Thu Sep 13, 2007 11:20 am

Post by ergophobic »

I'm a little bit grumpy right now about my ratings, which it seems are down in the pits! Is there anybody besides me who is curious about the average Sensitivity and Specificity ratings, and how they compare between Phase 1 and Phase 2? Maybe comparing the Top 100 vs. overall?? Seems it would be a simple query of the user database to find out...
Or maybe it would make me even more depressed about my ratings.... :?

Dee
Posts: 12
Joined: Fri Aug 18, 2006 10:03 am
Location: VANCOUVER, CANADA

ratings

Post by Dee »

I'd like to know other people's ratings too. I'm uncomfortable about how many mistakes I make.
My ratings are Spec. at 96%. Sensitivity is only 84%, yet I seem to identify a lot of tracks that other people are not identifying as tracks.
Hopefully some other people will share their scores.
Dee

jelfving
Posts: 12
Joined: Tue Aug 08, 2006 2:34 pm

Post by jelfving »

Out of 220 CMs, my specificity is still 100%, but the sensitivity 82%. We can't complain: they warned us that the rates will go down! And I try to keep in mind, that I'm not looking for CM-points, but for possible tracks in the actual Stardust-movies. Spotting those tracks seem a lot tricker than we were (in my opinion) led to believe: Many of the passed candidate tracks are such as I wouldn't have dreamed of clicking on during Phase I! But that's science.

bmendez
Stardust@home Team
Stardust@home Team
Posts: 530
Joined: Wed Apr 19, 2006 11:28 am
Location: UC Berkeley Space Sciences Lab
Contact:

Post by bmendez »

Indeed that is science. We are also surprised at the subtle nature of the best candidate tracks.

I could do a search on the database to figure the average sensitivity, however right now I'm having difficulty connecting. I'll try again later.

If the average is down about 80% then that means we at about the level where our smallest CM tracks are at the detection threshold for our set of volunteers. So that's pretty cool to know.

-Bryan
"I am made from the dust of the stars, and the oceans flow in my veins"
- RUSH

Pupper
Posts: 8
Joined: Mon Dec 18, 2006 6:27 pm

Post by Pupper »

If the average is down about 80% then that means we at about the level where our smallest CM tracks are at the detection threshold for our set of volunteers. So that's pretty cool to know.
The size of the CM tracks may not be the main reason that average sensitivity is down. It is not that I don't miss the small tracks sometimes, but my main problem is with CMs where the "tracks" don't really look like anything. The grey blob on one CM is nothing and on another CM it is a track. It might be worth while is see what CMs are missed most frequently to determine if size of the track is a factor or quality of the CM.

elainekeefe
Posts: 190
Joined: Fri Aug 04, 2006 4:38 am
Location: Massachusetts, USA

Post by elainekeefe »

Hi Ergophobic, Dee and others:

You may want to check out "Post your stats" in the Community section of the forum. Only a handful of us have been brave enough (or good enough) to post our new Phase II stats!

I already have 25 errors, compared to the 12 I had in all of Phase I. Stats are at 100/97.9. I don't think I'm ever going to see 98%, while my Phase I stats were both in the 99% range. Particularly troublesome to me are those microscopic specks, which these aging eyes have trouble seeing. Also have trouble with multiples, which they are trying to trick us with! LOL. Most frustrating of all is when I click "no track" and then see the track.

Well, I guess this is what they wanted. So happy to oblige!

Post Reply