I answered a calibration movie right but was scored wrong

Frequently Asked Questions

Moderators: Stardust@home Team, DustMods

Post Reply
steenkh
Posts: 2
Joined: Fri Oct 20, 2006 3:26 am

Post by steenkh »

Nikita wrote:Hey There Diamond Girl!
Yea, I knew it wouldn't come up, I just wanted to show them it was a CM. When I got dinged, I wrote the message and told everyone in my house that I got scored wrong and that they should fix it soon. Then a wise duster by the name of fjgiie clued me into the fact that there was another, true track there. My response was something along the lines of, "Huh...look at that...well, guess I can just kiss my perfect scores goodby!" :oops: So look at that one carefully, it's not what you think. Oh, and I let the issue quietly drop at home, no one has asked about it - yet.
I have also encountered 1727143V1 and got dinged for noticing the same spot. It may be true that there is another true track on this movie (I think I have found it), but I do not see how we should be able to distinguish which one is true, and which is not. Both should be considered correct, or this CM should be removed altogether.
Steen
Dee
Posts: 12
Joined: Fri Aug 18, 2006 10:03 am
Location: VANCOUVER, CANADA

Re CM 6201084V1

Post by Dee »

Thanks Laserphil. Work goes on. Dee
BruceHull
Posts: 35
Joined: Mon Mar 05, 2007 11:18 am
Location: Silver Spring, MD USA

CM 133844V1

Post by BruceHull »

I clicked on CM 133844V1 at 436,92 and was scored a miss. There are three other spots on this CM but I thought I chose the correct one.
http://stardustathome.ssl.berkeley.edu/ ... rds=?-2,-2
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: CM 133844V1

Post by jsmaje »

BruceHull wrote:I clicked on CM 133844V1 at 436,92 and was scored a miss.
I guess you meant 463,92 rather than 436,92?
Definitely a CM track, the other spots being inclusions, and another one for Bryan's list of infamy.
Rotten luck!
yvart
Posts: 7
Joined: Thu May 24, 2007 4:16 pm
Location: France
Contact:

dobble check calibration movie 8598558V1

Post by yvart »

Movie 8598558V1 has two little tracks in top left corner , but I´ve received a bad point for answered incorrectly ! Please dobble check calibration movie !
I answered the upper left track : bad choise. sensitivity -1
you really dont want that I select this sort of tracks on other movies ?
:lol: (yes - no ?)
I go sleeping now... good night

now :
Cal. Movies Ans. Correct : 88
Cal. Movies Ans Incorrct : 3 (very more difficult in Phase 2 !)
Overall score : 85
Total Viewed : 419
Rank : 402 out of 24624
Specificity : 100%
Sensitivity : 94 %
In Dust we Trust ?
laserphil
Posts: 81
Joined: Sat Dec 02, 2006 8:28 pm
Location: Sydney Australia

Post by laserphil »

Hi yvart,

The track is located at co-ordinates 201,203 - more towards the middle of the frame.

Laserphil
laserphil
Posts: 81
Joined: Sat Dec 02, 2006 8:28 pm
Location: Sydney Australia

CM 9958181V1

Post by laserphil »

About ten days ago I was pinged on this CM. Tonight I was pinged again on the same CM. Can someone in the stardust team confirm that the track is located at 301,338? The movie was viewed at approximately 1004UT on 27/9/07.
I thought these bad scoring movies were to be removed from the database!

Laserphil
Nikita
DustMod
Posts: 994
Joined: Wed May 17, 2006 8:33 pm
Location: Indiana, USA

Post by Nikita »

They will be in time. Best suggestion now is to avoid them by refreshing if you get it. I know, you get in the groove of dusting and last thing you want to do is check the numbers before you answer. But unless there has been newer news, that's the best we can do!
(Groove of dusting...boy, am I telling on my age!)
From dust we come
Crumplestiltskin
Posts: 20
Joined: Sat Feb 03, 2007 5:41 am
Location: Durham, North Carolina

Post by Crumplestiltskin »

Still no reply [from Stardust team] to my last post: Sat Sep 15, 2007 12:43 pm Post subject: Messages in Bottles ?
http://stardustathome.ssl.berkeley.edu/ ... &start=165
(a post which would help all searchers, not just me)

While waiting, I'm avoiding the problems by not searching. It's hard to work for someone, volunteer or otherwise, who doesn't respond to worker's concerns.

I guess that's what we get for thinking.
"I thought these bad scoring movies were to be removed from the database! "_laserphil
SPACE IS THE PLACE !
bmendez
Stardust@home Team
Stardust@home Team
Posts: 530
Joined: Wed Apr 19, 2006 11:28 am
Location: UC Berkeley Space Sciences Lab
Contact:

Post by bmendez »

Crumplestiltskin wrote:Still no reply [from Stardust team] to my last post: Sat Sep 15, 2007 12:43 pm Post subject: Messages in Bottles ?
http://stardustathome.ssl.berkeley.edu/ ... &start=165
(a post which would help all searchers, not just me)

While waiting, I'm avoiding the problems by not searching. It's hard to work for someone, volunteer or otherwise, who doesn't respond to worker's concerns.

I guess that's what we get for thinking.
"I thought these bad scoring movies were to be removed from the database! "_laserphil
My apologies for not responding to your posts sooner. While I was away an undergraduate was checking this post and investigating reports made here. She sent me her thoughts on them. Unfortunately, I have not had a chance to follow up on all of them and have had yet another trip in the interim.

On a few of these movies, like this last one mentioned, there was actually some debate amongst the team whether ones which I felt were bad were actually bad or just super hard.

I need to spend some time (probably an entire day) to follow up on all of the reports. I simply have not had that kind of time in the past two weeks. Again, my apologies.

For the most part, the way things had been shaping up, before my Chicago trip, was that most of the reports were not actually bad movies. They were either instances of delayed scoring, or they were very difficult to spot inserted features. The actual number of bad coordinate CMs was a very small percentage of the total reports.

I would also like to reiterate this post and this post. In light of the need to maintain some measure of integrity of the CMs we are not going to give out the coordinate of features in CMs. Every once in a while I will give a general sense of where to look. But If we did give out the coordinates, that would render them all but useless (not only are there ways to know it's a CM, but you would also know exactly where to click). Also remember, the purpose of CMs is not to test or educate, but only to calibrate individual responses.

I'll work on these reports early next week.

Thanks everyone,
-Bryan
"I am made from the dust of the stars, and the oceans flow in my veins"
- RUSH
laserphil
Posts: 81
Joined: Sat Dec 02, 2006 8:28 pm
Location: Sydney Australia

Re Reply on Bad scoring movies

Post by laserphil »

Hi Bryan,

I appreciate your response to the complaints about these movies and accept that the revealing of co-ordinates would be counter productive to the whole project.

I have revealed to others where I thought the tracks were when other dusters have requested clarification but this will cease henceforth.

You stated that the CM's are not "to test or educate". I would respectfully suggest that this is incorrect. Earlier in Phase 1 it was emphasised that no one has seen a positively identified interstellar dust particle track and that the tracks in the CM's were generated in laboratory experiments. If this is still true then nobody knows that we (dusters) are actually looking for the correct feature. I'm sure that not many of the dusters have actually seen a high velocity particle track other than in this search. The only reference dusters can use are the +ve CM's thus we are 'learning' from the only 'text book' available to us and responding to features which resemble the features of the CM's.

In your post of Sat Sept 1 you referred to the introduction of +ve bias to your data by some dusters knowing in advance that a particular movie was a CM. If the six instances I have referred to in my posts were all incorrectly scored even if I clicked on the track then this has introduced a -ve bias of 27.3% to my sensitivity. I'm sure that this will not be correctable at the end of Phase 2 as all dusters will not have received equal exposure to these movies. I would not even guess at the bias introduced to the high scoring dusters sensitivities!
I think I am going fairly well and am enjoying this experience - it takes me back to my university days doing micro fossil and mineral analysis under the microscope - I seem to be getting most of the CM's but the small ones slip through occasionally!
Will keep plodding on with the dusting. Thanks for the experience.
Laserphil
Crumplestiltskin
Posts: 20
Joined: Sat Feb 03, 2007 5:41 am
Location: Durham, North Carolina

The Wolf Who Cried Boy

Post by Crumplestiltskin »

Mr. Mendez, apology accepted. I'm 'happy' to hear that you'll try to do some catching up next week.

FYI: I do not use a zoom feature, and I do not use 2 or more windows for viewing. I watch the scores/%'s closely from movie to movie [that's what signals that something is amiss], so most of my posted problem movies are probably not 'delayed scoring' instances. Often, after hitting the 'back' button, I could see a track I missed and I obviously did not try to report those to the Stardust@Home team. I consider myself a darn good duster. I too would like my individual responses to be calibrated accurately.
With this in mind, I hope you'll be able to give my posts (with 22 or so odd instances) a good look, even though they've piled up over the last month, making the task a little more 'ominous'.

My 5 unanswered posts all need your attention or I would not have posted them:

Wed Aug 29, 2007 6:36 pm
Fri Sep 07, 2007 11:40 am
Sat Sep 08, 2007 3:14 pm
Tue Sep 11, 2007 12:08 pm
Sat Sep 15, 2007 9:09 am

*summed up on Sat Sep 15, 2007 12:43 pm Post subject: Messages in Bottles ?
http://stardustathome.ssl.berkeley.edu/ ... 4750#14750 .
(Again, a post which would help all searchers, not just me)

I still don't know if your 'list' has been updated, or which, if any, bad CM's have been removed from the server, so if/when I start searching again I suppose there's still the chance that I'll re-encounter some of the same problem movies for the 2nd, 3rd, or 4th time.
I do know that, as with cans of tuna that I buy and eat almost every day, the more I consume (or view, as in Stardust searching) the more likely I am to stumble on a can that's just not quite right. I always call the number on the can to let the folks @ BumbleBee know, give them the codes off the lid, and they send me some coupons.


Really looking forward to your replies next week. Thanks, Crumplestiltskin
SPACE IS THE PLACE !
bmendez
Stardust@home Team
Stardust@home Team
Posts: 530
Joined: Wed Apr 19, 2006 11:28 am
Location: UC Berkeley Space Sciences Lab
Contact:

Post by bmendez »

Greetings Everyone,

I have now gone through all of the reports posted here in this thread. It has taken quite some time to do that, but I feel pretty good about the results overall. I'll not respond to every single post that was made over the past month since it is not really necessary. I'll summarize what I've found:

1) Since the first few weeks following the start of Phase 2, I've found no evidence of any more +CMs with bad coordinates. All of the ones that were identified have now been removed. A big "Thank You" to everyone for bringing them to our attention. Apologies for the time it took to get them out of the system. It was only about 1% of the total count of CMs that were bad, so that is reassuring, actually.

2) A large number of CMs that people have reported here have good coordinates and do not have other ambigous features in the field of view. But, they are extraordinarily difficult to find. In general, if 80% of dusters are finding them correctly, then we think they are fair. There have been just a few cases (3-6) of a CM that had some other ambigous features in it that we have decided to remove. So most of the CMs appear to be just fine.

3) The biggest problem we are having is that there is definitely an occasional delay in the score updating. In fact, the majority of all the reports in this thread are due to this problem. What appears to happen is that a duster will miss a CM (usually one of those really hard ones that they didn't see and simply clicked "No Track" on), but their score doesn't update immediately as it should with the next movie. Instead, the score may not update for as many as a dozen movies later. When the score does update, the duster thinks that they missed the previous movie. A very large number of the movies reported in this thread have been Real movies, which cannot possibly affect your score in any way. Plently of others have been where people have correctly found a track in +CM and then because they see their score having been negatively affected on the next page, they think that they got it wrong. In fact, they got it right, but the one that they missed was several movies back.

We don't know what is happening here and we are going to try and focus on this problem now to see if we can find out what is happening.

One way you can help us is by giving us as much information as possible when reporting a problem. Please give us:
- your username,
- the date and time when the error occured (Pacific time, if possible),
- the movie number when the error occured
--information on your computer set-up (operating system, browser, are you logged in multiple times in multiple windows?, etc.).

The good thing about this error is that your scores are being recorded correctly. They are simply not being updated in a timely manner as expected, which makes it difficult for you to monitor when you may have missed a CM.

Thank you all for your patience and your help.

-Bryan
"I am made from the dust of the stars, and the oceans flow in my veins"
- RUSH
KieranC
Posts: 1
Joined: Tue Oct 16, 2007 11:55 am
Location: UK

Post by KieranC »

Hi there,
I have just done a CM which I feel is not totally correct. There is one dot on the CM that stood out in particular, however as this dot appeared above and below the surface when I zoomed up and down. I marked the CM as "No Track" as this dot appeared above the surface. I then recieved a point for an incorrectly answered CM.

Correct me if I am wrong, but I thought you are not meant to see tracks when you are focused above the surface?

The details are;
ID # : 952390V1
VM Username : KieranC
Date : 17 Oct 2007
Time : 13:05 BST (London) (12:05 Univeral Time)

Many thanks,
Kieran
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Post by jsmaje »

KieranC wrote:Hi there,
I have just done a CM which I feel is not totally correct. There is one dot on the CM that stood out in particular, however as this dot appeared above and below the surface when I zoomed up and down. I marked the CM as "No Track" as this dot appeared above the surface. I then recieved a point for an incorrectly answered CM.
Correct me if I am wrong, but I thought you are not meant to see tracks when you are focused above the surface?
Hi Kieran - I've had the same experience of a 'mid-air' track, see here, but no response yet from the team.

And there is this rather worrying statementfrom Bryan Mendez that "afterall, the CMs are generated by an automated program" (which was certainly news to me).

Given that the whole rationale of this project is that human eyes/brains are supposed to be better than any current pattern-recognition algorithm, how come they are evaluating our efforts against what is quite clearly a poorly-discriminating computer program?

Oh well, I suppose we can only hope it will all come out right in the statistical wash.
John
Post Reply