I answered a calibration movie right but was scored wrong

[bmendez]

Updated August 16, 2007

I believe that I answered a calibration movie right but was scored wrong.

A: You must click within a 30 pixel radius of the simulated tracks in order for the click to count as correct. Clicking outside that radius or clicking on "No Track" in a positive calibration movie will be counted as an incorrect answer. Clicking on any feature in a calibration movie that is empty of simulated tracks will be counted as an incorrect answer. Such movies have been pre-screened by the Stardust@home Team and deemed to be empty of possible interstellar dust tracks. All calibration movies should have good focus. However, since choosing the "Bad Focus" option is a bit of a judgement call, doing so will neither be counted toward or against your score on a calibration movie.



Volunteers report that they are being incorrectly marked wrong from time to time on some calibration movies. Some possible causes for this have been identified:

* Using a zoom feature to view the movies at greater than 100% of their size: Depending on the browser and operating system this can cause the coordinates of your click to be recorded incorrectly. If you search using a zoom, try returning to 100% zoom when you click and then back to your preferred zoom level for searching.

* Mouse or Touch Pad problems: Touch pads on laptop computers often have a tapping feature that counts a click when you tap or press down on the touch pad. This could lead to clicks that you did not intend. Try turning off the tapping for your touch pad. Others have discovered that when using a wireless mouse the coordinates recorded for their click would be wrong when the batteries were beginning to drain.

* Mozilla Browsers do not ask for a confirmation of a click: Browsers like Firefox and Netscape do not pop up a confirmation box asking to confirm a click on a movie. Because of this, if you click when you didn't mean to, you might very well be marked wrong on a calibration movie. Extra care should be taken to avoid unintentional clicks when using these browsers.

If you are absolutely certain that you did not make an error, but were still marked wrong please do the following:

Post a message in this thread. Do not send a private message to the Stardust@home Team or one of the moderators of the message board. In the message, please include the following information:

* The ID # for the movie that you clicked on. You can find this by clicking the "back" button on your browser after you notice that your score has changed.

* The coordinates that you clicked on. You can find these recorded in the URL of the page that loads after clicking on a movie. The coordinates will be the last set of numbers in the URL of that page. Here is an example:
http://stardustathome.ssl.berkeley.edu/ ... ds=?164,73
That part right at the end means that you clicked on coordinates (164,73) in the movie previous to the one shown in the link.

* The date and time (in Universal or Pacific Time) around when this happened (very important).

* Your username on the VM.

We will investigate these cases when we can and let you know the outcome.

We have very rarely found calibration movies with incorrect coordinates or with ambiguous secondary features. When they are brought to our attention we will correct them. In this post we list movies to look out for that are waiting to be removed: http://stardustathome.ssl.berkeley.edu/ ... 4465#14465

We have also discovered a very rare (1 in a 1,000) bug that causes the score of a duster to be marked incorrect even though the event is correctly logged in our database. We continue to track down this bug, and thank you for your assistance.

However, it is not practical for us to adjust individual scores in any case. We can and will occasionally update the scores of all dusters using the event database to reconstruct their correct scores.

[bmendez]

I noticed that I seemed to be getting double scored on some calibration movies, and strangely, I was receiving one correct credit as well as an incorrect credit. Lo and behold, I paid attention on the next few movies, and when I identified a track I was given both a correct identification credit and an incorrect identification credit. I did not record the IDs of these videos, but if there is a bug in the system left over from some of the fixes, it seems likely that numerous people are having the same problems and the results may ultimately be skewed.

Thanks for reporting this problem, mr. buck. Others have reported a similar behavior, but we have not been able to track down what is happening, as we cannot seem to duplicate the problem here. If you observe this to happen again, please let us know here what movie #ID it happened on, and what the approximate time and date was.

Thank you,
Bryan

[fschwab]

I think I've figured out the problem:

I had been doing Stardust from my office desktop and my laptop PCs running
Linux. Then I booted up my laptop in Windows mode, so I could listen to
music from iTunes. I started out with a few "no track" selections, then got
the calibration movie errors.

I'm using a relatively new Evoluent ergonomic (vertical) mouse. It seemed
to plug and play just fine under Windows, so I didn't bother installing the
special driver that came with it. I believe the pixel coordinates that the
computer infers when I click on an image may be in error.

Sorry I didn't read the FAQ before posting. That issue is discussed there.
I bellieve the scenario was the same the previous night that I had trouble
with the calibration movies.

- Fred Schwab

[bmendez]

I've got yet another: 4197152V1.
I'm sure there /is/ something in the upper middle, seems like a ring below the surface.

T.

Indeed that is an "empty" calibration frame. Meaning it was pre-screened by the Team and deemed to be empty of tracks.

The feature I think you are referring to looks more like a manufacturing defect or dimple in the surface. It is not in focus beneath the surface the way a track would be.

Several people have expressed frustration concerning empty calibration movies that, while empty of tracks, may contain some other subtle features that some volunteers might think worth further investigation. The frustration is that clicking on those features negatively impacts your scores. We do appreciate having these features brought to our attention, and just because your score is marked down does not mean we will ignore these movies. We can see how many times a movie has been viewed and how many times it was clicked on. If a calibration movie has a higher than average number of "incorrect" clicks we can find that out and investigate it.

The calibration movies are not absolutely perfect. We have been pruning out bad calibration movies whenever appropriate and will continue to do so. The purpose of the calibration movies is to measure how good you are at finding tracks (or recognizing when they are not present). Other features might be very interesting to study as well, but the calibration movies are designed to help us locate interstellar dust tracks firstly.

A final note, as far as we are concerned sensitivity and specificity scores greater than 95% are as good as 100%.

Thanks as always,
-Bryan

[Nikita]

There is no way to review a calibration movie unless you immediately hit the back button. You can review the movies in "your Events" by clicking on them.
If this is the only one you've missed, congrats!
Nikita

[DustBuster]

:? How can I get better at identifying tracks when I get a negative CM and have no way to determine what I missed and therefore keep making the same mistakes over and over.

I think we all need some way to know where tracks in misidentified CM's are so we can learn in order to get better for the time when we do see a real track. HELP us! :?:

Thanks,
Basketcase

(If I understand you correctly)
The negative calibration movies have no tracks in them, so 'missing them' means it was clicked on when the movie has no track.
You can determine if you got it wrong immediately by your score- and then you can use the back button to review the movie again (but you cannot rate it again, you will have to click the link above the VM that reads "Click here to go to a new movie").

Hope that helps.

[ac220]

Got both right and wrong answer for the same click.(maybe there was another one but i'm not sure. But tis one was certainly that bug.)

Movie 5059831V1

Call me stupid, but i do not know, how to see page's URL, not caption in firefox history. Best guess - (7350, 295)

user ac220

Time: around 00:10-00:22 local (EEST (Helsinki, Riga, Tallin))

Browser Firefox 2.0 ( Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 )

[jsmaje]

Sorry everyone for having sown unnecessary confusion.
In what could have turned out to be reckless disregard for my own interests, I decided to see what happened by deliberately clicking 'Bad Focus' on a few obvious positive and negative CMs (subsequently confirmed to be CMs using the appropriate URL).
And what do you know - as Elaine & Bram predicted, I neither lost nor gained a point on any of them! So much for my (& DustSabre's) theory!

So, Howie, you're right that:

Track, Click it for a point, or if you miss it you lose a point!

and

NO Track, and you Click something there, then you lose a point!

But not about:

NO Track, you can hit no track or bad focus and get a point!

If it's a negative CM you'll get a point for saying 'No Track', but neither a point nor penalty for saying 'Bad Focus'.

As DustSabre originally said (& you really didn't, albeit inadvertently, click on the movie), then you may well still have a valid complaint.

[bmendez]

Hi All,

This is indeed the place where I would prefer complaints of incorrect scoring to be reported. Please see the first post in this thread on how to report problems.

For example, one of the things that could be helpful in Howie's case where he believes that he did not click on a negative CM but was marked as if he had, would be to check the URL for the next movie, which contains coordinates for where you clicked on the the previous movie (-1,-1 for no track).

So, if you click "No Track" and notice your score has changed, look at the URL on the current page. The last part of the URL gives the coordinates of where you clicked on the previous page. If it is something other than (-1, -1) then something is indeed wrong.

Thanks,
Bryan

[Wolter]

Bryan just said:

Click on a track and that will result in the coordinates. i.e something between 1,1 and 500,375

Click on No Track and that will result in -1,-1

And i just had a look myself after clicking Bad Focus and found that that results in -2, -2

So there you have it.

[fjgiie]

CM 2603853V1

There seems to be something wrong with this callibration movie. The coordinates are at 420,88 where I think I clicked, not 208,338.

Time was about 10:35am 16 May. logged in as fjgiie
I was watching the "answered incorrectly" and did not see anything that I did wrong. The site was very slow taking my clicks. In other words the movies were loading in about four seconds, but the clicks were taking a good bit longer. You could not log out either. I would wait maybe a minute or two and then just quit without logging out.

Thanks,

[bmendez]

CM 2603853V1

There seems to be something wrong with this callibration movie. The coordinates are at 420,88 where I think I clicked, not 208,338.

Time was about 10:35am 16 May. logged in as fjgiie
I was watching the "answered incorrectly" and did not see anything that I did wrong. The site was very slow taking my clicks. In other words the movies were loading in about four seconds, but the clicks were taking a good bit longer. You could not log out either. I would wait maybe a minute or two and then just quit without logging out.

Thanks,

So you believe that you clicked around about 420,88 but the URL recorded you as having clicked at 208,338?

I'll see if someone can check the database to see what is recorded there.

Thanks for the report.
-Bryan

[fjgiie]

HI Bryan,

I am not sure what happened there, I just did not believe I made an error, that's all.

Just a while ago at 5:10pm 18 May, movie 6799730V1 almost did the same thing. I clicked on what I thought was a calibration movie but did not get a point. After checking my events, it was not there either. My history was still on my machine so I saved these two URL's.
6799730V1 no points, not in events either, track at 398,36

next movie with coordinates listed 399,35

The only difference was I did not get docked with a miss this time, only lost one point. Must be something wrong with movie 6799730V1.

[bmendez]

Hi fjgiie,

Okay, we'll look into this. We looked at that other one (turns out to not be a very straightforward process) and something is indeed a little off. The database recorded that you clicked in the right place. Yet, somehow you saw something different. Not quite sure how that is possible.

We will continue to investigate.

-Bryan

[bmendez]

We have been trying to track down the recent scoring problems that have been reported here.

We have not yet been able to reproduce the problem, as it appears to be quite rare. We did verify in the cases that Evelyn and others posted on the forum that the responses were recorded in the database correctly, but
that the scores were not correctly updated. We have not yet figured out what causes the problem. But since the responses appear to be recorded correctly, we can correct the scores. Unfortunately, in order to be sure not to write over anyone's work, we have to take a snapshot of the database, and the computation takes about 24 hours, so we want to do this when we will be down anyway. We will take advantage of the power shutdown over the weekend to do this. This will result in scores going up slightly in most cases.

We are very grateful for everyone who posted to the forum about this. People gave enough information that we could make some progress to correct it. This is a wonderful example of collaboration in action. It was also very helpful that people were collegial and cordial despite being quite frustrated!

-The Stardust@home Team