True discoverer

This forum is for discussing space science topics related to Stardust@home.

Moderator: DustMods

Allen
Posts: 6
Joined: Thu May 18, 2006 5:05 pm
Location: Canada, eh

True discoverer

Post by Allen »

Hi all and just wondering,

"If at least two of the four volunteers viewing each image report a track, that image will be fed to 100 more volunteers for verification. If at least 20 of these report a track, UC Berkeley undergraduates who are expert at spotting dust grain tracks will confirm the identification. Eventually, the grain will be extracted for analysis. Discoverers will get to name their dust grains." - http://www.berkeley.edu/news/media/rele ... dust.shtml

So who will eventually be the ONE rightful discoverer to name his or her particle :?:

Siegfried
Posts: 105
Joined: Wed May 17, 2006 7:18 pm
Location: La Quinta, California
Contact:

Post by Siegfried »

That's a good question. I've been wondering this myself. Will you put ALL of the discoverers on your scientific paper? Wow, that would be one long front page!
And God said: E = +mv^2 - Ze^2/r ...and there *WAS* light!

WANTED:
Dead or Alive
^-^
( ^ )
Schroedinger's Cat

littlebhawk
Posts: 8
Joined: Thu May 18, 2006 6:18 am
Location: Army Base, Germany

Post by littlebhawk »

It should be the first person to report that there was a track within that movie, and everyone else after that is just verifying that there could be a track in that area of the gel.

Bosniac
Posts: 22
Joined: Fri May 19, 2006 4:05 pm
Location: Sara-is-an-ox; B&H

Re: True discoverer

Post by Bosniac »

Allen wrote:Hi all and just wondering,

"If at least two of the four volunteers viewing each image report a track, that image will be fed to 100 more volunteers for verification. If at least 20 of these report a track, UC Berkeley undergraduates who are expert at spotting dust grain tracks will confirm the identification. Eventually, the grain will be extracted for analysis. Discoverers will get to name their dust grains." - http://www.berkeley.edu/news/media/rele ... dust.shtml

So who will eventually be the ONE rightful discoverer to name his or her particle :?:
Good question, because if you know that four people will be looking at the same thing, you lose that sense of discovery. But, of course, someone has to go over it to make sure you didn't miss something.
You see things; and you say Why? But I dream things that never were; and I say Why not? (B.S.)

bmendez
Stardust@home Team
Stardust@home Team
Posts: 530
Joined: Wed Apr 19, 2006 11:28 am
Location: UC Berkeley Space Sciences Lab
Contact:

Post by bmendez »

The discoverer who gets naming rights will be the first person to identify the particle track.

The way the system will work is actually a little different than what was in the original press releases. Now there is no minimum or maximum number of times a focus movie will be viewed. Instead each movie will receive a score indicating whether it is likely or not likely to contain a particle track. The score each movie receives will be based on the number of times it is flagged as either having or not having a track and will be weighted by each volunteers individual score.

Each volunteer will get a score based on how well they do with "calibration" movies that will be shown to them at random. The calibration movies are movies that we know ahead of time to contain or not contain a track. The volunteer won't know when they are viewing real data or a calibration frame.

This means that a person who simply click, click, clicks through the movies flagging them all as having tracks will end up with a low score and thus how they flag any given movie will have little effect on that movie's score.

We are estimating that each movie will need to be viewed an average of 10 times before it can definitively be declared as either having or not having a track in it. Truth is, we don't know for sure how may times or what the threshold score will be to decide if a movie does or doesn't contain a track until we actually start doing the project.

Folks, this is real science! It's filled with mystery and questions. That's what makes it so fun.

-Bryan
"I am made from the dust of the stars, and the oceans flow in my veins"
- RUSH

DustTrailFinder

"Discoverer"

Post by DustTrailFinder »

Okay, but if the person who just clicks, clicks, clicks and "gets a low score" is the first to identify a movie with a track, does he/she still get the right to name it? I don't really think that would be fair, but I also think that not many, if any, are here to do it that way.

Somewhere along the line I thought that we would actually mark the spot we believe is a track. That would make sense to me, but now I'm wondering if that is possible and part of the program.

Franz
DustMod
Posts: 23
Joined: Fri May 19, 2006 6:27 pm
Location: Netherlands

Post by Franz »

The number of expected particle trails is low, and the number of images is high. Finding one is considered rare. So if someone would find one, it would be natural that they pop the champagne. Whatever scheme is being used, stardust@home should and cannot linger with high expectations. If a positive is signaled one should immediately be informed if one has the 'naming' right or not. If it should be undetermined, then one should be adequately informed of it's progress.

This also applies for the calibration movies. Rating these movies will influence you calibration score. These movies might also contain trails, so being presented with such a movie, signaling a positive, and afterwards realizing it was a dummy sucks. stardust@home should inform a participant directly after scoring a calibration that it was a calibration and what the effects the result had.

There are over 100000 volunteers, you can place your bets that there will be individuals who will mark a large number of movies positive in an attempt to claim a naming right. The system needs to be protected against these false positives. A positive trail is rare, so it would be obvious that excessive positives is abuse. To protect against this type of abuse, I suggest that a person loses the naming right if a false-positive was submitted upto 20 submissions before a true-positive.

In the case that a positive (either true/false) will result in the movie being viewed more often, then someone generating excessive false-positives will generate lots of extra rechecking work. There should be guidelines with regard to banning people who abuse the system. Something like you get nominated for banning if all your submissions consist of 20% false-positives.

In the past I have somewhere read a statement in which it was suggested that rechecking would be performed by individuals who have a high calibration score. It should not be the case that at some certain time, these individuals are only performing rechecks, leaving less/no opportunity to be the first viewer of movies.

To recap:
- Do not play with expectations
- If a person submits a positive, either true or false, then that person should be immediately informed if that resulted in a naming right claim (or not) and why.
- Naming-rights should be forfeited if there is a recent history of submitting false-positives.
- After a person has scored a calibration movie, it should be made clear that this has occurred and what its effects are.
- Submitting excessive false-positives should be demotivated.

DustTrailFinder

Post by DustTrailFinder »

Franz,

I don't see how we can be immediately informed if we have a naming right when we make a positive identification, unless you mean we be notified that we are the first to make it. Of course, if it turns out that it is a false trail, then there would be no naming right.

I like your suggestions, especially about banning people who are submitting a large number of false positives. I think that would be a good idea, even if it's not intentional, since these people will tend to make the project less efficient. Post notification of viewing a calibration movie sounds good, too, since we would not be anticipating the possibility we "discovered" a particle.

Since we don't know the priorities of, other commitments by, and resources available to the people who are managing/operating this program, I don't think we can expect that they will be able to implement any or all of the suggestions we make. But I do think we should make the suggestions, so they have a chance to evaluate the benefits and possibility of implementing them.

Siegfried
Posts: 105
Joined: Wed May 17, 2006 7:18 pm
Location: La Quinta, California
Contact:

Post by Siegfried »

Franz wrote:The number of expected particle trails is low, and the number of images is high. Finding one is considered rare. So if someone would find one, it would be natural that they pop the champagne. Whatever scheme is being used, stardust@home should and cannot linger with high expectations. If a positive is signaled one should immediately be informed if one has the 'naming' right or not. If it should be undetermined, then one should be adequately informed of it's progress.

This also applies for the calibration movies. Rating these movies will influence you calibration score. These movies might also contain trails, so being presented with such a movie, signaling a positive, and afterwards realizing it was a dummy sucks. stardust@home should inform a participant directly after scoring a calibration that it was a calibration and what the effects the result had.

There are over 100000 volunteers, you can place your bets that there will be individuals who will mark a large number of movies positive in an attempt to claim a naming right. The system needs to be protected against these false positives. A positive trail is rare, so it would be obvious that excessive positives is abuse. To protect against this type of abuse, I suggest that a person loses the naming right if a false-positive was submitted upto 20 submissions before a true-positive.

In the case that a positive (either true/false) will result in the movie being viewed more often, then someone generating excessive false-positives will generate lots of extra rechecking work. There should be guidelines with regard to banning people who abuse the system. Something like you get nominated for banning if all your submissions consist of 20% false-positives.

In the past I have somewhere read a statement in which it was suggested that rechecking would be performed by individuals who have a high calibration score. It should not be the case that at some certain time, these individuals are only performing rechecks, leaving less/no opportunity to be the first viewer of movies.

To recap:
- Do not play with expectations
- If a person submits a positive, either true or false, then that person should be immediately informed if that resulted in a naming right claim (or not) and why.
- Naming-rights should be forfeited if there is a recent history of submitting false-positives.
- After a person has scored a calibration movie, it should be made clear that this has occurred and what its effects are.
- Submitting excessive false-positives should be demotivated.
I second that. Where were you when the Martians of Xanthe uprooted the totalitarian rulers over on Elysium? :P
And God said: E = +mv^2 - Ze^2/r ...and there *WAS* light!

WANTED:
Dead or Alive
^-^
( ^ )
Schroedinger's Cat

Nikita
DustMod
Posts: 994
Joined: Wed May 17, 2006 8:33 pm
Location: Indiana, USA

Post by Nikita »

Great job of troubleshooting possible glitches to Startdust and posted volunteers! Boy, I never thought of repeatedly reporting positive results just to get naming rights. While it would be neat to find one and get to name it, I thought of this as just the mission to find one.

My only suggestion would be to send a reminder alert to us whenever we submit a positive that we may be the first or last to view and identify, or that it may just be a calibration movie. Then give a timeframe of when we should expect to hear what the movie was (real or calibration) and the results of the movie, if it was there or not. Oh, and if we were first or not. That may reduce frustrations and prevent you from getting constant requests (demands) for status updates. Also, we would need to know how we will find out - will there be a post on a specific movie number to go to, or contacted via e-mail.... I don't know if you will have a way to track and notify all the individuals who viewed a movie and send out to all positives, false positives and false negatives. Perhaps no one will be notified until a movie is retired and it's number and status are posted, say, all negatives on one spot, all positives on another, all rescanned, ect.

I think we all need to keep in mind that this is science and not a treasure hunt, and as they have already said, it is not a race. Things do take time (as we know while we wait!) It is hard to wait for results, especially if you want to be sure you are doing it right, but wait we may have to do. I don't know if it would be possible for them to notify us as soon as we'd like. They are how many of them and how many of us? We've got them outnumbered! Although we are volunteers and they want to make us happy, we may just have to settle for a glass of milk (Yes, I am in Indianapolis and it is May) and do the champagne in a few weeks, or months.

Am I making sense? I hope so. If not, excuse the post. :wink:
From dust we come

DustTrailFinder

Post by DustTrailFinder »

Hi Nikita

I would prefer to be Jeffersonian, rather than Hamiltonian, about the motives of those who want to participate here. However, I do think that some of the 100,000 or more participants might be seeking to cheat to gain an advantage, and those people should not be rewarded or recognized. In point of fact, I believe that Hamilton saw the world as it actually exists and Jefferson wanted to see it as perfect or able to move toward being perfect. In such a special group as those wanting to participate in this program, I think Jefferson's ideal would be closer to the truth. At least, I would hope that to be true.

I agree with you that this is not a race. However, I also think that a certain degree of competition would increase interest and efficiency in finding trails and accelerating the completion of the project, as long as we can also eliminate those who would make it inefficient by being counterproductive with many false positives.

I am concerned about the fact that fewer than 200 people seem to show an interest during the first several days available to sign up to the message board out of the 100,000 who pre-signed to participate in the program. Even fewer have shown enough interest to actually post comments. I have to wonder how many of those 100,000 individuals will ultimately assist us in finding trails.

Sharqua
DustMod
Posts: 245
Joined: Thu May 18, 2006 1:02 am
Location: Bradenton, FL

Post by Sharqua »

DustTrailFinder wrote:I am concerned about the fact that fewer than 200 people seem to show an interest during the first several days available to sign up to the message board out of the 100,000 who pre-signed to participate in the program. Even fewer have shown enough interest to actually post comments. I have to wonder how many of those 100,000 individuals will ultimately assist us in finding trails.
This happened with Spacewatch. In the beginning there was a surge of interest. After awhile a large percentage of the group sloughed off. What remained was a group of dedicated volunteers who were interested not only in the science, but had the patience to examine images carefully and were not in a particular hurry to "claim" their image.

Spacewatch required that you actually click on the location of the trail in the image. Perhaps Stardust might consider this same strategy? It's one thing to click a button that says "Look at this image again." It's another thing entirely to say "I found an interesting trail HERE." Those folks who are actually trying to find trails should bubble straight to the top, while those who are simply trying to earn a high image count will reveal themselves soon enough.

My two cents,

-Donna

DustBuster
DustMod
Posts: 694
Joined: Wed May 17, 2006 8:12 pm
Location: Horsetown, USA

Post by DustBuster »

I beginning to feel that the best idea would be to drop the discovery/naming rights period.
This task is far too important to jeopardize and backlog the data with people solely concerned with finding/naming a particle or those who would 'click, click, click' in an effort to do the same. THIS IS NOT A VIDEO GAME!
If there's anyone who is not up to the task of spending countless hours thoroughly and accurately examining thousands of non-particulate samples, please save everyone else the trouble of having to re-do your work and leave now.

The true discoverer of ALL of the particles is StarDust and her teams.

Nikita
DustMod
Posts: 994
Joined: Wed May 17, 2006 8:33 pm
Location: Indiana, USA

Post by Nikita »

DustTrailFinder wrote:I agree with you that this is not a race. However, I also think that a certain degree of competition would increase interest and efficiency in finding trails and accelerating the completion of the project, as long as we can also eliminate those who would make it inefficient by being counterproductive with many false positives.
Good point! It is certainly a double edged sword and the folks at Berkeley are coming up with a good balance. I hope that when this is up and going, it all comes together to be an enjoyable experience for us all!
From dust we come

smartsam
Posts: 8
Joined: Sun May 21, 2006 8:39 am
Location: England
Contact:

Post by smartsam »

DustBuster wrote:I beginning to feel that the best idea would be to drop the discovery/naming rights period.
This task is far too important to jeopardize and backlog the data with people solely concerned with finding/naming a particle or those who would 'click, click, click' in an effort to do the same. THIS IS NOT A VIDEO GAME!
Every project like this will have people who will cheat, such is the human nature. It happened with SETI@home and it will happen here. But, rather than dropping the naming idea altogether, maybe a different way how to determine who the lucky person is should be found.

Post Reply