determine detection threshold

marcia · Post by **marcia** » Sun Jul 29, 2007 2:08 am

May be this message is all speculation, but I've been thinking about this subject and want to share it. And if there are REALLY more subtle actual candidate tracks, forget everything I said here.
After seeing app. 52.000 real movies and 21.000 CM's I doubt if the team can better determine a detection threshold by giving us dusters more CM's with subtle actual candidate tracks. I am afraid that by giving almost invisible or very, very tiny tracks in CM's (finding the most tiny specks wasn't/isn't the problem, isn't it?), people will get indeed lower sensitivity en specificty scores, so the team has a better choice, BUT I am also afraid that people will be tended to click on Real Movies much more too. And than they have the same problem as in phase 1, where we had to learn what tracks could be and I clicked a long time and too much on inclusions en other weird things I now know there absolutely not idp-candidates.
I think it's much more important to show people all the possible property's of a track. We did have learn something in the past year, isn't it?
What helped me much was a picture of a longitudinal section of a cosmic-dust track, from the page of JPL/NASA-Stardust. I know it's not the same, but it helped me to see the differences with inclusions etc.

But I am also wondering what this tiny candidates can be, so I just looked to all 'past cut 1' movies in My Events-list, may be there was one, or more. I coundn't find one. But from the 33 'passed cut 1-possible idp's' there are only 10 movies I am convinced are tracks or very good candidates and the rest - 23! - I am UTTERLY convinced are no tracks at all!! They are all inclusions or surface dust. It's very possible that I missed all of this this subtle actual candidates, but still...So if someone has such a small candidate, please can you show me?
And BTW, most of the convincing tracks (most LAT's) I have on my events-list, I have found from about december 2006 on. So I wish I could clean up my ME-list!! If we could all clean our ME-list, you guys from the team had lot less work to do!

fjgiie · Post by **fjgiie** » Sun Jul 29, 2007 4:48 am

This post by cthiker shows one of the best looking tracks that I've seen.
Also 1029716V1 has a good chance. (circle left of center)

I have no way of knowing for sure but this star looks very much like it was caused by a fast-mover that splashed or exploded on the surface of the aerogel.

______

marcia · Post by **marcia** » Sun Jul 29, 2007 7:19 am

Hai fjgiie,

May be I'm seeing it wrong, but I never would click on this â€˜small tracks'...
I think they are all surface dust. For if you look to cthiker's movie, you can see that the sharpest image is right on the surface of that part of the movie. And moreover: above and under this â€˜track' you can see a ring getting fuzzier. In my humble opinion you never see that in case of a real track. Under the sharp ring it would stay sharp...
If I'm wrong on these movies, it's may be the reason why I haven't them in my events-list... so I'm still curious with what kind of CM's the team will coming.
Thank anyway for your reaction... see you next weeks on Galaxy Zoo!

cthiker · Post by **cthiker** » Mon Jul 30, 2007 10:51 am

Hi Marcia!

(...and thanks for the vote of confidence, fjgiie!)

Just as a follow up to Marcia's comment, your point about the lack of a "sharp ring" is well taken. However, as you might note in the note (sorry - Dept. of Redundancy Dept.

) I indicated that this was as close to what the CM's were showing as I had found...and that actually still holds in my current experience. However, Nikita brought up some strong arguments against this as a valid track as well.

My problem with the clarity issue is that, well, quite frankly, we haven't (as far as we know) come across a definitive track to date, so given the parameters of speculation involved, something being "kinda-like" should probably be marked so that further investigation by the Berkeley Team would make it known one way or the other (knowing, of course, that, by the end of this process, ALL VMs will be analyzed by them - we're essentially a prioritization mechanism and not the final word!).

However, it is indeed part of the charm of these forums that each of us takes our speculations and makes them public - not for chest-pounding reasons (well, ok....maybe just a little

) but for enhancing our understanding of just what it is we're looking for.

Thanks to all, and best of luck in "Dust-2: The Sequel - Tracking Up Close and (somewhat more) Personal"...opening in August at a browser near you!!

Nikita · Post by **Nikita** » Mon Jul 30, 2007 5:37 pm

cthiker wrote:However, Nikita brought up some strong arguments against this as a valid track as well.

Let it be known, I am going strickly by the guidelines given by the team, so don't let me spoil your party, it's probably the unexpected that will be the track!

Titanium · Post by **Titanium** » Mon Jul 30, 2007 6:57 pm

I am quite new to this, but I would certainly agree that the movie fjgiie showed is indeed a track. I don't know if there is any proof behind it, but it seems that if it has a "satellite" (as ERSTRS called it).

Since I am new to this whole thing, I am looking forward to Phase II. My main concern about the CM and training, but I have no idea really what to expect. I think the idea of subtle CM is good however. I have personally clicked on way too many inclusions (I realize now), but still have a very high percentage. I think that the new movie will allow us to refine our skills.

As for it causing us to click too much on RMs, I think it's better to click than not to. Better to get a low score but possibly identify something than to pass it up because you are worried about your score.

I'll go back to lurking now.

marcia · Post by **marcia** » Mon Jul 30, 2007 9:56 pm

Hi Titanium,

I'm not worried about my scores, I only started this topic, because I'm worried that the same problem -- way too many possible tracks clicked by us, like all this inclusions we had to learn about were not tracks -- could rise in phase 2. I really hope I will be proved wrong and that in the new phase I can refine my skills.
I think we just have to wait patiently, and see what happens in phase 2. I enjoyed to contribute to phase 1 and I'm looking forward to phase 2.

Nikita · Post by **Nikita** » Tue Jul 31, 2007 6:44 am

Well, what is the worst we could do by clicking on too much stuff? We cause an undergrad to get extra credit to sift through the low percentage movies to check on them after we have a confirmed track? Seriously, I don't see anything that anyone has presented that is so off the mark of what we are looking for to be absurd. Phase two might be better, but even if not, the team will have a much easier job sorting through slides looking at exact locations only to find inclusions vs. looking at all the movies. We do want to do the best job we can, but not sweat the accidental clicks. It is a pleasure to work with people who pay so much attention to detail!

Groundling · Post by **Groundling** » Tue Jul 31, 2007 1:10 pm

Hi marcia;
I wonder at the team's reasons for the proposed change in CMs. According to the original announcements, they had real interstellar tracks from the Muir station as well as the test tracks that were created in the lab. Now they say they will use "tracks" from the movies we are testing.
I have seen a couple of movies that only needed another 5 bars down to confirm but there have been no rescans as far as I know. Even then, a particle should be extracted before we use that track as a rule.
We seem to be going from a certain example to an absolute guess.
Groundling

marcia · Post by **marcia** » Tue Jul 31, 2007 9:57 pm

I think I still don't understand how a threshold can be better determined by using candidate tracks as CM's. (Like you, Groundling, say: We seem to be going from a certain example to an absolute guess. )
Or may be it's a language question, English is not my native tongue.
I'll wait and see.

the moon · Post by **the moon** » Fri Aug 03, 2007 7:08 pm

I also think that making CMs out of candidate tracks that haven't been extracted or even rescaned seems unscientific at best, and a big waste of time at worst.

But it's their project. It's up to them to say what type of features they want us to identify, even if the examples aren't real tracks, as long as they look like real tracks.

Titanium · Post by **Titanium** » Sun Aug 05, 2007 7:00 pm

Well, what is the worst we could do by clicking on too much stuff? We cause an undergrad to get extra credit to sift through the low percentage movies to check on them after we have a confirmed track?

If I were in school still, that would probably have been me... I always got mundane tasks.

I also think that making CMs out of candidate tracks that haven't been extracted or even rescaned seems unscientific at best, and a big waste of time at worst.

As for the new CMs made from Candidate Tracks (CT from here in because I am lazy), I think it's a rather good idea if you look at the big picture. Even though they are not 100% sure, they must feel confident that the CTs used are most likely real tracks. Even if a "fake" track is included in the CMs, it's still works in favor of the project, statistically speaking. Perhaps it is not as accurate, but it still trains us to look for things similar to it and they can narrow it down later and figure out if it's real or not (and that's the "real" science part). Like Nikita said, there is nothing wrong with clicking things you think might be a track (even if it's not).

I agree that it would be frustrating to us dusters to be clicking on these new CMs and then find out later that it is not really a track. Score adjustments and mayhem would follow.

If one of the new CMs is found not to be a track, we can all learn from it. It would also give us reasons to flood the forum with posts.

DiamondGirl · Post by **DiamondGirl** » Tue Aug 14, 2007 10:35 am

marcia wrote:May be this message is all speculation, but I've been thinking about this subject and want to share it. And if there are REALLY more subtle actual candidate tracks, forget everything I said here.

Hi, Marcia.

I don't know if anyone else has encountered this the same as I have, but I have noticed a definite difference in the rescanned slides. It's not that I am finding more candidate tracks, it's just that being able to more clearly see what happens below surface is giving me more clarity on what I'm seeing. At least in my opinion. For instance, before, a dot that may have appeared near the bottom level might have been very tough to tell if it fuzzed out or stayed clear, which is one of the primary indicators I use to tell if somethng is potentially a "zap pit" (fuzzes out or more than 1) or a possible track (gets clearer or moves in a different direction than others fuzzing out).

Also, using our prior guesses as calibration movies, I think, is just an attempt to refine what we've already been starting to learn on our own. When we first started on Phase 1, some of us might click on all the zap pits (even if there are 3 or 4 coming up on the same slide). However, over time, we learned to recognize these. True, some of them are still under consideration, but I think the stardust team recognizes that while they don't want to rule them out until they check, they are most likely not anything we should be clicking on. So, to help those who may not have had the time or level of experience as others, they use these to help train.

It's not perfect yet, I'm sure, but things in life rarely are. I do have to say that I think I've ruled out several that I might have been on the fence about previously, and identified one that I probably wouldn't have seen otherwise.

Not an answer, but my opinion.

Hope it helps.

marcia · Post by **marcia** » Tue Aug 14, 2007 12:27 pm

Hi DiamondGirl,

Now that I have been dusting a (part of a) few days, I must say that the 50-micron movies gives me more hope. We can indeed see much more, may be make better decisions, and I indeed see structures now I have not seen before.
But it still stands for me that I wonder if people who start now for the first time, will learn from de CM's. May be they learn to look for everything that's different, be more alert...
(BTW, I just saw Stardust in a tv-programma on the BBC. It was said that there was a track found. But it was such a sort fragment. I think it was a High Angle Track.)

And Diamond Girl, what a beautiful poem! My compliments, I enjoyed it very much.

Marcella

Stardust@home