Skill score decreasing for found movies

[jsmaje]

Hi Evelyn & Mike.

In fact I'm an idiot for forgetting my own caveat that the sim assumes an 'average' duster who gets as many PMs wrong as right. Since both your skills are markedly better than that, you will naturally be encountering significantly fewer unexpected rises than falls, and it's for me to apologise for having been missed this obvious point.
I don't even know to what extent differences in skill level would affect the sim's quantitative results and my conclusions regarding the benefit of a weighted filter. It should be easy to incorporate this feature, however, and will be doing so. Watch this space!

John

[SmithES]

Hey, Caprarom, How to delete a post? Once you hit the "submit" button and your post comes up, there is an "edit" provision (in red) on top right. Click on it to correct your post and then re-submit it. If you want to delete it all together, just erase the text, and sign out. It will totally disappear. How do I know all of this? Don't tell a soul, but I have had to hit that "red edit" nearly every time I post! Evelyn (ERSTRS)

[jasonjason]

I am still getting no points for found movies and large drops for missed movies current stats are
Statistics for jasonjason

Power Score: 8400
Total Movies Viewed: 3177
Your Current Skill: 0.8822 ( since December 20, 2013 )
Power Movies Missed: 70

It is still frustrating especially getting no points for a found movie and even more so when my points go down!!

Jason T Fusaro

[jasonjason]

I have been keeping track of my scoring to see if someone can make anything out of it, my JTF account

PDF http://www.mediafire.com/download/i883s ... racker.pdf

Libreoffice http://www.mediafire.com/view/uj7uzngxg ... racker.ods

HTML http://www.mediafire.com/view/gb7381b68 ... cker.html.

Code: Select all

Movie ID	        Power	Power	Running	Movies	Power	Power	Skill	        Skill Score	  Percent	      Power Movie
	                score 	increase	Total 	Viewed	Found	Missed	score 	Inc / Dec	  Run Total	Value
9028044V1	5820		0	                         2206	273            45    	0.7611	0.0000	
117835V1	        5820	       0	         0	         2210	273	         46	        0.7455	-0.0156	-0.0156	          55
2618031V1	5840	       20	         20	         2231	274	         46    	0.7467	 0.0012	-0.0144	
7866385V1	5865	      25	         45	         2233	275     	46     	0.7427	-0.0040	-0.0184	
366732V1	        5875	      10	        55	         2235         276  	46     	0.7427	0.0000	-0.0184	
7867065V1	5895       20       	75	         2236	277   	46	        0.7444	0.0017	-0.0167	
9518868V1	5915	      20	        95	         2237	278   	46	        0.7438	-0.0006	-0.0173	
4899092V1	5920	       5	       100	         2240	279   	46	        0.7415	-0.0023	-0.0196	
5483962V1	5965 	45	       145	         2248	280   	46	        0.7409	-0.0006	-0.0202	
5358023V1	5980 	15	       160	         2250	281    	46	        0.7421	0.0012	-0.0190	
5966790V1	5995 	15	       175	         2255	282   	46	        0.7421	0.0000	-0.0190	
9123881V1	6055 	60	       235	         2264	283   	46	        0.7472	0.0051	-0.0139	
7588576V1	6070         15	       250	         2265	284   	46	        0.7483	0.0011	-0.0128	
7530412V1	6085 	15	       265	        2270 	285           46	        0.7489	0.0006	-0.0122	
5060403V1	6090         5	       270	        2275 	286   	46	        0.7489	0.0000	-0.0122	
4255633V1	6100	        10	       280	        2276 	287   	46	        0.7495	0.0006	-0.0116	
3583347V1	6120         20	       300	        2288 	288   	46	        0.7505	0.0010	-0.0106	
5776397V1	6130	        10	       310	        2289 	289   	46	        0.7500	-0.0005	-0.0111	
8495983V1	6140 	10	       320	        2309 	290   	46	        0.7505	0.0005	-0.0106	
7968937V1	6150 	10	       330	        2317 	291    	46	        0.7505	0.0000	-0.0106	
9474565V1	6155 	5	       335	        2324 	292   	46	        0.7500	-0.0005	-0.0111	
8495303V1	6180 	25	       360	        2329 	293   	46	        0.7478	-0.0022	-0.0133	

Can anyone make sense of why so many reductions and losing points for found movies, thats like 21 finds in a row for a .0018 increase which is about .0001 per correct find.

[caprarom]

Jason, if it helps, those look like normal fluctuations to me. As explained elsewhere, they stem from the skill score being calculated from a "moving window" of the most recent 100 PMs you've encountered. So, to get a true picture of what's happening, you would need to look at what happened 100 PMs previously. If you successfully flag a 20-point PM, but had flagged a higher-value PM, say a 50-pointer, 100 PMs back, your skill score will drop a bit. This is perfectly normal and expected when using such a moving window approach to the skill score. Hope that helps.

[SmithES]

Jason, I am amazed at your chart! I had attempted to produce one of my own track record, but I cannot equal the detailed work you have done. Your chart is brilliant, but it has discouraged me to the point of quitting. When a duster can have a long string of successes at finding tracks, and still drop so severely in Skill Score, then to me---Skill Score in no way---represents the skill of the duster! And for Caprarom to comment that your chart looks like "normal fluctuations" to him, was th"e last straw for me. I can no longer put everything I have into dusting, only to see my skill score keep dropping even though I find tracks; and have that considered "normal."
I was 77 years old when I began dusting in August 2006, and have enjoyed it enormously in all of the other phases. But Phase 6 has done me in. It has brought to my attention something very important, and disturbing ---I have lost the "vision" I so joyously began with 8 years ago---that is, finding interstellar stardust in samples! I am appalled, that In Phase 6, I have become obsessed with finding those tiny tracks that have been inserted into samples by the Stardust lab techs, and have forgotten all about our original mission. Not once in any of my Phase 6 posts, have I even referred to interstellar stardust particles! I am ashamed to point that out.
So many of you have been extraordinarily kind, patient and considerate to this old lady, and I'll miss you. My goodness, I was even DOM one month! I'll remember that---and the patch I just received from Tom Yahnke---and I'll remember all of you---as I ride off into the sunset. My love to you all. Evelyn (ERSTRS. SmithES)

[jsmaje]

Elaine - please don't despair; we need you!

The current skill assessment methods are of course a compromise between addressing previously-raised issues, but I'm sure open to improvement.
I hope my upcoming adaptations to the 'Skill score simulation' including initial skill level will reassure you and confirm that use of a lead-in weighted filter to the moving average window will make a big improvement. If I'm wrong, I'll be happy to admit defeat.

John

[SmithES]

Jsmaje (John), "Well, there you go again . . . calling me "Elaine" again! My name is Evelyn -- E-V-E-L-Y-N. I will await your "big improvement" with your ingenious filter. Then, if I my score hasn't disappeared into oblivion, I may give it a try. I never had a problem with any of the Phases I thru V, but Phase VI has made me lose sleep. Believe me this is a serious matter----at my age---I need all the beauty sleep I can get. E-V-E-L-Y-N (ERSTRS, SmithES)

[jsmaje]

O my! So sorry Evelyn . I think there is an overactive neurone in my brain labelled 'Elaine' downstream from the one that responds to any name beginning with 'E', since we exchanged private messages for some time where she shared quite personal issues. With luck it will soon die off like many of the rest are doing.
John

[SmithES]

To jsmaje, That's all right J-A-M-E-S, non of us are perfect, and all of us have problems or some sort or another. Am awaiting your BIG improvement - honestly, I am grateful and pleased, and will give it a try. E-V-E-L-Y-N. (ERSTRS, SmithES)

[DanZ]

To All Our Amazing Dusters:

Believe it or not, we have been listening. I have communicated with Evelyn offline about all this too. Now the question is - what to do?

Part of why we ended up where we did is because we got more unexpected than expected results from Stardust@home. One of the more important unexpected results were the unusual and unpredicted looking tracks that became the tracks of most interest scientifically. So then, how do we "train" our dusters to look for such anomalies and become good at finding them? The Skill Score seemed like a good idea at the time, but depending on how one wanted to judge "Skill," it could come, and indeed did, in many forms. Obviously then, it never could meet everyone's needs. And perhaps that's a sign that it should be eliminated or perhaps, better yet, optional and only visible and employed by those dusters who find it valuable as a means to increase their ability to find tracks. I in fact convened an emergency meeting of the Team yesterday and made the suggestion of Skill being optional, and having us (and you) focus more on how to best find interstellar particles (as Evelyn so rightly points out is the gist of this project - thank you Evelyn!). The Team agreed and we are now looking at overhauling our homepage and launching a revamped, more duster-friendly Phase 7. So what will replace the Skill Score? We're looking at ways to do more visual training via blogs and making current particle candidates more prominent on the homepage (instead of buried where most people never look at them). In essence, we'd use multiple images of exactly what we are looking for and say "please find this - it looks like this" and so on. We'd also invite dusters who clicked on these more valued tracks to talk about their experience and share any tips they may have to offer. Those finding high-valued tracks would also be more regularly rewarded with recognition of some kind (still to be worked out).

Needless to say, this will not fix the frustrations some of you are currently having in Phase VI, and for that we are sorry. Note John that Dr. Westphal will try to get to your weighted filter suggestion as soon as he can. But he's currently going through the peer review process at a major journal, waiting patiently for a decision on our top-level ISPE paper. Apparently there was some sort of mix-up in the first review, and now the process has basically started over. Very frustrating and time-consuming to say the least, but at least now it's been straighten out. However, my guess is if the weighted filter gets employed, it will probably be in whatever form Skill Score takes in Phase 7.

Again, thanks for your input and patience on this. You've always been essential to the project, and completely fair (and smart!) in your criticisms - thus we must act! I will let you know as soon as I can when these attempts at improvements will commence. As Evelyn says, this project is first and foremost about "finding interstellar stardust" and we need to make sure that's what we're focused on. Hopefully Phase 7 will make that more obvious.

More to come, I promise!

Dan

[jsmaje]

Dan - good to know something is happening on the skill score front; it seems to have caused nothing but vexation so far!

Meanwhile, the 'jitter' simulation has now been adapted to also allow choice of initial skill level between 0 & 1.0 in 'custom' mode, rather than the previous set 0.5 level (though this remains for demo mode), and is available at the same web address here.

However, given the new multiplication of possible combinations of all the variables, it will take a while to analyse the results. I nevertheless hope to post some basic findings in the next week or so.
Perhaps you could let Dr. Westphal know before he makes his assessment.

John

[Edit (28 Feb): Actually the sim seems to be misbehaving, and has been temporarily withdrawn. Hope to get it back soon.]

[SmithES]

jsmaje, I received a private message from you, but was not allowed to sign in to Stardust@home---"You have exceeded the number of attempts to log in," message. I tried several times, and even though I "solved" the dumb puzzle, it still would not let me log into Stardust Home page. So, I signed into the Forum (different name and PW here), and here I am. I've had this problem before, and I can't face it again. Therefore, If you or the other fellows want to contact me, would you just use my AOL e-mail account instead of sending me private messages? I DO want to hear from you! I promise I won't send "junk" back to your e-mail boxes. Contact me at ERSTRS@aol.com, please. I can't keep trying to jump through all the hoops needed to by pass the "you've exceeded the number of attempts. . . ." Thanks, Evelyn (ERSTRS, SmithES)

[SmithES]

P.S. to last message on difficulty accessing private messages: Below--- is what comes up when I try to read your private messages. Even "solving" the CAPTCHA puzzle doesn't help. I can't access private messages to me. Please reach me at: ERSTRS@aol.com. Thanks, Evelyn

Login: You exceeded the maximum allowed number of login attempts. In addition to your username and password you now also have to solve the CAPTCHA below.
Username: ERSTRS
Password:***********

[jsmaje]

OK, the skill 'jitter' simulation is now back up & running.
My results covering the whole skill range are here, confirming that use of a weighted filter at the beginning of the moving average window would considerably reduce the impact of this inevitable but annoying phenomenon.

John