What is happening to this project?

Discuss your experiences with and ideas about Stardust@home here.

Moderators: Stardust@home Team, DustMods

Post Reply
harmony
Posts: 15
Joined: Fri Aug 18, 2006 5:18 pm
Location: England

What is happening to this project?

Post by harmony »

Currently only 42 participents are active in the top 100. I have not been active for some time because there are more than enough views of the movies I have highlighted to to extract the needed statistics, and I assume that is the same for all movies that are out there. I am very willing to continue the effort for science if this project is going to go ahead with phase three. It seems to me that the funding is ended and therefore we should be told as the overall time scale is now several years.
Regards
John
To be, that is the answer
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: What is happening to this project?

Post by jsmaje »

John (from another John), I share your concerns.

Having had no response to the following post regarding the latest video update under "Update for 30 Sept", I'm grateful that you have independently raised this issue, so have moved it here in the hope that someone on the team (DanZ or whoever) might eventually respond:

"Personally, using IE7 with the high-quality Flash version, I've had no problem viewing the whole update, except, as fjgiie says, for variable gaps of at least 3 to 10 seconds before the picture updates to what Andrew is then talking about. I don't know whether this is a programming bug or due to sloppiness on the part of whoever compiled it, but it does make matching the narrative to the graphics somewhat confusing.

Meanwhile, as I think I've said before, I don't really see the point of such video presentations on this site anyway, particularly if they don't work properly for different users, as appears to be the case. I can only presume they are being made to also serve extraneous educational/promotional purposes, &/or perhaps for the blind or deaf, even though each of these would lose out one way or the other!

Nevertheless, thanks Andrew for explaining how the project is progressing so far. It doesn't really come as a surprise that your initial predictions were over-optomistic, and those who have stayed with you through phase 2 have been very patient in this regard, and a few extraordinarily hard-working.

Can you give us any (realistic) indication at all as to when re-imaging of the isolated tiles may begin? Would we then need to restart from the beginning for every tile with whatever new photographic protocol is established? And would that be phase 3?

As promised when joining, knowing what practical science is like, I'm happy to be in this for the long haul, but now have five concerns:
1: how long we may need to wait before whatever phase 3 might be actually starts.
2: how long the entire project is now projected to take.
3: whether the funding (and the team's own enthusiasm) for persuing the project are sufficently secure in the long term.
4: whether the burden of hosting Stardust@Home, with all its demands (such as responding to this message) may be considered more than its worth, given the apparent reliance on the 'Red Team'.
5: whether by now there might be any imminent or planned satellite launches, from whichever nation, with more sophisticated collectors than Stardust that may be analysed more quickly, so overtaking and making this project redundant.

I don't expect definite answers to 1 & 2, but would like honest answers to 3 & 4, and an informed response to 5.
John"
DanZ
Site Admin
Posts: 777
Joined: Fri Feb 27, 2009 2:44 pm
Location: Berkeley, CA
Contact:

Re: What is happening to this project?

Post by DanZ »

Hi John (and John),

Yes, things have been quiet. This is mainly due to the climate controls being compromised in the clean room, where the tiles are being analyzed and stored (as discussed toward the end of Update #5). This incident necessitated new strategies/actions (including tile removal, new tile and foil holders, witness tracks, etc.), all of which have taken considerable time***.

(***We are vividly aware at all times that this collection is unique, and there is no prospect in the immediate future for collecting another sample like it.)

To answer your questions:

1: how long we may need to wait before whatever phase 3 might be actually starts.

The new extraction system worked in tests, and we have scanned overnight on Wednesday tile #92. So within a month or so, there will be at least 4,500 NEW movies uploaded for dusters to examine (with many more to come after that).

2: how long the entire project is now projected to take.

Years. Bear in mind that we are still very actively studying Apollo lunar samples, 40 years after they were returned, and that these analyses are still bearing valuable scientific fruit. Of course, the IS dust collection is different from Apollo samples, but even after we identify the first interstellar dust grains, there will be continuing work to find more over many years. For example, some areas of the aerogel are currently unscannable because they are too fractured. But years from now there may be an effort to find the impacts in those areas.

3: whether the funding (and the team's own enthusiasm) for persuing the project are sufficently secure in the long term.

Yes! The project is adequately funded for the next 3 years and the team remains quite enthusiastic. We are in this for the long haul.

4: whether the burden of hosting Stardust@Home, with all its demands (such as responding to this message) may be considered more than its worth, given the apparent reliance on the 'Red Team'.

No. Stardust@home and ALL dusters have been and will remain integral to the project’s success.

5: whether by now there might be any imminent or planned satellite launches, from whichever nation, with more sophisticated collectors than Stardust that may be analysed more quickly, so overtaking and making this project redundant.

There are no significant discussions in the community about an interstellar dust sample return mission in the foreseeable future – the Stardust team (including you!) remain on the cutting edge (with a new paper and findings to be published shortly - look for a blog update on that relatively soon). The community is waiting to see what comes from Stardust.

As always, we’d like to thank all the dusters for their incredible patience. We’ll be sure to keep you posted as best we can as we move forward.

Cheers,

Dan (on behalf of the Stardust team)
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: What is happening to this project?

Post by jsmaje »

Thanks Dan, that was very informative.

Re my Q1, you mention tile #92. Would that be the tile labelled #I092 in the top-right corner of the last update we've been provided of tile examinations so far - by Anna Butterworth way back on Thurs May 17, 2007?!
It hadn't been examined at that time, and I don't know if it has since then.

My next question is therefore whether you are planning to apply the new technique only to so-far-unexamined tiles, or will be doing so also with those already examined during phases 1 & 2?
Will the new movies be using the same planar and depth resolution as in phase 2?
Will the same, different (or any sorts of) calibration movies be included?
The bottom line: by the time you hopefully start phase 3 (if such it may be called), will we effectively be starting all over again? No problem if so, but it would be good to know and and be prepared for whenever that may be!

Re my Q2, I was referring to Stardust@Home dusters and their part in this initial search, before the completion of extractions and all the subsequent scientific analyses that will of course need to take place.
Once produced, how many new movies are there now projected to be?
At the present resolution I understood that there were about 15,000 potential movies per tile. You mention the figure of producing 4,500 within a month, with 'many more' to come. How fast do you expect to process each complete tile, and how fast might you progress between one and the next?

So again, how long do you estimate the Stardust&Home search project is likely to take?
(given experience so far I won't believe you anyway!)

Sorry to be a pain :wink:
John
KevinACourtney
Posts: 21
Joined: Wed Jan 16, 2008 4:57 am
Location: St. Louis, MO

Re: What is happening to this project?

Post by KevinACourtney »

Hi John!
Hope you've been well! I saw your posts and have to say your points were
very interesting. I kinda figured that since the end of Phase 2 is about a year late
that would give Phase 3 a soft start date. But hey...we'll get some new material to
search soon anyway!
I've been checking your Top 100 site...and now see that you're going to just wait for
Phase 3 to start it up again.
Since it may be quite some time before we get to Phase 3, if you agree and think it's
possible, I'd like to volunteer my assistance somehow if you would be interested in restoring
the Top 100 for Phase 2. I don't know what's all involved, but if you care to, I could perhaps do the weekly
maintenance and postings for you.
Just let me know if you're interested...I always looked forward each week to the updates!

Take Care & Best Regards,
Kevin
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: What is happening to this project?

Post by jsmaje »

Well Kevin, just for you I'll try to get the Stardust Top 100 site running again!

As well as the lull in activity, a large part of my decision to put it on hold was because Berkeley's source code was subtley changed a while back (deliberately to foil me or not!), and I couldn't immediately work out how to get around it. Meanwhile I got involved with another project regarding clock puzzles, which has gone well so far but is getting more complicated and beginning to do my head in too, so I'm happy to set it aside for the moment!

There will exist a substantial gap between my previous Stardust data and the present (I can't access the team's complete database), which will make the programming and meaningful graphic presentations less than straightforward. And of course I have a life outside Stardust, so can't promise anything on-line within, say, a couple of weeks.

Meanwhile, thanks for your enthusiasm for the program and the challenge you've given me, as well for the extraordinary contribution you continue to make to the Stardust search. I'll keep you informed.
John
nitram819
Posts: 90
Joined: Mon Mar 02, 2009 8:38 am
Location: Quebec, Canada

Re: What is happening to this project?

Post by nitram819 »

Thank you for the update Dan, the news that new movies will be updated in the next month or so definitely makes me want to get back on that virtual microscope right now to get a little practice.

Really looking forward for those new movies!

Martin
DanZ
Site Admin
Posts: 777
Joined: Fri Feb 27, 2009 2:44 pm
Location: Berkeley, CA
Contact:

Re: What is happening to this project?

Post by DanZ »

Thanks Martin, glad to hear you're gearing up :D

John - regarding your other questions, Anna Butterworth was kind enough to provide some additional information as follows:

1. Re my Q1, you mention tile #92. Would that be the tile labelled #I092 in the top-right corner of the last update we've been provided of tile examinations so far - by Anna Butterworth way back on Thurs May 17, 2007?! It hadn't been examined at that time, and I don't know if it has since then.

yes: "Tile 92" is correctly called I092 (not to be confused with C2092, which is "tile 92" on the Stardust cometary collector) and has not been scanned before. We are scanning I092 now because it has been extracted from the collector by our colleagues at Johnson Space Center (JSC). Eventually all of the tiles will be extracted from the collector and be scanned or rescanned.

JSC are choosing the order to extract the tiles, based on position and releasing the aluminum foils as they go. We will then scan the extracted tiles as they become available. Some will be new, some will be rescans.

BTW. The Scanning Progress graphic from May 17, 2007 is up to date! There were no new scans in 2008 as we concentrated on extracting candidates. At that time, scanning and candidate extraction were mutually exclusive. As of this month, that is no longer the case: we can scan one individual tile while extracting candidates from another tile, at the same time, on a brand new, separate extraction system.
The rate-limiting step is now the very careful removal of whole tiles from the collector and placement in their new permanent storage holders. The holders were designed during this summer's closure and and they are being produced at the moment. There's a picture of the new holder and the changes at JSC in Andrew's update 5.

2. My next question is therefore whether you are planning to apply the new technique only to so-far-unexamined tiles, or will be doing so also with those already examined during phases 1 & 2?

We will scan all of the tiles extracted from the collector, in the order they are extracted by JSC. Some will be new, some will be rescans.

3. Will the new movies be using the same planar and depth resolution as in phase 2?

The raw movies will be the same as before: same field of view, same depth. We have decided (for now) to process the new movies just as we did for Phase 1. The Phase 2 'high resolution' focus movies are a lot of work to produce (typically an extra week of processing per tile) and we'd like to move data out a lot faster in Phase 3. That could change, of course, and we can always go back and reprocess raw movies to make the higher resolution images if necessary.

4. Will the same, different (or any sorts of) calibration movies be included?

TBD. We are discussing this very topic as we gear up for the next phase. We are wondering whether to include the low-angle tracks in the calibrations, though in that case we would have to either rewrite the scoring code or redo the training. We'd have to figure out how to class as correct a click along a long thin track, and not simply a click within a small circle as the code runs now.

5. The bottom line: by the time you hopefully start phase 3 (if such it may be called), will we effectively be starting all over again? No problem if so, but it would be good to know and and be prepared for whenever that may be!

No, we're not starting again. We are taking big strides forward and we are hopeful the improvements and redesign at JSC this summer will allow us to scan and churn out new data at a much faster pace. As we will inevitably rescan some tiles at JSC, we can prioritize the all-new data at Berkeley for processing and uploading to stardust@home.

6. Once produced, how many new movies are there now projected to be?

We are about 1/3rd of the way through scanning the collector having approximately 180,000 regular focus movies. When we're all done, we will have an estimated 500,000 focus movies (Phase 1 style, not the high-res Phase 2). That's the same number as our original estimate.

7. At the present resolution I understood that there were about 15,000 potential movies per tile. You mention the figure of producing 4,500 within a month, with 'many more' to come. How fast do you expect to process each complete tile, and how fast might you progress between one and the next?

Each 40 mm by 20 mm tile is scanned with about 3,500 to 4,500 fields of view depending on how much surface is viewable around the edge (i.e. how steep the aerogel curves). Those raw movies are shipped to Berkeley on hard drives, and we produce the same number of Phase 1 type focus movies for the tile i.e. average 4000 per tile.

It takes about a day to process a tile using Quicktime 7 and Apple scripts. We take another day to do an automated quality check using MATLAB code to see that the scans were generally within focus; and then another day to upload for searching. Most of this time is computer cycles but does require manpower to start processes, check things are working correctly, and keep careful track of Tiles and data.

We are not planning to produce the 15,000 or so 'high-resolution' focus movies for the time being.

I would expect turnaround will be 1 to 2 weeks for each Tile, depending on shipping time from JSC and our other duties at the lab, including analyzing candidates! We are excited to get a look at the new scan data, very soon.

-----

I'd like to ask now that you give us some time to get these new movies out. We'll of course make an announcement and will probably have more information/explanations at that time as well. After that, we'll be happy to answer any lingering questions.

Cheers!

Dan
jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: What is happening to this project?

Post by jsmaje »

Fair enough Dan - I get the message: "Don't call us, we'll call you!"

Meanwhile, Kevin:
I've now got Stardust Top 100 working again quicker than expected, though some of the graphs are necessarily estimates, based on a linear interpolation of scores between 5 Aug & 14 Oct '09.

During this 10-week gap there has been just one new entrant to the Top 100: irena55 has replaced Paul Pallier, and seems to be quite keen, having already risen by 5 positions.

Overall though, as harmony/John pointed out above, fewer than half of Top 100 dusters are currently active (& I have to confess to not been among them).

John
endust
Posts: 53
Joined: Wed Aug 02, 2006 9:42 am
Location: Kansas City

Re: What is happening to this project?

Post by endust »

Once an avid duster, I haven't been to this site for months (years?). I was very pleased to see the update movies and even happier to see that things will begin rolling again and at a faster pace. Now that you are pulling the gel, I assume you will have the opportunity to light them better. I cant help but wonder about all those identified tracks that are still indeterminate from phase 1 and 2. It would seem that starting over in phase 3 would be the best approach. Keep the prior information for reference but take advantage of the better lighting to have a higher success rate on spotting tracks.

Look forward to the relaunch. Lets start dusting again!
DanZ
Site Admin
Posts: 777
Joined: Fri Feb 27, 2009 2:44 pm
Location: Berkeley, CA
Contact:

Re: What is happening to this project?

Post by DanZ »

That will indeed be the case. In addition, Phase 2 will end with some new movies of a few new tiles and old tiles. When Phase 3 begins (sometime in January), it will be all new movies and a lot more new tiles.

Stay tuned.

Dan
Post Reply