Activity › Forums › Astrosoftware › Astro Pixel Processor › APP stacking process slowing down with 157 Lights › Reply To: APP stacking process slowing down with 157 Lights
Thank you @keesscherer,
“I am running the 8GB version on the 16GB machine. With sigma clipping. The time consuming step is 6, all the others were done in 3 hours.”
Okay, so the major bottleneck is the stacker currently… looking at the name of your stack, I think I’ll investigate the Linear Fit outlier rejection filter in APP. Possibly this becomes really slow in APP due to the regression that is done in each pixel stack to calculate linear fit parameters. The more frames you use the slower it becomes probably. So I’ll run some tests with that filter and compare it to the sigma and winsor filters. If this is the culprit, I’ll need to think of a smarter way to use this filter.
Having said this, I also notice you are using 3 iterations with kappa 2 (the default values currently). For 150 lights this would normally be way too aggresive for outlier rejection. I will set the default values to much more conservative settings in the next version ( 1 iteration kappa 3), since with stacks of 50 frames, usually in APP, this is all you need to reject all outliers. Probably, depending on implementation, the setttings that you need to use in different programs can be quite different due to different implementations of the calculation of statistical properties of the pixel stacks.
And this is probably good news for your Monkey Head stack, I am quite certain that with less clipping the stack will be much better when judged on noise and SNR.
In the FITS header of the stack (to be seen with details selectbox above image viewer), you’ll find stack statistics. the median and reference noise reduction are only 6 for a stack of 150 lights. If you have dithered your data, this is really low. Ideally, the reduction is the square of the number of lights. So for 100 frames, the best possible noise reduction is 10. So for 150 lights, the noise reduction could possible be doubled to 12 ;-). This is a clear sign to me that possibly the outlier rejection is way too aggresive.
”
I still don’t understand how i should handle the 200 iso flats, i loaded the lights 1600 iso, bias 1600 iso and 200 iso bias frames i selected sigma clippng and clicked stak on tab 6. But the flats still do not work ( i saw this :
” So your masterbias of iso200 for your flats and your masterbias of iso1600 for your lights will automatically be applied to the correct frames (that have the same ISO), since APP has the strict rule that master bias frames must match the iso of the frame that needs calibration.”)
so i must assume APP needs some other manual setting to work with the 2 sets of flats?”
I guess 2 sets of flats, would be 2 sets of bias?
I’ll make a calibration video with sound to show what should happen. There is no manual setting. APP should apply a master bias automatically to the frame needs calibration with the same ISO.
“All my systems work with a LAN only Teamviewer and i don’t like to change that setting.”
No problem, I’ll investigate the Linear Fit outlier rejection filter. I strongly suspect this is the problem.
Adding to this, I actually never use the Linear Fit filter. I consider it deprecated due to the implementation of LNC. A linear fit filter is a solution for conventional normalizaion of data where the gradients in the stack change clearly between the frames. Linear Fit can account for this and help with outlier rejction. But LNC will solve this automatically by correcting all subs in your stack for gradients before data integration. The concept of a Linear Fit filter then becomes deprecated and unadvisable. I’ll make work of this to explain this more thoroughly in the manual, since LNC is a new concept.
So my advise would be, try to stack your 150 lights with normal sigma clip 1 iteration, kappa 3 and LNC on 1 iteration first degree. I wouldn’t be surprised if the stack comes out much better ?
(I can also put it differenly: a linear fit filter is a solution for bad data normalisation, LNC solves this at it’s root by strongly improving data normalisation in your stack in the entire field of view and therefore a linear fit filter becomes useless and a normal sigma/winsor filter will suddenly work much better due to much better data normalisation.)

