Astro Pixel Processor

Java errors when running 4Gb, CR2 stacking still slow.

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #15149

    KeesScherer
    Participant
    posts: 1148

    Stacking FITS files is really fast using just the 4Gb i have allocated, but Canon Raw is still slow. I am now stacking 78 lights, first on the 8Gb machine (v1.037) with 4Gb allocated but that was no success with all the Java “not enough memory”errors. Over to the 16 Gb laptop with 12Gb allocated and the “Creating lights stack” is now at 50% after 2 hours.  EDIT: the 16Gb Laptop was running v1.035, after upgrade to v1.037 stacking progress is at 50% after 1hr45min…

    • 1 person likes this.
    #15151

    Haverkamp
    Participant
    posts: 640

    Thank you @keesscherer for this bug and usefull screenshot.

    The screenshot shows that you have very high star counts (3000-4000). So I tried to register some frames with that high star counts and only 2GB of memory reserved to APP. I do get the same memory problem, so clearly there is a memory leak in the registration engine.The good news is, that I have located it now, so I’ll fix it.

    I am wondering about the high star count, was registration not good with a maximum of 2500 stars, so you decided to increased the limit? (For regular stacks, 1000 stars should normally be enough to get very good registration ) Or just testing?

    Regarding speed, I know I can do more optimizations in the integration module regarding outlier rejection of light frames.. I’ll work on that soon, because that’s a clear bottleneck currently…

    #15168

    Haverkamp
    Participant
    posts: 640

    @keesscherer,

    Thank you for reporting this. I have done a lot of work in the registration engine to prevent this from happening in the future.

    Basically, the amount of starpairs between the frames put limits on memory and cpu usage at the same time when the total amount of star pairs become bigger then a couple of 1000 star pairs.

    Reason being, to calculate a homography between 1000 stars, APP needs

    (1000*2 (x,y coordinate) * 8 (double precision calculation) )^2 bytes

    for the Hessian Matrix in the regression.

    For 1000 stars this means = 256MB of memory. This is no problem

    2500 stars between two frames => 1.6GB now it starts to become a problem.

    To solve this problem for different maximum amounts of memory available to APP, I have built in dynamically scaling of the registration engine for both the amount of memory that is needed and the number of CPUs that are/can be used. The scaling is done such that the memory usage can’t top the available amount of memory to APP and thereby preventing future Out of Memory errors.

    Also the amount of stars that can be used in the registration is now capped depending on the available MAX memory for APP.

    From version 1.038, this “out of memory” bug in the registration engine should be a thing of the past I hope 😉

     

    • 1 person likes this.
Viewing 3 posts - 1 through 3 (of 3 total)

You need to log in or to reply to this topic.