View Issue Details

IDProjectCategoryView StatusLast Update
0000193Siril[All Projects] Sirilpublic2017-12-07 21:15
Reporterhn_88Assigned Tovinvin 
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformLinuxOSMintOS Version18.1
Product Version0.9.7 
Target Version0.9.8Fixed in Version0.9.8 
Summary0000193: stackall seems to return double expected value
DescriptionSum Stacking is supposed to add the pixels and divide by the number of frames added, right?

I have done stackall for a series of SER files, having approx 100 frames each. Importing the SER files into Octave by exporting them as FITS frames, I find the max intensity to be approx 44000 (16 bit images). But stackall creates images which are saturated - max value 65535. Checking non-saturated pixels, I found the stacked value to be approx twice the average non-stacked value.
Steps To ReproduceUsing stackall for SER frames. To test, exporting the same frames to FITS.

Opening the frames in Octave, checking intensity numbers. Opening the stacked FITS file, checking intensity numbers.
TagsSER stacking stackall

Activities

vinvin

2017-11-01 21:40

administrator   ~0000391

Hello, no sum stacking does not work like that in siril, the average stacking does.
Sum stacking adds all pixel values from all images. If the maximum pixel value across all channels in the whole image is above 65535, the image is normalized with 65535/max, otherwise the sum is kept.
The advantage of this is that if your input images are 8 or 12-bit deep, the stack will have dynamic for 16 bits, the precision is better.
Does that answer your question?

hn_88

2017-11-02 09:46

reporter   ~0000393

Hi vinvin,

Thanks for the clarification.

Just fyi, then it looks like stackall does not check for integer overflow during the stacking.

Example: SER file frame pixel values below,
Frame 1: 14112 14864 15552 14368 etc
Frame 2: 14528 13616 14864 15856 etc
Frame 3: 13392 13760 15168 15440 etc
etc for 102 frames.

But stacked file has pixel values
0 26838 27009 26992 etc - so, probably 16bits overflowing multiple times as the 102 frames are stacked.

And all stacked files for my set of SER files seem to start with 0.

vinvin

2017-11-03 10:26

administrator   ~0000394

It's quite strange this first zero indeed. I'll look into it.
There should be no overflow since the sum is not done on 16-bit memory space but 64-bit. It is normalized to 16 at the end with the maximum value of the image.

vinvin

2017-11-19 03:46

administrator   ~0000408

Last edited: 2017-11-19 03:48

View 2 revisions

Hello, could you confirm you have the same problem with the starting 0 value when loading a sequence and using the sum stacking instead of using the stackall command?
Also, with what tools did you check the values of the SER and FITS files? Data in the FITS is stored bottom line first, maybe that's why you don't get what you expect by comparing with the SER? (oh sorry I didn't see the note about Octave, I didn't know it would be able to do that, but maybe it does not respect the software standard for SER, it's not the same as in the pdf unfortunately, see https://free-astro.org/index.php/SER#Specification_issue_with_endianness)
Can you also indicate if you have used registration on the sequence that has problems? I would think that you didn't open them individually to register them if you use stackall, but I'd like to have a confirmation.
Thanks

hn_88

2017-11-19 07:40

reporter   ~0000409

Hi, some additional info.

0. I had done my earlier tests with the 0.9.7 AppImage downloaded from your website. When I tried with the current 0.9 branch on github, downloaded on 5 Nov,
found that stackall causes a crash in that version. I suppose you are aware of this.

1. If I individually load an SER file and do sum stacking with the 0.9.7 AppImage, then also the first element is 0. I have attached this single stacked file, 15_05_42_stacked_onlyone.fit

2. I was opening the FITS files in Octave using the read_fits_image function, using load pkg fits.
For opening SER files, I opened them in siril first, exported the sequence as FITS, and then opened in Octave.

3. No, I did not use registration. (This is not actually astro photo data).

15_05_42_stacked_onlyone.fit (158,400 bytes)

vinvin

2017-11-19 22:32

administrator   ~0000411

0. I have parallelized the sum stacking in the past weeks, and I broke something indeed. I am fixing it.

I will investigate the data values issue, this is very disturbing.
And thanks for the report!

vinvin

2017-11-20 00:15

administrator   ~0000412

For the record, in the image you provided, the 0 is on the bottom left corner and it seems to be the only 0 in the image. I was able to reproduce the issue with another dataset.

vinvin

2017-11-20 00:25

administrator   ~0000413

Fix committed to Siril (1868).

Related Changesets

Siril: 0.9 r1867

2017-11-20 00:06:08

vinvin

Details
fixing the crash in stackall (issue 0000193) due to the new stacking function that assumed that the sequence was well-known Affected Issues
0000193
mod - /branches/0.9/src/core/command.c
mod - /branches/0.9/src/core/siril.h
mod - /branches/0.9/src/io/seqfile.c
mod - /branches/0.9/src/io/sequence.c
mod - /branches/0.9/src/io/sequence.h
mod - /branches/0.9/src/stacking/sum.c

Siril: 0.9 r1868

2017-11-20 00:25:06

vinvin

Details
fixes 0000193: the issue was quite simple, only one pixel was not processed because of a programming error, stacking is probably fine otherwise. Affected Issues
0000193
mod - /branches/0.9/src/stacking/sum.c

Siril: 0.9 r1906

2017-12-07 21:15:32

vinvin

Details
Fixing the fix for stackall (commit 1867 - bug 0000193) that was overwriting the imgparam and regparam on sequence load.
Also fixing the way selection is used in some registration methods.
Affected Issues
0000193
mod - /branches/0.9/src/io/sequence.c
mod - /branches/0.9/src/registration/registration.c

Issue History

Date Modified Username Field Change
2017-11-01 05:11 hn_88 New Issue
2017-11-01 05:11 hn_88 Tag Attached: SER stacking stackall
2017-11-01 21:40 vinvin Assigned To => vinvin
2017-11-01 21:40 vinvin Status new => feedback
2017-11-01 21:40 vinvin Note Added: 0000391
2017-11-02 09:46 hn_88 Note Added: 0000393
2017-11-02 09:46 hn_88 Status feedback => assigned
2017-11-03 10:26 vinvin Note Added: 0000394
2017-11-03 10:27 vinvin Target Version => 0.9.8
2017-11-19 03:46 vinvin Note Added: 0000408
2017-11-19 03:48 vinvin Note Edited: 0000408 View Revisions
2017-11-19 07:40 hn_88 File Added: 15_05_42_stacked_onlyone.fit
2017-11-19 07:40 hn_88 Note Added: 0000409
2017-11-19 22:32 vinvin Note Added: 0000411
2017-11-19 22:32 vinvin Priority normal => high
2017-11-19 22:32 vinvin Severity minor => major
2017-11-20 00:06 vinvin Changeset attached => Siril 0.9 r1867
2017-11-20 00:15 vinvin Note Added: 0000412
2017-11-20 00:25 vinvin Changeset attached => Siril 0.9 r1868
2017-11-20 00:25 vinvin Note Added: 0000413
2017-11-20 00:25 vinvin Status assigned => resolved
2017-11-20 00:25 vinvin Resolution open => fixed
2017-11-20 00:25 vinvin Fixed in Version => 0.9.8
2017-12-07 21:15 vinvin Changeset attached => Siril 0.9 r1906