Skip to content

[ENH] OWFFT: Configure Complex FFT manually#809

Merged
stuart-cls merged 3 commits intoQuasars:masterfrom
ngergihun:owifgupgrade
May 26, 2025
Merged

[ENH] OWFFT: Configure Complex FFT manually#809
stuart-cls merged 3 commits intoQuasars:masterfrom
ngergihun:owifgupgrade

Conversation

@ngergihun
Copy link
Contributor

Hi Everyone!

The complex FFT using complex-valued interferograms is crucial for asymmetric interferometry data such as DFTS or nanoFTIR.
Currently, the OWFFT widget only calculates complex FFT for a specifically defined datatable organization. Namely, if the data is not read by NeaReaderGSF or NeaReaderMultiChannel, there is no option for the user to do a complex FFT using amplitude and phase input data tables separately.

I modified the widget to correct the lack of flexibility of the widget so users can apply the processing to any data.

Additionally, there is an option for asymmetric or symmetric apodization, in general. As far as I heard, @stuart-cls might wanted to have something like this.

Modifications:

  • New checkbox option for complex FFT
    • stored_phase input is used as the phase of the interferograms
    • disables phase_corr since it does not make sense in this case
  • Asymmetric apodization
    • Added option for asymmetric or symmetric apodization functions

The modified widget should be backward compatible since I have not removed any functionality.

@stuart-cls
Copy link
Member

Thanks for this @ngergihun . I did not go through in detail yet but it seems that some tests are failing.

A few high-level requests:

  1. Can you remove the changes that are just formatting changes? I assume your IDE started fixing stuff but it's a bit distracting.
  2. If we are touching apodization, can we please address [WIP][FIX] irfft: ramp correction and apodization fix for asymmetric interferograms #641 ? As discussed there, we have a language problem of what "assymetric" actually means in practice: sometimes the interferometer itself is asymmetric (SNOM) and sometimes the FT-IR truncates the acquisition on one side to save time (but the interferometer is still symmetric). That PR just got stuck on how to present the difference to the user.
  3. I think the complex-for-everyone stuff is straight-forward, let's split into 2 PRs

FYI I'm currently carrying a patched irfft.py (which I think should be a separate module) for the apodization case.

@ngergihun
Copy link
Contributor Author

ngergihun commented May 15, 2025

I am correcting the tests. I made a change in the UI settings update, so it does not put back the HeNe wavelength to the dx edit field all the time, when you change the input. This messes up one test function.

  1. Can you remove the changes that are just formatting changes? I assume your IDE started fixing stuff, but it's a bit distracting.

That was my lint workflow. Tried to revert it not to apply for the old part of the code, but unfortunately, some stuff got committed, my bad. I make sure they are reverted.

  1. If we are touching apodization, can we please address [WIP][FIX] irfft: ramp correction and apodization fix for asymmetric interferograms #641 ? As discussed there, we have a language problem of what "assymetric" actually means in practice: sometimes the interferometer itself is asymmetric (SNOM) and sometimes the FT-IR truncates the acquisition on one side to save time (but the interferometer is still symmetric). That PR just got stuck on how to present the difference to the user.

I'll have a look. I agree that the naming is sometimes misleading. I guess you don't want an asymmetric (different slopes for negative and positive zpd side) apodization for truncated data, since it distorts the ifg. So far the owfft widget always did that. Now, with this PR, unchecking the asymmetric checkbox, it gives you the symmetric but truncated apodization function.

  1. I think the complex-for-everyone stuff is straight-forward, let's split into 2 PRs

Okay, so should I remove the apodization-related part of this PR?

@stuart-cls
Copy link
Member

I guess you don't want an asymmetric (different slopes for negative and positive zpd side) apodization for truncated data, since it distorts the ifg. So far the owfft widget always did that. Now, with this PR, unchecking the asymmetric checkbox, it gives you the symmetric but truncated apodization function.

Precisely. This would be great!

should I remove the apodization-related part of this PR?

If it's not too much work, it would be nice to discuss them separately.

@ngergihun
Copy link
Contributor Author

should I remove the apodization-related part of this PR?

If it's not too much work, it would be nice to discuss them separately.

OK, done. I removed the apodization part. All tests seem to run.

Copy link
Member

@stuart-cls stuart-cls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should take this opportunity to simplify things rather than make them more complex (haha).

self.use_phaseinput_for_complexfft should just be self.complexfft and controls which calculate*FFT function is called.

self.use_polar_FFT should only inform that the data on the input has the interleaved complex structure. (And maybe rename it to clarify that)

I put some changes above, but also at


        try:
            channel_data, detail = self.data.attributes["Channel Data Type"]
            if channel_data == "Polar":
                self.use_polar_FFT = True
ADD
                self.complexfft = True
                self.controls.complexfft.setDisabled(True)

which should end up with the same behaviour as before when "Channel Data Type" = Polar

out_limit2 = settings.Setting(4000)
autocommit = settings.Setting(False)
use_phaseinput_for_complexfft = settings.Setting(False)
asym_apod = settings.Setting(False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this can be dropped from this PR

from Orange.widgets.utils.widgetpreview import WidgetPreview
WidgetPreview(OWFFT).run(load_test_gsf())
data = Orange.data.Table("20250412-TGQ1-O2A-ifg-lowdrift.xyz")
phase = Orange.data.Table("20250412-TGQ1-O2P-ifg-lowdrift.xyz")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data is not available in the repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry, this was changed in the commits regarding the apodization and after removing them, it came back. I'll change it.

else:
self.controls.phase_corr.setDisabled(False)
self.controls.phase_res_limit.setDisabled(False)
self.controls.phase_resolution.setDisabled(False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This results in some strange GUI behaviour where things get disabled but not enabled when use_polar_FFT = True


cb0 = gui.checkBox(
self.optionsBox, self, "use_phaseinput_for_complexfft",
label="Complex FFT with phase input",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just "Complex FFT" (see general comment below)

def commit(self):
if self.data is not None:
if self.use_polar_FFT:
if self.use_polar_FFT or self.use_phaseinput_for_complexfft:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just if self.complexfft (re general comment)

self.Error.clear()
self.Warning.clear()

if self.use_phaseinput_for_complexfft:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flip this to if self.use_polar_fft -> interleaved

wavenumbers_domain, X=phases, metas=self.data.metas[1::2]
)

if self.use_phaseinput_for_complexfft:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flip to if self.use_polar_FFT

@ngergihun
Copy link
Contributor Author

@stuart-cls, sorry for the late reply, I was busy with measurements.

I think we should take this opportunity to simplify things rather than make them more complex (haha).

self.use_phaseinput_for_complexfft should just be self.complexfft and controls which calculate*FFT function is called.

self.use_polar_FFT should only inform that the data on the input has the interleaved complex structure. (And maybe rename it to clarify that)

Thanks for the suggestion for the logic. So as far as I understand the complex fft with external phase or the interleaved complex fft both only run with the complex fft enabled. What should happen if you connect an interleaved datatable while the complex fft checkbox is not checked? Should be just raise and error to warn the user?

@stuart-cls
Copy link
Member

stuart-cls commented May 16, 2025

@ngergihun no problem, it's morning for me :)

What should happen if you connect an interleaved datatable while the complex fft checkbox is not checked?

As I understand it, the previous code only recognized the interleaved data by the presence of data.attributes["Channel Data Type"] == "Polar" . So I proposed that when that condition is met, we set the self.complexfft Setting/checkbox to True (and also the interleaved data flag, previously called use_polar_fft).

So the behaviour for that data type is the same as before, but we standardize on a single setting to control complex calculation. I tried to show that with this later piece:

        try:
            channel_data, detail = self.data.attributes["Channel Data Type"]
            if channel_data == "Polar":
                self.use_polar_FFT = True
ADD
                self.complexfft = True
                self.controls.complexfft.setDisabled(True)

Another reason I like this separation is when we find a solution to #808 we can just remove this part but the user interface and settings don't have to change.

@ngergihun
Copy link
Contributor Author

@stuart-cls thanks for the clarification. I tried to decomplexify ( :D :D :D) the code as you suggested. Also, made sure that the phase correction controls are enabled again. Please check if you see any stupid mistakes.

@ngergihun
Copy link
Contributor Author

Another thing that we discussed with @borondics , that the checkbox solution for the data spacing is sometimes very annoying as it is currently. When you (or in some cases the code) set it checked, the number in the input field is overwritten. Thus it is a bit inconvenient.

We thought what if we use radiobuttons with 3 options:

  • HeNe - display label
  • Manual - Edit field
  • Auto - display label
    Auto would call check_metadata but the flexibility is still there to change. What is your opinion @stuart-cls ? This is just an idea for now.

Another question regarding this topic. As far as I see, the dx is the pathlength stepsize and not the mirror stepsize. Is this correct?

Copy link
Member

@stuart-cls stuart-cls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

There's something funny with the auto HeNe but I agree that it's no longer appropriate. Most data contains the actual spacing used, we should prefer that. I'll put something in another PR.

@stuart-cls stuart-cls changed the title OWFFT upgrade [ENH] OWFFT: Configure Complex FFT manually May 23, 2025
@ngergihun
Copy link
Contributor Author

Thanks for your answer and help @stuart-cls, I guess after the others' approval, we could merge this.

@borondics
Copy link
Member

Hey Guys! I think it is great. Let's merge it.

@stuart-cls stuart-cls merged commit 0123310 into Quasars:master May 26, 2025
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants