[WIP][FIX] irfft: ramp correction and apodization fix for asymmetric interferograms

[stuart-cls]

Fixes an old error in the apodization function generation for asymmetric interferograms, found while adding a ramp correction to avoid double-counting in the symmetric section. Illustration of the issue:

The above figure illustrates how the original apodization code was handling the taper to zero needed in asymmetric interferograms. While it does bring the amplitude to zero at the edge of the short side, it obviously badly violates the symmetry requirements. This would not have been obvious on the original symmetric test interferograms.

[borondics]

@stuart-cls I think the "original" (on your figure) asymmetric apodization was meant for SNOM IFGs and the intention was to have that shape, where not only the position of the ZPD is offset but also the apodization. In my understanding the reason for this is that the near field signal is contained only on one side of the ZPD and the other should be down-weighted. There is some literature on this: http://hdl.handle.net/10810/26669

NeaSpec does something very similar, but their calculations are proprietary.

However, I think there is some actual scientific discussion missing... @raul-freitas, @levandoskije, what do you think?

[stuart-cls]

@borondics I'm pretty sure this part of the code predates SNOM!

That said, I definitely don't want to break anything correct. I will simply note that for asymmetric interferograms in an FT-IR, downweighting the short side via shrinking the apodization window ("old") is not the same as downweighting the short side via the ramp ("new + ramp"), and the latter is the correct way.

I'll review the literature you sent, I'm also waiting on other feedback.

If you have any before / after data of what Neaspec is doing, please send it along: we can compare the two approaches. This is basically what I did for the Agilent imaging data to check for correctness.

[raul-freitas]

Hello @borondics and @stuart-cls.
I will try to answer you soon with some old data. I will get back to this subject quite soon and I hope to have some direct feedback from the company, as we are processing raw interferograms (.gsf files). I remember playing a bit in the past and comparing my processing with Neaspec processing, I could not exactly reproduce (at least the high frequency features).

Maybe we can plan some synchronized activities soon. I have two new folks that will give some support to Orange-Quasar in here.

[borondics]

I was just browsing SciPy for something and realized that many (all?) of the apodization functions are implemented there. Since SciPy is a dependency, should we simply use those functions to decrease the code?

[stuart-cls]

@borondics I'm aware of those functions. Originally I wanted to keep the dependencies of irfft minimal for easy re-use in other projects. scipy is not so annoying to install anymore but I'd still rather keep that.

Dependencies aside, I looked at how those windows are implemented this time around and they don't work for either of the types of apodization we would like to do.

Which brings me to the summary of the current state of this PR: based on our discussions offline, the thesis above, and feedback from others, I think we will have to distinguish between interferometer types:

1. Symmetric (Michelson-type) interferometer
   a. Fully symmetric interferogram ("Symmetric" in current GUI)
   b. Truncated symmetric interferogram ("Asymmetric" in current GUI)
2. Asymmetric (s-SNOM type) interferometer

where type (1) will use this symmetric + ramp approach and type (2) will use the existing approach of different apodization functions for each wing.

[ngergihun]

After @stuart-cls mentioned this existing issue in ##809, probably it is time to revisit this. As discussed above this is a very very important. The asymmetry apodization (which is currently in the widget) introduces serious artifacts.

However, it is necessary for SNOM interferograms. https://doi.org/10.1364/oe.520793

There should be an option in the apodization section. Either just a checkbox for asymmetric apodization or a radiobutton, or dropdown to choose.

[stuart-cls]

@ngergihun Could you take a look? I restored the true asymmetric mode and added the option. Could you test and compare with your other code?

Something is still not quite right with the phase correction data I think, still thinking about that.

[ngergihun]

@stuart-cls , yes they are still giving different results. Hmmmm.... I will try to send something more useful report in a few hours....
Also, what I noticed, that are is a difference in the number of features, that should not be the case. The ifg I used has 530 points, and my code with zerofilling of 2 gives back 1060 points, while the owfft results in 1025 points.

[stuart-cls]

Two outstanding things from @ngergihun observations:

• Change default in irfft to apod_asym = True so that the output doesn't change by surprise. Raise a warning if unset to allow porting to the new interface.
• Figure out what's going on with the zero filling difference*.
• Regression in match to agilent data (phase correction?)

• For posterity, the irfft code constains the zero-filling padding to the nearest power of 2 for DFT efficiency, see

  orange-spectroscopy/orangecontrib/spectroscopy/irfft.py

  Line 151 in 8ab04d3

  return int(np.exp2(np.ceil(np.log2(Nzff))))