mysofa_load() returns hrtf object that is missing actual data in DataIR

[janhenhan]

Dear authors,

Using libmysofa's mysofa_load(), I would like to load the attached testfile.sofa.zip sofa file of type 'General FIR-E' that has been created with sofar in python.
mysofa_load() returns an hrir without throwing an error. All the metadata and delay properties are read correctly. However, DataIR has 0 elements and does not contain the actual IR data from the sofa file.

This can be reproduced with e.g. with mysofa2json which also then is lacking the output of DataIR in the json.

It would be great if this could be fixed and libmysofa's mysofa_load could support reading these sofa files.

Thank you!

[janhenhan]

Or if you have any recommendations how I can best understand what is going on and figure out a fix myself that would be very much appreciated.

Thank you very much

[daniel-embody]

@janhenhan, unsure if this is still an open issue for you, but I've just encountered what I believe to be the same issue with a new sofa my coworkers have generated. (I don't know the method yet that they use to do so, but previous sofas have worked without issue)

In using your testfile.sofa, I've stepped through attempting to load and the "problem" occurs is dataobject.c.

In your sofa, the layout_class is returned as 2, with a dimensionality of 5. Your data->datalayout_chunk is then read in as {3, 2, 256, 2, 8}.

With a dimensionality of 5, we fail the condition if (validAddress(reader, data_address) && dimensionality <= 4) {, which then skips over the allocation/reading of the sofa data, resulting in an empty Data.IR field.

I see you're using https://sofar.readthedocs.io/en/stable/sofar.html to generate your sofa, and will look into this library to see if there anything I can find!

[hoene]

@daniel-embody @janhenhan Shall we fix that in libmysofa, now finally?

[daniel-embody]

Herr @hoene, if you know the solution, a fix would be wonderful!

[hoene]

@daniel-embody Sorry, I was a bit overloaded the past months and have forgotten this request. I have some free days now.

[janhenhan]

I too would appreciate a fix very very much. If I can assist in any way, e.g. by running some tests, happy to do so.

[daniel-embody]

@hoene No apologies are necessary whatsoever, but thank you for the prompt response! For my own understanding, are there any white papers or technical specifications you can link that can clarify the ODHR data layout technical specification?

In our own case, we have 3 separate sets of IRs being baked in. The "Data.IR" field itself returns with a data_layout of 1, which reads in correctly, but our additional IR sets get the same data_layout == 2 (with all metadata/properties read correctly as @janhenhan was encountering).

[janhenhan]

@hoene Would the right approach here be to remove/adapt the dimension checks in dataobject.c (ln 544) and btree.c (ln 233) and then adding higher dimension cases to the data->ds.dimensionality switch statement (ln 327)? How exactly those cases would be handled, I am not too sure about..

[hoene]

please check #240 @janhenhan @daniel-embody

[janhenhan]

I integrated the fix into my project and it all seems to work without issues! I'm seeing the IRs that I expect to see for each dimension.
Super happy to be finally able to switch to libmysofa. Thank you so much @hoene for taking the time to develop this fix/support these kind of sofa files!

[janhenhan]

Dear @hoene, I might have celebrated a bit too early.
While the small test file opened, I'm still struggling to open some of the files of interest with many IRs in them. Now it is less a matter of dimensionality support but rather size limits, I believe. I stumbled over this in the past with classical HRIR sofa files and was able to just change it by changing these limits here, but this time it seems more complex. Or at least I struggle to identify which size limits I would need to increase so that libmysofa doesn't throw a MYSOFA_UNSUPPORTED_FORMAT or MYSOFA_INVALID_FORMAT.

I'm attaching a toy example with mostly zeros, but significant number of IRs. Could you help me raising the appropriate limits so that I can read the file? I understand the unpacked IRs require a lot of memory, but modern computers have it and the SOFA format supports it, right?

Test File Large, again created with Sofar:
testfilelarge.sofa.zip

[hoene]

@janhenhan You might play around with the branch https://github.com/hoene/libmysofa/tree/feature/232-more-dimensions . However, the size of the array is getting to big for my computer and it reports an out-of-memory error.