Error in function test_operator_derivative() in test_fhcl_force.py

[sunneos]

Hi,

I encountered a few issues while running the test_operator_derivative function in tests/test_fhcl_force.py, and I’d like to confirm whether these are expected behaviors or if I misunderstood something.

---

1. Argument order in csv_to_molecular_reps

The arguments passed to generate_fchl18 and its variants seem to be in the wrong order. It should be:

rep = generate_fchl18(
    nuclear_charges, coordinates, max_size=max_atoms, cut_distance=CUT_DISTANCE
)

disp_rep = generate_fchl18_displaced(
    nuclear_charges, coordinates, max_size=max_atoms, cut_distance=CUT_DISTANCE, dx=DX
)

disp_rep5 = generate_fchl18_displaced_5point(
    nuclear_charges, coordinates, max_size=max_atoms, cut_distance=CUT_DISTANCE, dx=DX
)

Currently, it uses coordinates, nuclear_charges, which appears to cause issues in descriptor generation.

---

2. Force padding and shape mismatch

In test_operator_derivative, this line causes issues if molecules have different atom counts:

Fall = np.array(Fall)

Since Fall contains force arrays with variable shape (based on atom count), this conversion fails or produces incorrect concatenation. I removed this line and used padding only where necessary for specific operations.

---

3. MAE assertions failing

The following assertions fail for me with current default settings:

assert mae(Ess, Es) < 0.08, "Error in operator test energy"
assert mae(Et, E) < 0.04, "Error in operator training energy"
assert mae(Fss, Fs.flatten()) < 1.1, "Error in operator test force"
assert mae(Ft, F.flatten()) < 0.1, "Error in operator training force"

The computed MAEs exceed the specified thresholds and vary between runs, sometimes significantly(lager than 20 eV).
In some cases, the errors are much higher than expected.
I wonder if:

• These values are outdated for the current kernel/hyperparameters?
• Or additional regularization should be introduced ?

---

System Info

Python: 3.10.18 | packaged by conda-forge | (main, Jun  4 2025, 14:45:41) [GCC 13.3.0]
Platform: Linux-5.14.0-503.40.1.el9_5.x86_64-x86_64-with-glibc2.34
NumPy: 2.2.6
SciPy: 1.15.2
qmllib: 1.1.9

---

test_code.txt

The code I used is attached below.
Please let me know if I’m missing something or if this behavior is expected.
I’d be happy to help contribute fixes as well.

Thanks!

[charnley]

Those tests are broken and are flagged with pytest.skip(). I recommend using fchl19 instead.

P.S., did you create this issue with ChatGPT?

[sunneos]

Yes, I've used ChatGPT to help create this issue since I wanted to make it clearer and more formal.Please forgive me if using ChatGPT is inappropriate in any way.
Thank you for your reply. I have tried FCHL18 trained only on energy, and now I want to compare training with both energy and forces versus training with only energy.
I will also try using FCHL19 soon. Thank you for your advice!

[sunneos]

@charnley Hi, when I use FCHL19, I noticed that it didn't support systems with PBC like FCHL18. Are there any plans to support PBC in future versions ?

[charnley]

There is an open PR #6, can you check out that development branch and see if that works for you?

[sunneos]

Thanks, that’s very kind of you, and I’ll check it out later.