fix: stabilize DE Spearman correlation floats

[giovp]

Cast DE fold-change columns to Float64 at correlation time and add a regression test for mixed Float32/Float64 inputs.

With latest cell-eval and python=3.12 the de_spearman_lfc_sig calculation failed

[gemini-code-assist]

Summary of Changes

Hello @giovp, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical stability issue in the DESpearmanLFC metric calculation. By enforcing Float64 precision for fold-change columns during Spearman correlation, it prevents failures that occurred with specific cell-eval and Python environments. The inclusion of a new test case ensures the robustness of the metric when encountering varied floating-point input types, thereby enhancing the reliability of differential expression analysis.

Highlights

• Float Type Stabilization: The pull request explicitly casts fold-change columns to Float64 before calculating the Spearman correlation, ensuring consistent numerical precision.
• Regression Test Added: A new regression test has been introduced to validate that the DESpearmanLFC metric correctly handles inputs with mixed Float32 and Float64 types.
• Bug Fix: This change resolves an issue where the de_spearman_lfc_sig calculation failed with recent cell-eval versions and python=3.12 due to floating-point precision inconsistencies.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request fixes a bug in the Spearman correlation calculation when dealing with mixed float types by casting the columns to Float64. It also adds a regression test. The fix itself is correct and robust. However, the new regression test is flawed as it doesn't correctly set up the mixed-float-type scenario it intends to test, and its assertions are weak. I've provided a suggestion to improve the test to ensure it's effective.

[gemini-code-assist]

This regression test is intended to check for issues with mixed float types, but it doesn't seem to set up the test data correctly to achieve this. Additionally, the assertions could be stronger.

• Incorrect test setup: The DEResults class's __post_init__ method casts the fold_change column to pl.Float32 for both real and pred dataframes. This means the mixed Float32/Float64 scenario described in the PR is not actually being tested. Both dataframes will have Float32 columns.

• Weak assertions: The test only asserts that the result is a dictionary of floats/ints. It would be more robust to assert the actual correlation values, which should be 1.0 in this case for both perturbations since the fold_change values are identical.

Here is a suggested improvement to the test to correctly simulate the mixed-type scenario and add stronger assertions:

def test_de_spearman_lfc_mixed_float_types() -> None:
    """Regression test: DESpearmanLFC handles mixed Float32/Float64 columns."""
    base_df = pl.DataFrame(
        {
            "target": ["pert1", "pert1", "pert2", "pert2"],
            "feature": ["gene1", "gene2", "gene1", "gene2"],
            "fold_change": [1.5, 2.0, 0.5, 1.2],
            "p_value": [0.01, 0.02, 0.03, 0.04],
            "fdr": [0.01, 0.02, 0.03, 0.04],
        }
    )

    # DEResults casts to Float32 by default
    real_res = DEResults(base_df.clone(), name="real")

    # Create a pred result and then manually cast its fold_change to Float64
    # to simulate the mixed-type scenario.
    pred_res = DEResults(base_df.clone(), name="pred")
    pred_res.data = pred_res.data.with_columns(
        pl.col(pred_res.fold_change_col).cast(pl.Float64)
    )

    comparison = DEComparison(
        real=real_res,
        pred=pred_res,
    )

    result = DESpearmanLFC(fdr_threshold=0.05)(comparison)

    expected = {"pert1": 1.0, "pert2": 1.0}
    assert result == expected

[noamteyssier]

@giovp thanks!