Confidence Intervals for Difference of Binomial Proportions

Computation of confidence intervals for binomial proportions and for difference of binomial proportions.

[GitHub Pages] [Read the Docs]

🚀 NEW 🚀 Streamlit support! See here for an app deployed on Streamlit Community Cloud.

Installation

Run

python -m pip install diff-binom-confint

or install the latest version in GitHub using

python -m pip install git+https://github.com/DeepPSP/DBCI.git

or git clone this repository and install locally via

cd DBCI
python -m pip install .

Numba accelerated version

Install using

python -m pip install diff-binom-confint[acc]

Usage examples

from diff_binom_confint import compute_difference_confidence_interval

n_positive, n_total = 84, 101
ref_positive, ref_total = 89, 105

confint = compute_difference_confidence_interval(
    n_positive,
    n_total,
    ref_positive,
    ref_total,
    conf_level=0.95,
    method="wilson",
)

Implemented methods

Confidence intervals for binomial proportions

Click to view!

Method (type)	Implemented
wilson	✔️
wilson-cc	✔️
wald	✔️
wald-cc	✔️
agresti-coull	✔️
jeffreys	✔️
clopper-pearson	✔️
arcsine	✔️
logit	✔️
pratt	✔️
witting	✔️
mid-p	✔️
lik	✔️
blaker	✔️
modified-wilson	✔️
modified-jeffreys	✔️

Confidence intervals for difference of binomial proportions

Click to view!

Method (type)	Implemented
wilson	✔️
wilson-cc	✔️
wald	✔️
wald-cc	✔️
haldane	✔️
jeffreys-perks	✔️
mee	✔️
miettinen-nurminen	✔️
true-profile	✔️
hauck-anderson	✔️
agresti-caffo	✔️
carlin-louis	✔️
brown-li	✔️
brown-li-jeffrey	✔️
miettinen-nurminen-brown-li	✔️
exact	❌
mid-p	❌
santner-snell	❌
chan-zhang	❌
agresti-min	❌
wang	✔️
pradhan-banerjee	❌

Creating report

One can use the make_risk_report function to create a report of the confidence intervals for difference of binomial proportions.

from diff_binom_confint import make_risk_report

# df_train and df_test are pandas.DataFrame providing the data
table = make_risk_report((df_train, df_test), target = "binary_target")
# or if df_data is a pandas.DataFrame containing both training and testing data
table = make_risk_report(df_data, target = "binary_target")

For more details, see corresponding documenation. The produced table is similar to the following:

Click to view!

References

1. SAS
2. PASS
3. statsmodels.stats.proportion
4. scipy.stats._binomtest
5. corplingstats
6. DescTools.StatsAndCIs
7. Newcombee

NOTE

Reference 1 has errors in the description of the methods Wilson CC, Mee, Miettinen-Nurminen. The correct computation of Wilson CC is given in Reference 5. The correct computation of Mee, Miettinen-Nurminen are given in the code blocks in Reference 1

Test data

Test data are

1. taken (with slight modification, e.g. the upper_bound of miettinen-nurminen-brown-li method in the edge case file) from Reference 1 for automatic test of the correctness of the implementation of the algorithms.

2. generated using DescTools.StatsAndCIs via

   library("DescTools")
   library("data.table")
   
   results = data.table()
   for (m in c("wilson", "wald", "waldcc", "agresti-coull", "jeffreys",
                   "modified wilson", "wilsoncc", "modified jeffreys",
                   "clopper-pearson", "arcsine", "logit", "witting", "pratt",
                   "midp", "lik", "blaker")){
       ci = BinomCI(84,101,method = m)
       new_row = data.table("method" = m, "ratio"=ci[1], "lower_bound" = ci[2], "upper_bound" = ci[3])
       results = rbindlist(list(results, new_row))
   }
   fwrite(results, "./test/test-data/example-84-101.csv")  # with manual slight adjustment of method names

3. taken from Reference 7 (Table II).

The filenames has the following pattern:

# for computing confidence interval for difference of binomial proportions
"example-(?P<n_positive>[\\d]+)-(?P<n_total>[\\d]+)-vs-(?P<ref_positive>[\\d]+)-(?P<ref_total>[\\d]+)\\.csv"

# for computing confidence interval for binomial proportions
"example-(?P<n_positive>[\\d]+)-(?P<n_total>[\\d]+)\\.csv"

Note that the out-of-range values (e.g. > 1) are left as empty values in the .csv files.

Known Issues

1. Edge cases incorrect for the method true-profile.