-
Notifications
You must be signed in to change notification settings - Fork 11
Description
When done, need to unify unit tests so there is only one test for "anti-biased" ("rating_bias").
See PR for the comparison case:
#265
good_agent = {"buyers":[1,80], "products":[101,180], "qualities":[0.5,0.75,1.0], "transactions": good_transactions}
bad_agent = {"buyers":[81,100], "products":[181,200], "qualities":[0.0,0.25], "transactions": bad_transactions}
days = 101
The results are:
$ python reputation_buying_test.py
RS=Anti-biased_def5_dec5_c5_P PLRo=30 agents=80/20/80/20 days=101 Organic: 16280 Sponsored: 4000 Organic2Sponsored: 3070 Organic/Sponsored: 4.07 LTS: 0.19 PFS: 0.77
RS=Anti-biased_def5_dec5_c5_J PLRo=30 agents=80/20/80/20 days=101 Organic: 15880 Sponsored: 4000 Organic2Sponsored: 3050 Organic/Sponsored: 3.97 LTS: 0.19 PFS: 0.76
Next, try larger scale:
good_agent = {"buyers":[1,400], "products":[1001,1400], "qualities":[0.5,0.75,1.0], "transactions": good_transactions}
bad_agent = {"buyers":[401,500], "products":[1401,1500], "qualities":[0.0,0.25], "transactions": bad_transactions}
days = 501
The discrepancy gets much more significant:
$ python reputation_buying_test.py
RS=Anti-biased_def5_dec5_c5_P PLRo=30 agents=400/100/400/100 days=501 Organic: 377420 Sponsored: 100000 Organic2Sponsored: 87480 Organic/Sponsored: 3.77 LTS: 0.23 PFS: 0.87
RS=Anti-biased_def5_dec5_c5_J PLRo=30 agents=400/100/400/100 days=501 Organic: 362830 Sponsored: 100000 Organic2Sponsored: 72890 Organic/Sponsored: 3.63 LTS: 0.2 PFS: 0.73
Java version performs better so need to align to it:
https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/peer/Reputationer.java#L449
Look for "pessimism" in Java code.