Adaptive Amazon-kind Simulation Add-ons

[akolonin]

• Explore why different simulation runs with fulnorm=True have sellers in range below 100 (phenomena found with red/blue scattered periods). The "inactive agents with reputation higher than active agents" does not make sense - this is either bug or has to be explained
• Extend the model for "much more than 1 product in each price category"
• Explore why OMU metric is not working as in non-adaptive simulation and either find how to make it working or explain why it should not work at such scale
• Explore the situation when honest buyers have to game the regular reputation system so we can apply the advanced reputation system to minimise the SGL (also exploring the point at which overlap of good and bad agents fails to work)

Origin: #214

[deborahduong]

Q. Explore why different simulation runs with fulnorm=True have sellers in range below 100 (phenomena found with red/blue scattered periods). The "inactive agents with reputation higher than active agents" does not make sense - this is either bug or has to be explained
A. You have to remember that your reputation system is judging products, not people. Once we consider how the average ranks are calculated, it makes sense. A cumulative average of agent ranks is kept and reported out at the end of the run. Every day, the ranks of each one of every agent's active products is put into this cumulative average, as long as the agent is active. Note that each agent can sell one hundred products. Yes, if fullnorm is true, then one of the products of one of the suppliers , past and present, is rated 99, but it could be an inactive product whose score has drifted up towards the decayed value, in which case, it does not get into the calculation. Further, since an agent has about 100 products, even if one of its active products is a 99, and is recorded as such, the rest would likely be lower, so that you would never see a 99 score for an agent at all. Now take the case of decayed being 0.1. You define decayed as the number that products increment towards if they are not active any day. It makes sense that, if many of an agents products weren't active in any one day, that this would drag the average of all an agents products down. Please tell me if you do not feel this is adequately explained. If you are concerned with a difference in our simulation results, the difference is, the simulation that keeps the requirements, my simulation, has products, while yours, which doesnt claim to keep the requirements, doesnt

[deborahduong]

moved here from closed issue...

Concerning the lower red and blue dots on the bias chart vs regular chart:I just checked and the reason that all scores are generally lower in the biased run is that the champion decayed parameter is 0.1. In runs with decayed=0.9, they are as high as they are on the regular run

[deborahduong]

Q. Extend the model for "much more than 1 product in each price category" and
Q. Explore why Predictiveness works not good enough compared to "non-adaptive simulation"
A. moved here (and modified) from closed comment in business doc....
However, I think there is a misunderstanding, Id like to explain here. There is not a one product per price category assumption. There is a one product category per one price category assumption, in the spec. It isn't relevant to the fact that good consumers latch onto one good supplier for their needs for a product category. There are only 5 product categories, and the bad suppliers deal in only one of the product categories, as says our spec. Consumers don't need one of every product, they need one of every product category, meaning they only need 5 things. However, the hundreds of products in the product category are competing to fulfill a consumers need for a single thing in that category. An example of a product category is an iPhone charger. They only need one iPhone charger because they only have one iPhone, even if the market has a thousand particular iPhone products, sold by 10 iPhone suppliers. In the spec, if the good consumers find an iPhone charger supplier they really like, they go back to them when they need another iPhone charger, and importantly, If they don't like the supplier then they don't go back to him to buy anything else - however, you asked me to simplify the simulation so that he has 100 different products of one category , for example, he is an iPhone charger sales man, or a car salesman, but not both. So , once good consumer agents find a five star supplier in each product category, they don't change suppliers anymore, and the bad suppliers don't have a chance with them anymore. It is because agents have good memory of good suppliers that we need enterers and leavers. You told me that this is too complicated but I disagree. I believe this impacts predictiveness, because without enterers and leavers, you have to do you comparisons before all agents settle on their favorite five star suppliers and ignore the others for the rest of the simulation. Because this may be an early point, predictiveness does not have time to set in.

[deborahduong]

Q. Explore why OMU metric is not working as in non-adaptive simulation and either find how to make it working or explain why it should not work at such scale
A. moved here from closed issue...
Concerning OMU, in my simulation, good2bad, in the denominator added to good2good, is small compared to good2good because of the other 4 product categories present in the simulation, which have their own separate market volume that forms the bulk of the market volume in the economy. That is why ~good2good/(good2bad+good2good) is close to 1

[deborahduong]

I think its also important to include exploring the point at which overlap of good and bad agents fails to work, and this exploration can be added to "Explore the situation when honest buyers have to game the regular reputation system so we can apply the advanced reputation system to minimise the SGL."

[akolonin]

Explore why different simulation runs with fulnorm=True have sellers in range below 100 (phenomena found with red/blue scattered periods). The "inactive agents with reputation higher than active agents" does not make sense - this is either bug or has to be explained

Explained with per-seller averaging of product-specific ranks

Extend the model for "much more than 1 product in each price category"
Explore why OMU metric is not working as in non-adaptive simulation and either find how to make it working or explain why it should not work at such scale

Need to re-run the best results for PLR-dependency and across RS of different types with multiple product types per price category (50 instead of 5) and the same total number of products equal to the number of buyers.

Explore the situation when honest buyers have to game the regular reputation system so we can apply the advanced reputation system to minimise the SGL (also exploring the point at which overlap of good and bad agents fails to work)

It is expected that LTS and PFS will not work but BSL, SGP and SGL are expected to work and SGL should be opposed to "equity" #122 - so less SGL should mean greater "equity"

[deborahduong]

Fulnorm and OMU issues explained in comments above. Runs for large numbers of products and for honest buyers which switch to dishonest were done and analysis appears in technical report. So the existing issue is complete: however, we can add one more issue to this: that the Anomaly Detection, Predictiveness, and Hopfield net add ons that Duong wrote for the reputation system were not injected into the Python program because it was not working and should be injected in as soon as the program passes tests, so it results can be reported from the system and not the stub it ran on for the technical report until the system was ready. https://docs.google.com/document/d/1FEr4ir0jBZ5PZtOBas7naeHef8AnPo_E5IArSoxN0kE/edit . I am planning to have the new batch ready by the Nov 12 deadline of AAMAS