This is the code for the experiments in the paper.

Requirements

Python environment

environment.yml

Hardware?

Everything here we have run on CPU, except where specified. There, a single NVIDIA A40 GPU is used.

Create datasets

For all of these, the shell scripts must be modified to use your own data directories and wandb username. Run all of the below from the parent directory, not the data_prep directory.

Synthetic data

./data_prep/generate_data_synthetic.sh

MIND dataset (semisynthetic) (1) Download and process MIND (news data), (2) generate features using text, to be used for training, (3) generate features using text, to generate DGP, (4) generate DGP using features. Steps 2 and 3 use GPU.

./data_prep/process_data_MIND.sh
./data_prep/generate_text_features.sh
./data_prep/generate_text_labels.sh
./data_prep/generate_data_semisynthetic.sh

Train autoregressive sequence models $p_\theta$

For all of these, the shell scripts and/or config files must be modified to use your own data directories and weights and biases (wandb) username. The trained models are saved in subdirectories inside the data directories.

Synthetic dataset To train flexible neural net sequence models on synthetic dataset: run

./shell_scripts/train_context_synthetic.sh

MIND dataset (semisynthetic) To train flexible neural net sequence models on synthetic dataset: run

./shell_scripts/train_context_semisynthetic.sh

Run bandit algorithms

We run and cache some bandit algorithms, with commands listed here. Others (that are faster) we run directly alongside plotting in a jupyter notebook (see next section).

• For all of these, the shell scripts must be modified to use the directories containing the sequential models trained in the previous step.
• In these shell scripts, we have a for-loop over many environments. Please parallelize appropriately according to your resources.
• Adjust the bandit algorithm as appropriate (argument --bandit_alg).

Examples for TS-Gen:

./shell_scripts/run_bandit_synthetic.sh
./shell_scripts/run_bandit_semisynthetic.sh

Examples for TS-Neural-Linear:

./shell_scripts/run_bandit_neural_linear_synthetic.sh
./shell_scripts/run_bandit_neural_linear_semisynthetic.sh

Notebooks

Synthetic (bimodal) dataset

synthetic_regret.ipynb

MIND (news) dataset

semisynthetic_regret.ipynb

Fit priors for TS-Linear and TS-Neural-Linear

fit_prior_ts_linear.ipynb
fit_prior_ts_neural_linear.ipynb

Extras

Additional code for some experiments in the appendix are in the extras folder