Cannot find the code of guided diffusion with multiple objectives

[EvaFlower]

Hi,

Thanks for solving my former issue #3.

I notice that your upload code seems not including the LaMBO-2 part. I am really interested in the NOS combining with the multiple objective optimization. Would you consider uploading the relevant code?

Thanks!

[Morell1123]

Hi,
LamBO-2 works fine for me when I replicate their sasa experiments.
You need to first train the diffusion model with guidance (see last 3 parameters below)
change "OBJECTIVE NAME 1" to "sasa" for their example

PYTHONPATH="." python scripts/train_seq_model.py
model=[MODEL TYPE]
model.optimizer.lr=[MODEL LR]
data_dir=[DATASET DIRECTORY]
train_fn=[TRAINING CSV FILE]
val_fn=[VALIDATION CSV FILE]
vocab_file=[VOCAB FILE IN THIS REPO'S BASE DIR]
log_dir=[LOGGING DIRECTORY]
'target_cols=[[OBJECTIVE NAME 1], ..., [OBJECTIVE NAME K]]'
model.network.target_channels=[K]
discr_batch_ratio=[RATIO OF GENERATIVE LOSS UPDATES TO DISCRIMINATIVE] \

Then, you can select one of your model weights checkpoints, and do the LamBO-2 sampeling, which optimises for your model's objectives
PYTHONPATH="." python scripts/control/sample_diffusion.py
model=mlm
model.network.target_channels=1
ckpt_path=[CKPT PATH, MODEL TRAINED FOR BETA SHEETS]
+guidance_kwargs.step_size=1.0
+guidance_kwargs.stability_coef=0.01
+guidance_kwargs.num_steps=10
+seeds_fn=[PATH TO poas_seeds.csv]
+results_dir=[RESULTS DIR] \

[ngruver]

@EvaFlower We are planning to release updated LaMBO-2 code through the Prescient design repository before the paper's presentation at NeurIPS. This new codebase will include the full functionality for partial deep ensembles and the multi-objective acquisition functions described in the paper. Open source moves a bit slower in biotech because of intellectual property concerns. Thank you for your patience!

[Morell1123]

@ngruver Is LaMBO-2 not fully functional at the moment if one uses train_seq_model.py and subsequently sample_diffusion.py on ones own objectives? If not, what parts are missing?

[ngruver]

Yep, it is fully functional to run on your own objectives, and you could simply work with your own implementation of multi-objective acquisition functions, or optimize a simple linear scalarization in order to optimize multiple objective. You should be able to reproduce the experiments with the in silico objectives. The update will provide the additional primitives for multi-objective Bayesian optimization described in the paper, which were used in the in vitro antibody design experiments.

[Morell1123]

Thanks a lot for your reply.
Lets say one has only a small dataset for antibodies with objective scores. Is it possible to train the unguided infilling using your antibody sequences and then subsequently expand the model with the regression head and fine tune to ones antibody sequences with objective scores?

[ngruver]

Yes that should work!

[Morell1123]

About the coming update:
Are the independent discriminative model, which you used to filter the 30K LaMBO-2 suggestions down to 68, going to be disclosed in the NeurIPS update?
Also, can you say, how many sequences you had in the original seeds pool you used in the Antibody Lead Optimization? (Or was it only 3 as signified by the 3 "x" signs in the plot?)

[ngruver]

Sadly the ranking models were trained on data internal to Genentech, and therefore we will not be able to release the weights. We will try to add a more in depth description of how these models were trained in our next arxiv update. Re: seeds, that is correct. You can also take a look at Figure 15 in the current version on arxiv: https://arxiv.org/pdf/2305.20009.pdf

[tnkyj]

I'm sorry for the repeated question, but has the complete version of the LaMBO-2 code already been released? If it has been released, could you please provide the URL to the repository?

[ngruver]

@samuelstanton

[tnkyj]

The question was insufficient. I'm very interested in the complete code of LaMBO-2 as mentioned in this comment (especially the combination of the diffusion model and partial deep ensembles) Are you suggesting that we can supplement the missing parts of the current code by referencing LaMBO?

[ngruver]

Apologies. I tagged Sam because he is actively working on the release of the LaMBO-2 code and could give you an update. I believe he is on track to release it in upcoming week or two, but I'm not really sure, as the repo will be the intellectual property of prescient design, and I am not an employee.

[tnkyj]

I apologize for the misunderstanding. I appreciate your generous response.

[ngruver]

No worries at all, and thank you for your interest!

[EvaFlower]

@ngruver Hi, I wonder whether the code and data to replicate the results in Section 5.3 (Antibody lead optimization: in silico evaluation) has been released? Thanks for your attention!