result on llama3.2-1b-base

[SXxinxiaosong]

accelerate launch --num_processes 8 --multi_gpu --mixed_precision "bf16" run_ccot.py
--model_name_or_path models/Llama-3.2-1B
--config_name configs/pccot_llama3.2_1b.json
--config_overrides num_iterations=3,loss_gamma=20.0,use_layerwise_std=false
--num_latent_tokens 24
--train_file dataset/gsm/train.json
--validation_file gsm/valid.json
--test_file /mnt/geminisgceph1/geminicephfs/mmsearch-luban-universal/group_semantic_doc/user_mylasong/LatentReasoning/dataset/gsm/test.json
--label_names labels cot_labels
--use_peft False
--lora_target_modules q_proj-k_proj-v_proj-o_proj-down_proj-up_proj-gate_proj
--lora_modules_to_save ""
--remove_unused_columns false
--per_device_train_batch_size 16
--per_device_eval_batch_size 32
--auto_find_batch_size
--gradient_accumulation_steps 1
--block_size 1024
--attn_implementation flash_attention_2
--use_liger_kernel
--lr_scheduler_type cosine
--warmup_ratio 0.03
--learning_rate 8e-4
--weight_decay 1e-1
--bf16
--torch_dtype bfloat16
--do_train
--do_predict False
--num_train_epochs 10
--save_strategy epoch
--eval_strategy epoch
--logging_steps 50
--report_to wandb
--run_name pcot-llama3.2_1b-gsm-num_iterations3_num_latent_tokens24_gamma20
--output_dir outputs/pcot-llama3.2_1b-gsm-num_iterations3_num_latent_tokens24_gamma20
--seed 1
--overwrite_output_dir \

Thanks for your great work!
I trained llama-3.2-1b-base using the hyperparameters mentioned above for full- parameters tuning, but the final performance was quite poor (acc=0.02). Could you please suggest recommended training parameters for llama-3.2-1b? My understanding is that the base model and the instruction model should not differ too much in terms of configuration and results.

[why-in-Shanghaitech]

Hi, thank you for reaching out!

The observation from my experience is that full-parameter training has a very different behaviour and performs worse compared to peft approaches. See Appendix C.2 for more details. Therefore, I suggest

1. Switch to lora training;
2. Stick to full-parameter training but try other hyperparameters. My best experiment achieves 34.2 on GSM8K with a 160M Llama model:

accelerate launch run_clm.py --model_name_or_path outputs/JackFram--llama-160m-pcot --loss_gamma 1.0 --num_iterations 6 --num_latent_tokens 6 --train_file data/gsm_train.json --validation_file data/gsm_valid.json --remove_unused_columns false --per_device_train_batch_size 32 --per_device_eval_batch_size 32 --auto_find_batch_size --gradient_accumulation_steps 1 --block_size 2048 --attn_implementation flash_attention_2 --use_liger_kernel --lr_scheduler_type cosine --warmup_ratio 0.03 --learning_rate 6e-4 --weight_decay 1e-1 --bf16 --torch_dtype bfloat16 --do_train --do_eval --do_predict --num_train_epochs 40 --save_total_limit 1 --save_strategy steps --save_steps 200 --evaluation_strategy steps --eval_steps 200 --logging_steps 50 --load_best_model_at_end True --metric_for_best_model eval_ccot_exact_match --report_to wandb --run_name pcot-llama-160m-a100-v1.2 --overwrite_output_dir --output_dir outputs/pcot-llama-160m-a100-v1.2

Just for your reference since this script is based on a very early version of the code base.

[SXxinxiaosong]

Thank you for your timely reply! I’ll try out your suggestions.

[SXxinxiaosong]

When using the instruct LLM (llama3.2-1b-instruct), why wasn’t a chat template applied during dataset processing?

[why-in-Shanghaitech]

We mainly ignore this for simplicity reason. For GSM8K-AUG the reasoning process is very different from natural language so I didn't expect much performance gain with the chat template. It seems that CODI also didn't use the chat template and we just follow their setting.
Of course, you may apply the chat template and see what will happen. In GSM8K-NL I guess it would help.

[SXxinxiaosong]

Thanks for your reply! CODI uses the Llama3.2-1B base model instead of the instruct model. Have you observed any significant differences in performance between the base and instruct models?

[why-in-Shanghaitech]

Hi,

1. CODI actually uses the instruct model (https://huggingface.co/zen-E/CODI-llama3.2-1b-Instruct). The footnote in my paper turns out to be a misunderstanding (I finished the paper before codi releases their codes) and I'll fix it in the next version of our paper;
2. Yes, I use the instruct model because of better performance. My early experiments runs with 6 latent tokens with 6 extra iterations, the base model accuracy on GSM8K-AUG is about 49.8 and the instruct model can reach 53.0. These models are tuned separated, and tested on the validation set of GSM8k-AUG.

[SXxinxiaosong]

Thank you for the quick answer!

[why-in-Shanghaitech]

Hi, I want to share some results about applying chat template with you.

Setting	Dataset	Accuracy
w/o chat template	GSM8K	53.35 $\pm$ 0.18
w/ chat template	GSM8K	52.24 $\pm$ 1.14
w/o chat template	GSM8K-NL	50.72 $\pm$ 1.39
w/ chat template	GSM8K-NL	51.76 $\pm$ 0.37

These experiments are run on H20x4, Llama3.2-1B-Instruct. I apply the chat templates to the question as user query and let the assistant to generate the cot steps and answer. The performance does not change much -- it becomes worse on GSM8K but better on GSM8K-NL (but not much). I didn't tune the models, just follow the hyperparameters in the paper.

Still thank you for reminding me this thing about chat template, I think I will add these result to my next version of paper. The code has been added to the code base with an additional argument --use_chat_template.