From NVIDIA Megatron-LM for visibility#18

Open

RaymondLi0 wants to merge 6325 commits intobigcode-project:multi-query-attentionfrom

Collaborator

RaymondLi0 commented Jan 24, 2023

No description provided.

RaymondLi0 changed the base branch from multi-query-attention to before-merge

June 20, 2023 20:12

RaymondLi0 changed the base branch from before-merge to multi-query-attention

June 20, 2023 20:12

ko3n1g and others added 28 commits

January 24, 2026 09:28


          Revert "Support multimodule communication (#2031)" (#3068)

0972f02


          Revert "Remove calculation of padding token in moe routing loss (#2142)…

4cfaa7d

…" (#3069)


          Add ability to save wgrads and dgrads (#3032)

dbde759

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>


          ci: Mark test_mode_partial_cudagraph unit tests as flaky (#3064)

55fe705

Signed-off-by: Charlie Truong <chtruong@nvidia.com>


          Keep FSDP's and DDP's finish_grad_sync API identical (#3070)

3a7d74d

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>


          (REPLAY) Bug fix with --no-use-tokenizer-from-checkpoint-args (#3059)

389436b

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>
Co-authored-by: Charlie Truong <chtruong@nvidia.com>


          Optimizing post-processing of requests (#2920)

53a2b19


          Fix broken functional tests in #2920 (#3071)

369e0eb


          fix ep weight gradnorm/num_zero calculation error for muon (#3024)

b2e9390


          [training migration] Add LoggerConfig dataclass (#2414)

497d42d

Signed-off-by: Maanu Grover <maanug@nvidia.com>


          Added --ft-num-warmup-iters option. (#3052)

06d0f46


          Reapply "Various CUDA graph improvements on capture time, replay time…

642fdd9

…, memory footprint (#2572)" (#3056)

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>


          fix(fsdp): add CLI argument for outer_dp_sharding_strategy (#3053)

bb42a00

Co-authored-by: Jianbin Chang <shjwudp@gmail.com>


          ci: Log node name (#3081)

94d8186

Signed-off-by: oliver könig <okoenig@nvidia.com>


          add all_gather process-group for overlapping in fsdp disributed train…

528cb2e

…ing (#2663)


          docs: Release docs (#3055)

23a76d1

Signed-off-by: oliver könig <okoenig@nvidia.com>


          Support NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 FP8/NVFP4 PTQ in example (#…

b47c376

…3079)

Signed-off-by: jenchen13 <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Co-authored-by: jenchen13 <jennifchen@nvidia.com>
Co-authored-by: Jenny Chen <jc4686@columbia.edu>
Co-authored-by: Asha Anoosheh <ashaanoosheh1@gmail.com>


          Reapply "Support multimodule communication (#2031)" (#3068)

35e85a6

This reverts commit 0972f02.


          Add router replay for MoE models (#2101)

db6b895

Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Yan Bai <baiyan1996@icloud.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com>


          ci: Disable gpt_dynamic_inference_tp1_pp1_dp8_583m_throughputtest_zmq (…

…#3099)

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Repeat func tests, save logs of unit tests and lessen debug output (

dea21a0

#3089)

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Update improvement of step-time (#3104)

dd83fc6

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Add GPU health checks (#3100)

2bdf7e1

Signed-off-by: oliver könig <okoenig@nvidia.com>


          Harden GRPO functional tests (#3065)

d68721b

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>


          Inference functional tests: Write outputs to INFERENCE_OUTPUT_PATH in…

4015ff1

…stead of TENSORBOARD_PATH (#3061)


          Update moe readme. (#2830)

0888a06


          build: Bump to TE2.12 (#3086)

2b02a28

Signed-off-by: oliver könig <okoenig@nvidia.com>


          Logging cleanup (only log on rank 0 if possible) (#3036)

6cf285b

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

dndnda and others added 10 commits

February 17, 2026 13:01


          fix: correct typo in demo training example (#3428)

057c804

Co-authored-by: liming127 <liming127@meituan.com>


          Clean up logging inside inference flask server (#3437)

b218e64


          ci: Update release-docs workflow to use FW-CI-templates v0.72.0 (#3438)

3c69780

Signed-off-by: Charlie Truong <chtruong@nvidia.com>


          Fix --tokenizer-hf-include-special-tokens (#3422)

74ef64e

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>
Co-authored-by: Teodor-Dumitru Ene <34819528+tdene@users.noreply.github.com>


          Update num_tokens_to_generate default for Gym (#3453)

267cf1f


          Fix slowdown in inference flask server (#3445)


          Add a normalized scale for MTP per token loss (#3159)

a22c40e

Signed-off-by: lit <lit@nvidia.com>


          [Bugfix] Fix nan loss caused by zero token in MTP (#3396)

d7500d4

Signed-off-by: lit <lit@nvidia.com>


          ci: Add testing branches

ad5a627

Signed-off-by: oliver könig <okoenig@nvidia.com>


          chore: rotate oncall schedule

cd71d4c

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare

February 18, 2026 09:18

yobibyte and others added 19 commits

February 18, 2026 14:40


          Log RL metrics per environment (#3446)

f1908bc


          Move tensor offload/onload out of RL code (#3029)

1106df4

Co-authored-by: helen ngo <helen.ngo14@gmail.com>


          Add Engine event to the follow up requests after checkpointing (#3473)


          Fix another inference flask / Gym interaction (#3467)

7b016be

Co-authored-by: Robert Kirby <rkirby@nvidia.com>


          adding in copyright blurb at the top of md file (#3394)

acb7273

Signed-off-by: meg miranda <mmiranda@nvidia.com>


          [Megatron-FSDP] Add fsdp_all_gather_in_start_param_sync option in DDP…

fdde15e

… Config (#3095)


          ci: Update release workflow to include changelog and publish docs (#3472

77f22f2

)

Signed-off-by: Charlie Truong <chtruong@nvidia.com>


          ci(fix): Weekly GPT tests (#3443)

1666b45

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Remove environments (#3462)

0d0943c

Signed-off-by: oliver könig <okoenig@nvidia.com>


          update HF tokenizer defaults (#3440)

d07f16c

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          PTQ changes for upcoming QAD (#3124)

1d694c2

Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com>
Co-authored-by: Jenny Chen <jennifchen@nvidia.com>


          ci: Bump preflight to detect our svc (#3494)

655cc8e

Signed-off-by: oliver könig <okoenig@nvidia.com>


          build: Drop Python 3.10 support and pip install one-logger (#3485)

7f35af4

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Bump pre-flight for Bot SSO (#3497)

2d06cc9

Signed-off-by: oliver könig <okoenig@nvidia.com>


          Update copy-pr-bot.yaml [skip ci]

a781f3c


          Revert "build: Drop Python 3.10 support and pip install one-logger (#… (

50ebe8e

#3500)


          Fix chunked prefill edge cases (#3404)

c191ae8

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>


          ci: Enable MBridge downstream testing via PR (#3483)

9f611b7

Signed-off-by: oliver könig <okoenig@nvidia.com>


          ci: Remove gitlab docs build job and set LTS integration and function…

7b6e226

…al tests to allow failure (#3349)

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet