Skip to content

Conversation

@WenqingLan1
Copy link
Contributor

This pull request adds support for NVBench-based GPU micro-benchmarks to SuperBench.

  • Integrated the NVBench submodule
  • Implemented two benchmarks
    • nvbench-sleep-kernel
    • nvbench-kernel-launch
  • updated documentation and added example scripts

Example config:

version: v0.12
superbench:
  enable:
  # nvbench benchmarks
  - nvbench-sleep-kernel:single
  - nvbench-sleep-kernel:list
  - nvbench-sleep-kernel:range
  - nvbench-sleep-kernel:range-step
  - nvbench-kernel-launch
  var:
    default_local_mode: &default_local_mode
      modes:
      - name: local
        proc_num: 4
        prefix: CUDA_VISIBLE_DEVICES={proc_rank}
        parallel: yes
  benchmarks:
    nvbench-sleep-kernel:single:
      <<: *default_local_mode
      timeout: 300
      parameters:
        duration_us: "50"                   # Single value format
        timeout: 30
    nvbench-sleep-kernel:list:
      <<: *default_local_mode
      timeout: 300
      parameters:
        duration_us: "[25,50,75]"         # List format - no spaces after commas
        timeout: 30
    nvbench-sleep-kernel:range:
      <<: *default_local_mode
      timeout: 300
      parameters:
        duration_us: "[0:5]"           # Range format
        timeout: 30
    nvbench-sleep-kernel:range-step:
      <<: *default_local_mode
      timeout: 300
      parameters:
        duration_us: "[0:50:10]"         # Range with step format
        timeout: 30
    nvbench-kernel-launch:
      <<: *default_local_mode
      timeout: 300

@WenqingLan1 WenqingLan1 requested a review from a team as a code owner October 9, 2025 23:12
@WenqingLan1 WenqingLan1 added benchmarks SuperBench Benchmarks micro-benchmarks Micro Benchmark Test for SuperBench Benchmarks labels Oct 9, 2025
@codecov
Copy link

codecov bot commented Oct 10, 2025

Codecov Report

❌ Patch coverage is 89.11917% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.78%. Comparing base (c99380b) to head (2877feb).

Files with missing lines Patch % Lines
...rbench/benchmarks/micro_benchmarks/nvbench_base.py 80.39% 20 Missing ⚠️
...enchmarks/micro_benchmarks/nvbench_sleep_kernel.py 98.07% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #750      +/-   ##
==========================================
+ Coverage   85.69%   85.78%   +0.08%     
==========================================
  Files         102      105       +3     
  Lines        7699     7892     +193     
==========================================
+ Hits         6598     6770     +172     
- Misses       1101     1122      +21     
Flag Coverage Δ
cpu-python3.10-unit-test 71.41% <88.94%> (+0.43%) ⬆️
cpu-python3.7-unit-test 70.88% <89.11%> (+0.46%) ⬆️
cuda-unit-test 83.74% <88.94%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@guoshzhao guoshzhao self-assigned this Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmarks SuperBench Benchmarks micro-benchmarks Micro Benchmark Test for SuperBench Benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants