Mcv binary cache

[maryamtahhan]

Enable vllm binary cache support for MCV

[maryamtahhan]

TODO - add torch_inductor dir

[maryamtahhan]

No precache

(EngineCore_DP0 pid=22) INFO 02-23 01:35:37 [backends.py:812] Using cache directory: /root/.cache/vllm/torch_compile_cache/8d0a361fbc/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=22) INFO 02-23 01:35:37 [backends.py:872] Dynamo bytecode transform time: 28.30 s
(EngineCore_DP0 pid=22) [rank0]:W0223 01:35:45.613000 22 torch/_inductor/utils.py:1613] Not enough SMs to use max_autotune_gemm mode
(EngineCore_DP0 pid=22) INFO 02-23 01:35:55 [backends.py:302] Cache the graph of compile range (1, 2048) for later use
(EngineCore_DP0 pid=22) INFO 02-23 01:36:01 [backends.py:319] Compiling a graph for compile range (1, 2048) takes 18.20 s
(EngineCore_DP0 pid=22) INFO 02-23 01:36:01 [monitor.py:34] torch.compile takes 46.50 s in total

with pre-cache:

(EngineCore_DP0 pid=22) INFO 02-23 03:12:47 [backends.py:812] Using cache directory: /root/.cache/vllm/torch_compile_cache/8d0a361fbc/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=22) INFO 02-23 03:12:47 [backends.py:872] Dynamo bytecode transform time: 7.85 s
(EngineCore_DP0 pid=22) INFO 02-23 03:12:54 [backends.py:267] Directly load the compiled graph(s) for compile range (1, 2048) from the cache, took 1.273 s
(EngineCore_DP0 pid=22) INFO 02-23 03:12:54 [monitor.py:34] torch.compile takes 9.12 s in total