Partial torch compile only, consider updating pytorch BUG!

[JavadZohani]

ComfyUI-GGUF: Partial torch compile only, consider updating pytorch

I even tested the nightly version but it didn't fix it.

Unfortunately, this problem still exists, even in the version
https://github.com/cardenluo/ComfyUI-Apt_Preset.git
in this gguf module
. It seems that gguf has completely crashed.

[leriilx]

Yesterday did some tests and worked like a charm, but after iteration #4, I just get random color noise as video result.
Updates Comfy, updated dependencies, plugins, restarted GPU (Win+CTRL+B), restarted PC and still nothing. Didn't change any setting at all from seed

[JamesMowery]

I'm also seeing this on Linux. I use comfy-cli to manage my install and update everything. How badly does this impact performance?

[StrongerXi]

Hmm, that's strange, any pytorch version > 2.8 should fix this.

For the record, I think fixing it speeds up Compile with GGUF Qwen Image significantly, like 1.5-2x depending on the output image size (smaller image gives bigger speedup).

[StrongerXi]

@leriilx could you try this? thu-ml/SageAttention#162 (comment)

And what are you guys' pytorch version, from pip show torch?

[JamesMowery]

@StrongerXi I'm showing Torch 2.7.1

/mnt/big/AI/applications/ComfyUI master
❯ uv pip show torch
Name: torch
Version: 2.7.1
Location: /mnt/big/AI/applications/ComfyUI/.venv/lib/python3.12/site-packages
Requires: filelock, fsspec, jinja2, networkx, nvidia-cublas-cu12, nvidia-cuda-cupti-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12, nvidia-cufft-cu12, nvidia-cufile-cu12, nvidia-curand-cu12, nvidia-cusolver-cu12, nvidia-cusparse-cu12, nvidia-cusparselt-cu12, nvidia-nccl-cu12, nvidia-nvjitlink-cu12, nvidia-nvtx-cu12, setuptools, sympy, triton, typing-extensions
Required-by: accelerate, groundingdino-py, kornia, peft, spandrel, timm, torchaudio, torchsde, torchvision, transparent-background

Is there a recommended way to upgrade it without breaking comfy-cli (honestly surprised that comfy update doesn't upgrade it... if it's supposed to be upgraded)? I have other apps I use that use Pytorch 2.9 I believe, so I shouldn't have any issues running a higher version. (I'm running an RTX 4090.)

[Mr-Hazem-Eng-Artist]

I am wondering about this too !, I have Torch2.7.1 cu128, & many other stuff in my Python libraries are also compiled for Torch271cu128 , will "upgrading to Torch 2.8 or 2.9" break other python compatiblities ?? is it safer for me to rollback/downgrade ComfyUi-gguf ?? if so, which version is better? Help !

[NeoAnthropocene]

I just realized that I've stepped into the club :)

pytorch version: 2.7.1+cu128

Do you have any working solution guys?

[Mr-Hazem-Eng-Artist]

I just realized that I've stepped into the club :)

pytorch version: 2.7.1+cu128

Do you have any working solution guys?

No luck yet, I tried rolling back both gguf & comfyui gguf but still didn't solve this .... honestly I am not ready to make a move to Torch 2.8 or 2.9 "I am a humble digital artist with very humble experience about Python's bonanza!" since Qwen & WAN2.2 came out & I am living i na Python nightmare & each day 2 day Libs/ backward compatibility issues.

until the moment, for "us who have this warning & we own Torch2.7.1cu128 or older" the impact on performance is not that big, so , I guess we can live with this & still continue using all gguf nodes features, until a fix comes out from the community or the devs here.

[NeoAnthropocene]

I just realized that I've stepped into the club :)
pytorch version: 2.7.1+cu128
Do you have any working solution guys?

No luck yet, I tried rolling back both gguf & comfyui gguf but still didn't solve this .... honestly I am not ready to make a move to Torch 2.8 or 2.9 "I am a humble digital artist with very humble experience about Python's bonanza!" since Qwen & WAN2.2 came out & I am living i na Python nightmare & each day 2 day Libs/ backward compatibility issues.

until the moment, for "us who have this warning & we own Torch2.7.1cu128 or older" the impact on performance is not that big, so , I guess we can live with this & still continue using all gguf nodes features, until a fix comes out from the community or the devs here.

Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)

[Mr-Hazem-Eng-Artist]

Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)

LOL I totally get it, me too I kind of spent a hell of a time with losing my nerves, trying to reach the best compatible things altogether for Comfy's python Libs. I finally reached this moment lately for PyTorch 2.7.1cu128 Python 3.12.10 with a success of 99% (just few things that I can't get them to install or to work)... I AM NOT Ready for another ( Here we goooo again ) when I need to have all these features upgraded with newer PyTorch with all the other (Attention)s out there that really screwed my poor butts trying to install all of them together LOL.

[NeoAnthropocene]

Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)

LOL I totally get it, me too I kind of spent a hell of a time with losing my nerves, trying to reach the best compatible things altogether for Comfy's python Libs. I finally reached this moment lately for PyTorch 2.7.1cu128 Python 3.12.10 with a success of 99% (just few things that I can't get them to install or to work)... I AM NOT Ready for another ( Here we goooo again ) when I need to have all these features upgraded with newer PyTorch with all the other (Attention)s out there that really screwed my poor butts trying to install all of them together LOL.

The most stable version for me, where all the custom nodes work, is the same one you're using. PyTorch 2.7.1cu128 and Python 3.12.

Since we're now experienced in these matters, I guess we won't easily do anything that would break venv :)

[diveren2010]

Another user, excact same thing. Pytorch 2.7.1 . Macos.

[hwprinz]

C'mon people it's so easy...
there is a post, that it's expected to have min. torch2.8
but: the Nightly versions of 2.8, when this was programmed, were that time older than the meanwhile most actual 2.7.1

i haven't had any issue, leaving as is, at first glance...
but: just edit ops.py and change the version to '2.7'
for me its working

    if not chained_hasattr(torch, "compiler.disable"):
        logging.info("ComfyUI-GGUF: Torch too old for torch.compile - bypassing")
        return dummy_decorator # torch too old
    elif version.parse(torch.__version__) >= version.parse("2.7"):
        logging.info("ComfyUI-GGUF: Allowing full torch compile")
        return dummy_decorator # torch compile works
    if chained_hasattr(torch, "_dynamo.config.nontraceable_tensor_subclasses"):
        logging.info("ComfyUI-GGUF: Allowing full torch compile (nightly)")
        return dummy_decorator # torch compile works, nightly before 2.8 release
    else:
        logging.info("ComfyUI-GGUF: Partial torch compile only, consider updating pytorch")
        return torch.compiler.disable

or:

• make a backup....
  and just simply upgrade

python.exe -s -m pip install --upgrade torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu128

• Done

ho harm, no issues (for me)
and i did it meanwhile several times

Disclaimer:
for any accelerators, you will need to upgrade them
pip install -U xformers
pip install -U "triton-windows<=3.4"
for sageattention: go for a 2.2.0 release and use the appropriate URL from https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows , as like for py3.12
pip install https://github.com/woct0rdho/SageAttention/releases/download/v2.2.0-windows/sageattention-2.2.0+cu128torch2.8.0-cp312-cp312-win_amd64.whl

for flash-attention, i recommend 2.8.2, from https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10
pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.4.10/flash_attn-2.8.2+cu128torch2.8-cp312-cp312-win_amd64.whl

regards, Hans

[Mr-Hazem-Eng-Artist]

@hwprinz Thanks for the detailed workaround,, when I am ready as am busy nowadays, I will try the upgrade process in the steps you mentioned, (important to have backups , just in case, but this idea on its own means that we may not be sure that all will work fine ! anyways, I will try out the upgrade with the steps you mentioned & my choice would be based on Torch2.8 cu128 (i still feel things may not be all OK yet for Cuda12.9), OR simply as you said I will just Edit the ops.py file, Thanks again :)

Edit:
Your instructions worked fine so far, I followed exactly everystep & with exact commands & I am now having Torch 2.8 Cuda12.8 & gguf says (ComfyUI-GGUF: Allowing full torch compile) ✅

Just one change: I am still using Triton 3.2 & Triton-windows 3.2 , because I tried both 3.3 post 19 & 3.4 post 20 , both gave me issues with ComfyUi Whisper for Transcribe Audio effects (( Cannot set attribute 'src' directly. Use '_unsafe_update_src()' and manually clear ` .... )) ...etc
I could fix it by reverting back to Triton 3.2 with Sage Attn 2.2 . Otherthan that all seems to work fine so far until I discover something.

[hwprinz]

thanks

btw, triton-windows3.4 would have been 'only' for cuda129....
as far as the compatibility matriox goes..
but good to know to be able to go back to 3.20
regards..