Any way to optimize for 24gb and run on windows

[AdamGman]

So yeah is it possible to save VRAM switching out FLUX control net for SDXL turbo or something lighter, or is the tiling just always too much to handle? I was trying to compile for windows before I noticed the 48gb limit lol. Then I was wondering if this same idea could be integrated into a Comfy UI node with more optimized steps. I'm probably oversimplifying but who know

[SoftologyPro]

Even without optimization for 24 GB (would be awesome if possible) the main roadblock for Windows right now is that your code uses forkprocess. If you guys could add Windows compatible code there we could at least try it on 48 GB GPUs on Windows.

[paulengstler]

@AdamGman Thanks for the question! We've tried different image generation models but we found that there is a stark difference between them in terms of the visual quality. Flux has been the best model by quite a large margin. Replacing the model should be very straightforward, so please do feel free to run your own experiments.

@SoftologyPro One way to circumvent this requirement is by disabling the parallelisation in the pipeline (--parallel=False, and, if needed, removing the parallelisation code fully). The result is virtually identical (if not for the seeds).

[SoftologyPro]

@SoftologyPro One way to circumvent this requirement is by disabling the parallelisation in the pipeline (--parallel=False, and, if needed, removing the parallelisation code fully). The result is virtually identical (if not for the seeds).

Adding --parallel=False gives the same error about fork process. Can you guys add the required code so it works correctly when passed in?

[Clubbable]

Does it have to be Nvidia GPU? I have a mac studio with 64GB unified RAM which can i believe set to 48GB VRAM, is that fine as well?

[paulengstler]

@SoftologyPro Could I trouble you to comment out line 735 in the pipeline (https://github.com/paulengstler/syncity/blob/main/run_pipeline.py#L735) and see if the error persists? If so, could you post the error you are seeing here?

@Clubbable I'm afraid there are some dependencies in TRELLIS that require CUDA extensions to be compiled. As CUDA code does not run on Apple silicon (at least to my knowledge), an NVIDIA GPU is a hard requirement.

[CodeGameDev29]

Thanks @paulengstler I know for TRELLIS and FLUX (at least for sure for FLUX), there are net quantized versions Would net quantized versions being used for FLUX and TRELLIS work? This way to reduce NVIDIA gpu ram usage?

[SoftologyPro]

@SoftologyPro Could I trouble you to comment out line 735 in the pipeline (https://github.com/paulengstler/syncity/blob/main/run_pipeline.py#L735) and see if the error persists? If so, could you post the error you are seeing here?

That gets past the forkprocess error, but now I get this

{'pos': (0, 0), 'state': <States.ASSIGNED: '📋'>, 'task_id': 0, 'gpu_id': 0}
[SPARSE] Backend: spconv, Attention: xformers
Traceback (most recent call last):
  File "D:\Tests\SynCity\syncity\run_pipeline.py", line 865, in <module>
    fire.Fire(main)
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\fire\core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\fire\core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\fire\core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "D:\Tests\SynCity\syncity\run_pipeline.py", line 815, in main
    queue_available_jobs()
  File "D:\Tests\SynCity\syncity\run_pipeline.py", line 810, in queue_available_jobs
    res = worker(prefix, tile, gradio_url, blender_path, gpu_queue, generated_grid, first_tile_path, tile_mq, task_id, config, seed, verbose=True)
  File "D:\Tests\SynCity\syncity\run_pipeline.py", line 527, in worker
    from trellis.pipelines import TrellisImageTo3DPipeline
  File "D:\Tests\SynCity\syncity\trellis\__init__.py", line 3, in <module>
    from . import pipelines
  File "D:\Tests\SynCity\syncity\trellis\pipelines\__init__.py", line 2, in <module>
    from .trellis_image_to_3d import TrellisImageTo3DPipeline
  File "D:\Tests\SynCity\syncity\trellis\pipelines\trellis_image_to_3d.py", line 15, in <module>
    from ..representations import Gaussian, Strivec, MeshExtractResult
  File "D:\Tests\SynCity\syncity\trellis\representations\__init__.py", line 4, in <module>
    from .mesh import MeshExtractResult
  File "D:\Tests\SynCity\syncity\trellis\representations\mesh\__init__.py", line 1, in <module>
    from .cube2mesh import SparseFeatures2Mesh, MeshExtractResult
  File "D:\Tests\SynCity\syncity\trellis\representations\mesh\cube2mesh.py", line 5, in <module>
    from .flexicubes.flexicubes import FlexiCubes
  File "D:\Tests\SynCity\syncity\trellis\representations\mesh\flexicubes\flexicubes.py", line 11, in <module>
    from kaolin.utils.testing import check_tensor
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\__init__.py", line 1, in <module>
    from . import io
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\io\__init__.py", line 1, in <module>
    from . import dataset
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\io\dataset.py", line 31, in <module>
    from  ..utils.testing import contained_torch_equal
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\utils\__init__.py", line 1, in <module>
    from . import testing
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\utils\testing.py", line 29, in <module>
    import kaolin.ops.random as random
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\ops\__init__.py", line 1, in <module>
    from . import batch
  File "D:\Tests\SynCity\syncity\venv\lib\site-packages\kaolin\ops\batch.py", line 17, in <module>
    from kaolin import _C
ImportError: DLL load failed while importing _C: The specified procedure could not be found.

pip list shows kaolin 0.17.0

[paulengstler]

@CodeGameDev29 That is a great point. I haven't tried any quantized versions of models personally (and you'd might want to replace the inpainting model with FLUX.1 Fill). However, I can imagine this being a viable approach. Unfortunately, I currently don't have the capacity to work on this. Nonetheless, I would welcome any pull requests!

@SoftologyPro I'm afraid that's an issue that I can't help debug as I do not have a Windows computer at hand. I would suggest reinstalling kaolin based on the instructions on their website. So this would be pip install --upgrade --force-reinstall kaolin==0.17.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.4.0_cu118.html or possibly pip install --upgrade --force-reinstall kaolin==0.17.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.4.0_cu121.html. If neither work, may I point you to the kaolin repository to open an issue there? Thanks! I'm sorry I can't be of further assistance.

[JOY0430STAR]

Can I use multiple graphics cards smaller than 48G to run this project? For example, I have a server with six 3080 cards

[paulengstler]

@JOY0430STAR Thanks for the question. It doesn't work out of the box, but you might be able to distribute parts of the models onto separate GPUs (see https://www.reddit.com/r/StableDiffusion/comments/1el79h3/flux_can_be_run_on_a_multigpu_configuration/). I haven't tested this approach but there's a decent chance it might work!