Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions README_hfspace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: GuideFlow3D
emoji: 🤗
colorFrom: yellow
colorTo: blue
app_file: demos/run_gradio_demo.py
sdk: gradio
sdk_version: 6.0.1
pinned: false
license: apache-2.0
python_version: "3.12"
short_description: A HF Space that demonstrates all use-cases for GuideFlow3D
---

Note: If you're looking to get a working version of the Hugging Face Space, then please use [this link](https://huggingface.co/spaces/suvadityamuk/GuideFlow3D) to fetch files with all assets in place as well.

<p align="center">
<h2 align="center"> GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer </h2>
<p align="center">
<a href="https://sayands.github.io/">Sayan Deb Sarkar</a><sup> 1 </sup>
.
<a href="https://vevenom.github.io/">Sinisa Stekovic</a><sup> 2 </sup>
.
<a href="https://vincentlepetit.github.io/">Vincent Lepetit</a><sup> 2 </sup>
.
<a href="https://ir0.github.io/">Iro Armeni</a><sup>1</sup>
</p>
<p align="center"> <strong>Neural Information Processing Systems (NeurIPS) 2025</strong></p>
<p align="center">
<sup> 1 </sup>Stanford University · <sup> 2 </sup>ENPC, IP Paris
</p>
<h3 align="center">

[![arXiv](https://img.shields.io/badge/arXiv-blue?logo=arxiv&color=%23B31B1B)](https://arxiv.org/abs/2510.16136)
[![ProjectPage](https://img.shields.io/badge/Project_Page-GuideFlow3D-blue)](https://sayands.github.io/guideflow3d)
[![License](https://img.shields.io/badge/License-Apache--2.0-929292)](https://www.apache.org/licenses/LICENSE-2.0)
<div align="center"></div>
</p>

<p align="center">
<a href="">
<img src="https://github.com/sayands/guideflow3d/blob/main/assets/guideflow3d_teaser.gif" width="100%">
</a>
</p>

<h5 align="left">
<em>TL;DR:</em> 3D appearance transfer pipeline robust to strong geometric variations between objects.
</h5>

## 📃 Abstract

Transferring appearance to 3D assets using different representations of the appearance object - such as images or text - has garnered interest due to its wide range of applications in industries like gaming, augmented reality, and digital content creation. However, state-of-the-art methods still fail when the geometry between the input and appearance objects is significantly different. A straightforward approach is to directly apply a 3D generative model, but we show that this ultimately fails to produce appealing results. Instead, we propose a principled approach inspired by universal guidance. Given a pretrained rectified flow model conditioned on image or text, our training-free method interacts with the sampling process by periodically adding guidance. This guidance can be modeled as a differentiable loss function, and we experiment with two different types of guidance including part-aware losses for appearance and self-similarity. Our experiments show that our approach successfully transfers texture and geometric details to the input 3D asset, outperforming baselines both qualitatively and quantitatively. We also show that traditional metrics are not suitable for evaluating the task due to their inability of focusing on local details and comparing dissimilar inputs, in absence of ground truth data. We thus evaluate appearance transfer quality with a GPT-based system objectively ranking outputs, ensuring robust and human-like assessment, as further confirmed by our user study. Beyond showcased scenarios, our method is general and could be extended to different types of diffusion models and guidance functions.

# :newspaper: News

- [2025-09] GuideFlow3D is accepted to **NeurIPS 2025** 🔥 See you in San Diego!

## 🚧 Code Release

⏳ Code and data will be released by the end of November! Stay tuned for updates.

## 📧 Contact
If you have any questions regarding this project, please use the github issue tracker or contact Sayan Deb Sarkar (sdsarkar@stanford.edu).

# :page_facing_up: Citation

```bibtex
@inproceedings{sayandsarkar_2025_guideflow3d,
author = {Deb Sarkar, Sayan and Stekovic, Sinisa and Lepetit, Vincent and Armeni, Iro},
title = {GuideFlow3D: Optimization-Guided Rectified Flow For 3D Appearance Transfer},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2025},
}
```
Empty file added demos/__init__.py
Empty file.
66 changes: 66 additions & 0 deletions demos/build_wheels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/bin/bash

# Create a directory to store wheels
mkdir -p ./wheels

# Update system packages
apt-get update -y
apt-get install -y xvfb libx11-6 libgl1 libxrender1

# 1. Basic Dependencies
# We use 'pip wheel' to build/download wheels instead of installing
pip wheel --wheel-dir=./wheels \
torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 \
--index-url https://download.pytorch.org/whl/cu124

pip wheel --wheel-dir=./wheels \
pyvirtualdisplay \
pillow imageio imageio-ffmpeg tqdm easydict opencv-python-headless \
scipy ninja rembg onnxruntime trimesh open3d xatlas pyvista \
pymeshfix igraph transformers tensorview psutil \
lightning==2.2 h5py yacs scikit-image loguru boto3 \
mesh2sdf tetgen==0.6.4 pymeshlab plyfile einops libigl \
polyscope potpourri3d simple_parsing arrgh vtk numpy==1.26.4

# 2. Git Repositories
# pip wheel handles git urls perfectly
pip wheel --wheel-dir=./wheels \
git+https://github.com/EasternJournalist/utils3d.git@9a4eb15e4021b67b12c460c7057d642626897ec8

# 3. Extensions with Custom Build Steps (nvdiffrast, diffoctreerast, mip-splatting)
# These often require cloning first if they have submodules or complex setups

# nvdiffrast
mkdir -p /tmp/extensions
if [ ! -d "/tmp/extensions/nvdiffrast" ]; then
git clone https://github.com/NVlabs/nvdiffrast.git /tmp/extensions/nvdiffrast
fi
pip wheel --wheel-dir=./wheels /tmp/extensions/nvdiffrast

# diffoctreerast
if [ ! -d "/tmp/extensions/diffoctreerast" ]; then
git clone --recurse-submodules https://github.com/JeffreyXiang/diffoctreerast.git /tmp/extensions/diffoctreerast
fi
pip wheel --wheel-dir=./wheels /tmp/extensions/diffoctreerast

# mip-splatting (diff-gaussian-rasterization)
if [ ! -d "/tmp/extensions/mip-splatting" ]; then
git clone https://github.com/autonomousvision/mip-splatting.git /tmp/extensions/mip-splatting
fi
pip wheel --wheel-dir=./wheels /tmp/extensions/mip-splatting/submodules/diff-gaussian-rasterization/

# 4. Pre-built Wheels (Kaolin, torch-scatter, spconv)
# These are already wheels, so we just download them to the folder
pip download --dest ./wheels \
kaolin==0.16.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.5.0_cu124.html

pip download --dest ./wheels \
spconv-cu124

pip download --dest ./wheels \
torch-scatter -f https://data.pyg.org/whl/torch-2.5.0+cu124.html

# 5. Python-PyCG
pip wheel --wheel-dir=./wheels 'python-pycg[all]'

echo "All wheels built in ./wheels"
Loading