Finetunning stable difussion inpainting checkpoint

[leshhh-aneee]

My current goal is to train on the VITON-HD fashion dataset, where the clothing images are the target, and the mask image being the source. I am also planning to further train on the "stabilityai/stable-diffusion-2-inpainting" ckpt. How can i go about training this ckpt, as the original script is to use tool add sd21, then followed by sam train sd21, however I am met with channel mismatch as the original model has 4 channels while this inpainting ckpt has 9 channels. Previously i have trained on the original "512-base-ema.ckpt" with the target image being DressCode clothing items, and the source images being sketches images of the clothing items. However when it comes to using this as the base model inside of the sam2vlpart_edit.py, the inpainting area becomes a blank as seen in the image below.

Any assistance on how to further train this "stabilityai/stable-diffusion-2-inpainting" ckpt would be greatly appreciated.