SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Model

This is the official implementation of SIR-DIFFt: Sparse Image Sets Restoration with Multi-View Diffusion Model (CVPR 2025).

🔥 Update | 🔧 Installation | ⚡ Test | 🧩 Dataset Curation | 💻 Train

Update 🔥🔥🔥

• Release the paper of Sir-Diff
• Release the Test Code
• Release the Dataset Curation Details
• Release the Train Code

Installation 🔧

conda create -n sirdiff python=3.10
conda activate sirdiff
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

Inference ⚡

accelerate launch --config_file 1gpu.yaml distributed_eval.py --dataset_type hypersim --checkpoint_path /projects/bcdm/ymao3/ConsistentDepth/checkpoint/marigold-v1-0 --data_path /scratch/bcdm/ymao3/ml-hypersim/ --metric_save_dir base.txt --denoising_step 2 --using_samples 100 --output_dir /projects/bcdm/ymao3/ConsistentDepth

Train 💻

Single GPU Training

python train_code/train_sirdiff.py

Check config/train_sirdiff.yaml for deatiled setting, like SR/Deblur, weights store position.

Multi GPU Training (Default: 2GPU).

accelerate launch --config_file config/accelerate_config.json --main_process_port 28315 train_code/train_sirdiff.py

If you want to use more GPUs, check config/accelerate_config.yaml and the package accelerate.

HardWare Recommended: GPU slightly more than 24 GB is ideal for the training.