Draft
Conversation
* Add `remote_decode` to `remote_utils` * test dependency * test dependency * dependency * dependency * dependency * docstrings * changes * make style * apply * revert, add new options * Apply style fixes * deprecate base64, headers not needed * address comments * add license header * init test_remote_decode * more * more test * more test * skeleton for xl, flux * more test * flux test * flux packed * no scaling * -save * hunyuanvideo test * Apply style fixes * init docs * Update src/diffusers/utils/remote_utils.py Co-authored-by: Sayak Paul <[email protected]> * comments * Apply style fixes * comments * hybrid_inference/vae_decode * fix * tip? * tip * api reference autodoc * install tip --------- Co-authored-by: sayakpaul <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
fix-copies went uncaught it seems.
* fix-copies went uncaught it seems. * remove more unneeded encode_prompt() tests * Revert "fix-copies went uncaught it seems." This reverts commit eefb302. * empty
…eneration model (huggingface#10626) * Update EasyAnimate V5.1 * Add docs && add tests && Fix comments problems in transformer3d and vae * delete comments and remove useless import * delete process * Update EXAMPLE_DOC_STRING * rename transformer file * make fix-copies * make style * refactor pt. 1 * update toctree.yml * add model tests * Update layer_norm for norm_added_q and norm_added_k in Attention * Fix processor problem * refactor vae * Fix problem in comments * refactor tiling; remove einops dependency * fix docs path * make fix-copies * Update src/diffusers/pipelines/easyanimate/pipeline_easyanimate_control.py * update _toctree.yml * fix test * update * update * update * make fix-copies * fix tests --------- Co-authored-by: Aryan <[email protected]> Co-authored-by: Aryan <[email protected]> Co-authored-by: YiYi Xu <[email protected]> Co-authored-by: Dhruv Nair <[email protected]>
* Fix SD2.X clip single file load projection_dim Infer projection_dim from the checkpoint before loading from pretrained, override any incorrect hub config. Hub configuration for SD2.X specifies projection_dim=512 which is incorrect for SD2.X checkpoints loaded from civitai and similar. Exception was previously thrown upon attempting to load_model_dict_into_meta for SD2.X single file checkpoints. Such LDM models usually require projection_dim=1024 * convert_open_clip_checkpoint use hidden_size for text_proj_dim * convert_open_clip_checkpoint, revert checkpoint[text_proj_key].shape[1] -> [0] values are identical --------- Co-authored-by: Teriks <[email protected]> Co-authored-by: Dhruv Nair <[email protected]>
* Update pipeline_animatediff.py * Update pipeline_animatediff_controlnet.py * Update pipeline_animatediff_sparsectrl.py * Update pipeline_animatediff_video2video.py * Update pipeline_animatediff_video2video_controlnet.py --------- Co-authored-by: Dhruv Nair <[email protected]>
* Add example of Ip-Adapter-Callback. * Add image links from HF Hub.
* Update ip_adapter.py * Update ip_adapter.py * Update ip_adapter.py * Update ip_adapter.py * Update ip_adapter.py * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: hlky <[email protected]>
* initial comit * fix empty cache * fix one more * fix style * update device functions * update * update * Update src/diffusers/utils/testing_utils.py Co-authored-by: hlky <[email protected]> * Update src/diffusers/utils/testing_utils.py Co-authored-by: hlky <[email protected]> * Update src/diffusers/utils/testing_utils.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/controlnet/test_controlnet.py Co-authored-by: hlky <[email protected]> * Update src/diffusers/utils/testing_utils.py Co-authored-by: hlky <[email protected]> * Update src/diffusers/utils/testing_utils.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/controlnet/test_controlnet.py Co-authored-by: hlky <[email protected]> * with gc.collect * update * make style * check_torch_dependencies * add mps empty cache * add changes * bug fix * enable on xpu * update more cases * revert * revert back * Update test_stable_diffusion_xl.py * Update tests/pipelines/stable_diffusion/test_stable_diffusion.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/stable_diffusion/test_stable_diffusion.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/stable_diffusion/test_stable_diffusion_img2img.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/stable_diffusion/test_stable_diffusion_img2img.py Co-authored-by: hlky <[email protected]> * Update tests/pipelines/stable_diffusion/test_stable_diffusion_img2img.py Co-authored-by: hlky <[email protected]> * Apply suggestions from code review Co-authored-by: hlky <[email protected]> * add test marker --------- Co-authored-by: hlky <[email protected]>
* Update evaluation.md * Update docs/source/en/conceptual/evaluation.md Co-authored-by: Steven Liu <[email protected]> --------- Co-authored-by: Steven Liu <[email protected]>
* feat: support non-diffusers lumina2 LoRAs. * revert ipynb changes (but I don't know why this is required☹️ ) * empty --------- Co-authored-by: Dhruv Nair <[email protected]> Co-authored-by: YiYi Xu <[email protected]>
…e#10927) * [Quantization] support pass MappingType for TorchAoConfig * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…tization_config.py. (huggingface#10961) Update quantization_config.py
* update * refactor image-to-video pipeline * update * fix copied from * use FP32LayerNorm
) * Fix seed initialization to handle args.seed = 0 correctly * Apply style fixes --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…SDXL (huggingface#10951) * feat: add Mixture-of-Diffusers ControlNet Tile upscaler Pipeline for SDXL * make style make quality
* Update pipeline_cogview4.py * Use GLM instead of T5 in doc
* fix t5 training bug * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
use style bot GH action from hfh Co-authored-by: Sayak Paul <[email protected]>
…hs` is passed in a distributed training env (huggingface#10973) * updated train_dreambooth_lora to fix the LR schedulers for `num_train_epochs` in distributed training env * fixed formatting * remove trailing newlines * fixed style error
Co-authored-by: Sayak Paul <[email protected]>
* update * update * addressed PR comments * update --------- Co-authored-by: YiYi Xu <[email protected]>
…ce#11369) * Add stochastic sampling to FlowMatchEulerDiscreteScheduler This PR adds stochastic sampling to FlowMatchEulerDiscreteScheduler based on Lightricks/LTX-Video@b1aeddd ltx_video/schedulers/rf.py * Apply style fixes * Use config value directly * Apply style fixes * Swap order * Update src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py Co-authored-by: YiYi Xu <[email protected]> * Update src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py Co-authored-by: YiYi Xu <[email protected]> --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: YiYi Xu <[email protected]>
…e#11281) * initial commit * initial commit * initial commit * initial commit * initial commit * initial commit * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Bagheera <[email protected]> * move prompt embeds, pooled embeds outside * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: hlky <[email protected]> * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: hlky <[email protected]> * fix import * fix import and tokenizer 4, text encoder 4 loading * te * prompt embeds * fix naming * shapes * initial commit to add HiDreamImageLoraLoaderMixin * fix init * add tests * loader * fix model input * add code example to readme * fix default max length of text encoders * prints * nullify training cond in unpatchify for temp fix to incompatible shaping of transformer output during training * smol fix * unpatchify * unpatchify * fix validation * flip pred and loss * fix shift!!! * revert unpatchify changes (for now) * smol fix * Apply style fixes * workaround moe training * workaround moe training * remove prints * to reduce some memory, keep vae in `weight_dtype` same as we have for flux (as it's the same vae) https://github.com/huggingface/diffusers/blob/bbd0c161b55ba2234304f1e6325832dd69c60565/examples/dreambooth/train_dreambooth_lora_flux.py#L1207 * refactor to align with HiDream refactor * refactor to align with HiDream refactor * refactor to align with HiDream refactor * add support for cpu offloading of text encoders * Apply style fixes * adjust lr and rank for train example * fix copies * Apply style fixes * update README * update README * update README * fix license * keep prompt2,3,4 as None in validation * remove reverse ode comment * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Sayak Paul <[email protected]> * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Sayak Paul <[email protected]> * vae offload change * fix text encoder offloading * Apply style fixes * cleaner to_kwargs * fix module name in copied from * add requirements * fix offloading * fix offloading * fix offloading * update transformers version in reqs * try AutoTokenizer * try AutoTokenizer * Apply style fixes * empty commit * Delete tests/lora/test_lora_layers_hidream.py * change tokenizer_4 to load with AutoTokenizer as well * make text_encoder_four and tokenizer_four configurable * save model card * save model card * revert T5 * fix test * remove non diffusers lumina2 conversion --------- Co-authored-by: Bagheera <[email protected]> Co-authored-by: hlky <[email protected]> Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Small change requirements_sana.txt to requirements_hidream.txt
…e#11375) * fix * add tests * add message check
… for image resizing (huggingface#11395)
* Kolors additional pipelines, community contrib --------- Co-authored-by: Teriks <[email protected]> Co-authored-by: Linoy Tsaban <[email protected]>
* 1. add pre-computation of prompt embeddings when custom prompts are used as well 2. save model card even if model is not pushed to hub 3. remove scheduler initialization from code example - not necessary anymore (it's now if the base model's config) 4. add skip_final_inference - to allow to run with validation, but skip the final loading of the pipeline with the lora weights to reduce memory reqs * pre encode validation prompt as well * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Sayak Paul <[email protected]> * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Sayak Paul <[email protected]> * Update examples/dreambooth/train_dreambooth_lora_hidream.py Co-authored-by: Sayak Paul <[email protected]> * pre encode validation prompt as well * Apply style fixes * empty commit * change default trained modules * empty commit * address comments + change encoding of validation prompt (before it was only pre-encoded if custom prompts are provided, but should be pre-encoded either way) * Apply style fixes * empty commit * fix validation_embeddings definition * fix final inference condition * fix pipeline deletion in last inference * Apply style fixes * empty commit * layers * remove readme remarks on only pre-computing when instance prompt is provided and change example to 3d icons * smol fix * empty commit --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Fix Flux IP adapter argument in the example IP-Adapter example had a wrong argument. Fix `true_cfg` -> `true_cfg_scale`
…for resizing (huggingface#11421) * Set LANCZOS as default interpolation mode for resizing * [train_dreambooth_lora.py] Set LANCZOS as default interpolation mode for resizing
…s in pipelines during torch.compile() (huggingface#11085) * test for better torch.compile stuff. * fixes * recompilation and graph break. * clear compilation cache. * change to modeling level test. * allow running compilation tests during nightlies.
* enable group_offload cases and quanto cases on XPU Signed-off-by: YAO Matrix <[email protected]> * use backend APIs Signed-off-by: Yao Matrix <[email protected]> * fix style Signed-off-by: Yao Matrix <[email protected]> --------- Signed-off-by: YAO Matrix <[email protected]> Signed-off-by: Yao Matrix <[email protected]>
* enable test_layerwise_casting_memory cases on XPU Signed-off-by: Yao Matrix <[email protected]> * fix style Signed-off-by: Yao Matrix <[email protected]> --------- Signed-off-by: Yao Matrix <[email protected]>
fix import.
…follow up (huggingface#11426) * Update train_text_to_image.py * update
…ipts follow up (huggingface#11427) * Update train_text_to_image_lora.py * update_train_text_to_image_lora
* enable gguf test cases on XPU Signed-off-by: YAO Matrix <[email protected]> * make SD35LargeGGUFSingleFileTests::test_pipeline_inference pas Signed-off-by: root <[email protected]> * make FluxControlLoRAGGUFTests::test_lora_loading pass Signed-off-by: Yao Matrix <[email protected]> * polish code Signed-off-by: Yao Matrix <[email protected]> * Apply style fixes --------- Signed-off-by: YAO Matrix <[email protected]> Signed-off-by: root <[email protected]> Signed-off-by: Yao Matrix <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
remove unnecessary pipeline moving to cpu in validation Co-authored-by: Sayak Paul <[email protected]>
* Fixing missing provider options argument * Adding if else for provider options * Apply suggestions from code review Co-authored-by: YiYi Xu <[email protected]> * Apply style fixes * Update src/diffusers/pipelines/onnx_utils.py Co-authored-by: YiYi Xu <[email protected]> * Update src/diffusers/pipelines/onnx_utils.py Co-authored-by: YiYi Xu <[email protected]> --------- Co-authored-by: Uros Petkovic <[email protected]> Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: YiYi Xu <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…lNet training (huggingface#11449) Set LANCZOS as the default interpolation for image resizing
…gingface#11425) raise warning instead of error
Signed-off-by: Yao Matrix <[email protected]>
Signed-off-by: Yao Matrix <[email protected]>
* enable unidiffuser cases on XPU Signed-off-by: Yao Matrix <[email protected]> * fix a typo Signed-off-by: Yao Matrix <[email protected]> * fix style Signed-off-by: Yao Matrix <[email protected]> --------- Signed-off-by: Yao Matrix <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds SEGA for FLUX, thereby enable users to guide the diffusion process by applying editing prompts.