DOC: List of TODOs for the documentation

[BenjaminBossan]

Feature request

This issue serves to list areas in which the documentation can be improved. It can be extended in the future if we find more gaps.

Contributions are welcome, if you want to pick up one of the open topics, let us know beforehand so that we can avoid duplicate work.

• Conceptual guides:
  • Multitask prompt tuning (🎉 Add Multitask Prompt Tuning #400)
  • LoHa (Add implementation of LyCORIS LoHa (FedPara-like adapter) for SD&SDXL models #956)
  • LoKr (Add implementation of LyCORIS LoKr (KronA-like adapter) for SD&SDXL models #978)
  • AdaLoRA
• Low level API: Activating multiple adapters (support multiple ranks and alphas for LoRA #873)
• Quantization: Having a chapter on quantization would be useful (how can it help, what is needed, how to use it, which tuners support it) - assigned to @stevhliu (Quantization docs #1220)
  • QLoRA
  • LoftQ: LoRA initialization for quantized models (LoftQ: Add LoftQ method integrated into LoRA. Add example code for LoftQ usage. #1150) - done (DOC: Initialization options for LoRA #1218) but could be cross-referenced in quantization section
• Docstrings for each important, user facing method/function - done (DOC: Update & improve docstrings and type annotations for common methods and classes #1201)

Motivation

Better docs, happier users.

Your contribution

Possible.

[aldakata]

Wouldn't this PR in /transformers close Quantization docs issue?

[BenjaminBossan]

Thanks for the pointer. I think it would be good to link there, but I'd also like to see a PEFT-specific section on quantization in the PEFT docs.

[dame-cell]

Hi,

I'm interested in contributing to the project and would like to write documentations but only for quantization for now.

I've outlined an initial structure for the documentation, covering topics such as:

• Introduction to Quantization
• Benefits of Quantization
• How to Use Quantization in the Project
• Supported Tuners for Quantization
• Examples and Best Practices

I'd love to get your thoughts on this, and if there are specific guidelines or preferences for documentation, please let me know. I'm looking forward to working with the community to enhance the project's documentation.

[BenjaminBossan]

Thanks @dame-cell for offering your help. I think when it comes to your points 1 & 2, we can keep it short and reference existing sources on that, as those are not PEFT-specific. Apart from that, your suggestion looks good.

[dame-cell]

hey @BenjaminBossan just got done writing it , I've just submitted a pull request hopefully that's enough if you need more please tell me , I'll make sure to make it better

[younesbelkada]

@BenjaminBossan seems everything has been addressed, what about closing the issue?

[BenjaminBossan]

I'd still like to add (or have contributions for) LoHa, LoKr, OFT in the conceptual guides. Edit: See #1269.

[BenjaminBossan]

Done. Thanks so much to everyone involved, especially @stevhliu.