Add KappaTuneSelector: condition-number-based automatic LoRA target selection

[oswaldoludwig]

What this PR does

This PR adds lightweight target selection tooling, a new utility that lets users automatically select the best LoRA target_modules before creating a LoraConfig based on the lowest condition number per tensor, the metric used in KappaTune (https://arxiv.org/abs/2506.16289).

New API

from peft import LoraConfig, get_peft_model
from peft.utils.target_selection import KappaTuneSelector, find_kappa_target_modules

model = AutoModelForCausalLM.from_pretrained(...)   # your base model

# Option 1: class (full control)
selector = KappaTuneSelector(model)
optimal_targets = selector.get_best_targets(top_p=0.2)   # or num_modules=50, threshold=...

# Option 2: one-liner
optimal_targets = find_kappa_target_modules(model, top_p=0.2)

config = LoraConfig(
    target_modules=optimal_targets,
    r=16,
    lora_alpha=32,
    task_type="CAUSAL_LM",
)
peft_model = get_peft_model(model, config)

[githubnemo]

Thanks for the PR!

I think the interface is fine, KappaTuneSelector as a class to compute once and be able to experiment with different selections is good, having a short-cut one-liner is good as well.

There are only a few things missing:

• Let's add unit tests, e.g. in tests/test_target_selection.py
• Let's add an example, if possible re-creating the results from the paper
• Let's add documentation (entry in _toctree.yml under Utilities and a package reference file in package_reference/target_selection)

Did you test this with other PEFT methods like MiSS or SHiRA to see if it has a similar effect? I wonder if this method generalizes to other methods as well!

[githubnemo]

Let's document top_p, num_modules and threshold explicitly.

[oswaldoludwig]

I hadn't noticed that comment before. Check out the new comments in target_selection.py.

[oswaldoludwig]

I hope all reviewer feedback has been addressed (docstrings, tests, 4-bit support, processor note in the docs, and the experiment`) and the PR is ready for final review & merge @githubnemo

[githubnemo]

Let's write a user-friendly documentation that explains the usage, default parameters and impact of parameter choice. Recommendations and examples are good.