-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Open
Description
Problem Description
Currently, DVC provides excellent experiment tracking with `dvc exp run`, `dvc exp show`, and `dvc exp apply`. However, there is no built-in way to merge two experiment branches together, similar to how Git allows merging branches.
When working on ML projects, teams often want to:
- Combine hyperparameters from multiple experiments into one
- Merge metrics from baseline and variant experiments
- Create "ensemble" experiments from individual runs
Currently, users must manually copy values or write custom scripts to combine experiment results.
Proposed Solution
Add a new command: `dvc exp merge`
Example Usage:
```bash
Merge experiment exp-1 into exp-2, keeping exp-2 params where they differ
$ dvc exp merge exp-1 into exp-2
Create a new experiment combining metrics from both
$ dvc exp merge exp-1 exp-2 --name exp-ensemble --strategy combine
Merge with explicit parameter override
$ dvc exp merge exp-baseline into exp-new --force-params learning_rate
```
Use Cases
- Hyperparameter Search: Combine best learning rates from multiple search experiments
- Ensemble Creation: Merge metrics from different model variants
- A/B Testing: Combine control and treatment experiment results
- Baseline Preservation: Merge new insights into baseline without losing original
Implementation Suggestions
- Use a strategy pattern: `keep-target`, `keep-source`, `combine`, `average`
- Support `.dvc` file merging for params and metrics
- Integrate with existing `dvc exp apply` workflow
- Consider CLI flags: `--name`, `--strategy`, `--force-params`
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels