feat(pd): support gradient accumulation#4920
Merged
njzjz merged 6 commits intodeepmodeling:develfrom Aug 27, 2025
Merged
Conversation
HydrogenSulfate
changed the title
Contributor
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughAdd gradient accumulation controlled by a new Changes
Sequence Diagram(s)sequenceDiagram
participant Trainer
participant Optimizer
participant Scheduler
participant AllReduce as AllReduce (world_size>1)
rect rgba(200,230,255,0.25)
Note right of Trainer: Repeat for each micro-step (1..acc_freq)
Trainer->>Trainer: forward()
Trainer->>Trainer: backward() -- accumulate grads
end
rect rgba(220,255,200,0.25)
Note right of Trainer: On accumulation boundary\n((_step_id+1) % acc_freq == 0)
Trainer->>AllReduce: fused_allreduce_gradients() [if world_size>1]
Trainer->>Trainer: gradient clipping() [if gradient_max_norm>0]
Trainer->>Optimizer: step()
Trainer->>Optimizer: clear_grad(set_to_zero=False)
Trainer->>Scheduler: step()
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
deepmd/pd/train/training.py (2)
835-856: Do not clear gradients during validation; it breaks accumulation. Use no_grad instead.Calling self.optimizer.clear_grad() here zeroes partially accumulated gradients whenever display logging runs (e.g., on step 1), corrupting the accumulation window. Wrap validation forward in paddle.no_grad() and remove the clear_grad.
- for ii in range(valid_numb_batch): - self.optimizer.clear_grad() - input_dict, label_dict, _ = self.get_data( - is_train=False, task_key=_task_key - ) - if input_dict == {}: - # no validation data - return {} - _, loss, more_loss = self.wrapper( - **input_dict, - cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), - label=label_dict, - task_key=_task_key, - ) + for ii in range(valid_numb_batch): + with paddle.no_grad(): + input_dict, label_dict, _ = self.get_data( + is_train=False, task_key=_task_key + ) + if input_dict == {}: + # no validation data + return {} + _, loss, more_loss = self.wrapper( + **input_dict, + cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), + label=label_dict, + task_key=_task_key, + )
888-901: Do not clear gradients when logging other-task training metrics; it breaks accumulation. Use no_grad.Same issue as above. Clearing grads here will wipe partially accumulated grads in the current task when disp_training fires. Remove the clear and use no_grad.
- if _key != task_key: - self.optimizer.clear_grad() - input_dict, label_dict, _ = self.get_data( - is_train=True, task_key=_key - ) - _, loss, more_loss = self.wrapper( - **input_dict, - cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), - label=label_dict, - task_key=_key, - ) + if _key != task_key: + with paddle.no_grad(): + input_dict, label_dict, _ = self.get_data( + is_train=True, task_key=_key + ) + _, loss, more_loss = self.wrapper( + **input_dict, + cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), + label=label_dict, + task_key=_key, + )
🧹 Nitpick comments (3)
deepmd/pd/train/training.py (3)
136-138: Validate and document acc_freq inputAdd a guard and a short docstring comment to ensure acc_freq is a positive integer. This avoids silent misconfigurations (e.g., 0 or negatives) and clarifies semantics.
self.num_steps = training_params["numb_steps"] self.acc_freq: int = training_params.get( - "acc_freq", 1 + "acc_freq", 1 ) # gradient accumulation steps +assert isinstance(self.acc_freq, int) and self.acc_freq >= 1, "training.acc_freq must be an integer >= 1"
794-801: Optional: move gradient clipping into optimizer via grad_clip to reduce per-step python overheadPaddle supports optimizer-level gradient clipping (e.g., grad_clip=paddle.nn.ClipGradByGlobalNorm). This avoids per-step Python calls and makes behavior uniform.
Example:
grad_clip = paddle.nn.ClipGradByGlobalNorm(clip_norm=self.gradient_max_norm) self.optimizer = paddle.optimizer.Adam( learning_rate=self.scheduler, parameters=self.wrapper.parameters(), grad_clip=grad_clip, )
751-759: Consider using pref_lr tied to update count when accumulatingWhen acc_freq > 1, the “logical” step for LR scheduling is the optimizer update, not each micro-step. You’re calling scheduler.step() only on update (good). For consistency also consider computing pref_lr based on the current scheduler.get_lr() (update count), not _lr.value(_step_id). Today it’s benign but can diverge for nontrivial LR policies.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
deepmd/pd/train/training.py(2 hunks)source/tests/pd/test_training.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
deepmd/pd/train/training.py (2)
deepmd/pd/utils/utils.py (1)
nvprof_context(357-366)deepmd/pt/train/training.py (1)
step(715-1083)
source/tests/pd/test_training.py (2)
source/tests/pd/model/test_permutation.py (9)
setUp(437-440)setUp(445-448)setUp(452-455)setUp(459-462)setUp(467-472)setUp(477-480)setUp(485-490)setUp(495-498)setUp(503-507)deepmd/pd/utils/env.py (1)
enable_prim(124-204)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (22)
- GitHub Check: Test C++ (true)
- GitHub Check: Test C++ (false)
- GitHub Check: Test Python (6, 3.12)
- GitHub Check: Build C++ (clang, clang)
- GitHub Check: Test Python (2, 3.12)
- GitHub Check: Test Python (2, 3.9)
- GitHub Check: Test Python (1, 3.12)
- GitHub Check: Test Python (3, 3.9)
- GitHub Check: Build C++ (cpu, cpu)
- GitHub Check: Build C++ (cuda120, cuda)
- GitHub Check: Build C++ (rocm, rocm)
- GitHub Check: Analyze (python)
- GitHub Check: Build C library (2.14, >=2.5.0,<2.15, libdeepmd_c_cu11.tar.gz)
- GitHub Check: Build C++ (cuda, cuda)
- GitHub Check: Analyze (c-cpp)
- GitHub Check: Build C library (2.18, libdeepmd_c.tar.gz)
- GitHub Check: Build wheels for cp311-win_amd64
- GitHub Check: Build wheels for cp311-macosx_arm64
- GitHub Check: Build wheels for cp311-macosx_x86_64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Build wheels for cp310-manylinux_aarch64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
🔇 Additional comments (3)
deepmd/pd/train/training.py (2)
969-974: Metric tensor to scalar conversion looks goodSwitching to .item() for loss and more_loss clarifies tensorboard logging types.
790-801: Please confirmfused_allreduce_gradients’ scaling behavior before clipping/steppingThe use of
hpu.fused_allreduce_gradientsmay sum or average gradients across ranks. To maintain single-process semantics—and match typical DDP-style averaging—you need to ensure gradients are divided byworld_sizeif they are summed:• Location:
deepmd/pd/train/training.py, around lines 790–801
• Action: Inspectfleet.utils.hybrid_parallel_util.hpu.fused_allreduce_gradientsto determine whether it performs an all-reduce with SUM or AVG
• If it sums, insert immediately after all-reduce and before gradient clipping/optimizer step:for p in self.wrapper.parameters(): if p.grad is not None: p._set_grad(p.grad / self.world_size)This guarantees consistent gradient scaling across single- and multi-process runs.
source/tests/pd/test_training.py (1)
141-159: Good baseline test setupRe-using the existing water dataset and enabling prim matches the Paddle path; this provides a solid baseline.
HydrogenSulfate
force-pushed
the
pd_grad_acc
branch
from
ad2aeb3 to
ed992ad
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
deepmd/pd/train/training.py (2)
828-858: Do not clear training gradients during validation logging; wrap validation in no_gradCalling self.optimizer.clear_grad() here wipes accumulated training gradients before the accumulation boundary, breaking gradient accumulation correctness. Use no_grad for validation instead and avoid touching grads.
- def log_loss_valid(_task_key="Default"): + def log_loss_valid(_task_key="Default"): single_results = {} sum_natoms = 0 if not self.multi_task: valid_numb_batch = self.valid_numb_batch else: valid_numb_batch = self.valid_numb_batch[_task_key] for ii in range(valid_numb_batch): - self.optimizer.clear_grad() input_dict, label_dict, _ = self.get_data( is_train=False, task_key=_task_key ) if input_dict == {}: # no validation data return {} - _, loss, more_loss = self.wrapper( - **input_dict, - cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), - label=label_dict, - task_key=_task_key, - ) + with paddle.no_grad(): + _, loss, more_loss = self.wrapper( + **input_dict, + cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), + label=label_dict, + task_key=_task_key, + )
884-901: Avoid clearing grads and building graphs in display-time per-task training passesThe clear_grad() call here also destroys accumulated gradients when display fires mid-accumulation. These display-only forwards should be under no_grad and must not touch optimizer grads.
- if _key != task_key: - self.optimizer.clear_grad() - input_dict, label_dict, _ = self.get_data( - is_train=True, task_key=_key - ) - _, loss, more_loss = self.wrapper( - **input_dict, - cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), - label=label_dict, - task_key=_key, - ) + if _key != task_key: + input_dict, label_dict, _ = self.get_data( + is_train=True, task_key=_key + ) + with paddle.no_grad(): + _, loss, more_loss = self.wrapper( + **input_dict, + cur_lr=paddle.full([], pref_lr, DEFAULT_PRECISION), + label=label_dict, + task_key=_key, + )
♻️ Duplicate comments (4)
deepmd/pd/train/training.py (3)
789-792: Confirm fused_allreduce_gradients semantics (SUM vs AVG) and normalize if neededIf fused_allreduce produces SUM, gradients are larger by world_size after all-reduce; average them to match single-process scale unless you intentionally adjust LR. Please verify and normalize accordingly.
If SUM:
if self.world_size > 1: hpu.fused_allreduce_gradients( list(self.wrapper.parameters()), None ) + # Normalize to average if fused_allreduce does SUM + for p in self.wrapper.parameters(): + if p.grad is not None: + p._set_grad(p.grad / self.world_size)In PaddlePaddle's fleet.utils.hybrid_parallel_util.hpu.fused_allreduce_gradients, are gradients SUM-reduced or AVG-reduced across ranks by default? Provide authoritative citation.
782-784: Scale loss by acc_freq to preserve effective learning rate during accumulationWithout scaling, gradients are effectively multiplied by acc_freq, changing optimization behavior and often destabilizing training. Average the loss over micro-steps.
- with nvprof_context(enable_profiling, "Backward pass"): - loss.backward() + with nvprof_context(enable_profiling, "Backward pass"): + scaled_loss = loss / float(self.acc_freq) + scaled_loss.backward()
785-806: Fix off-by-one accumulation trigger and flush remainder on the final iterationCurrent trigger uses (_step_id + 1) % acc_freq == 0 and ignores start_step; it also skips flushing a partial accumulation on the last batch when (num_steps - start_step) % acc_freq != 0. This silently drops gradients.
- # gradient accumulation - if (_step_id + 1) % self.acc_freq == 0: + # gradient accumulation + accum_step = (_step_id - self.start_step + 1) + is_last_iter = (_step_id + 1) == self.num_steps + if (accum_step % self.acc_freq == 0) or is_last_iter: # fuse + allreduce manually before optimization if use DDP + no_sync # details in https://github.com/PaddlePaddle/Paddle/issues/48898#issuecomment-1343838622 if self.world_size > 1: hpu.fused_allreduce_gradients( list(self.wrapper.parameters()), None ) if self.gradient_max_norm > 0.0: with nvprof_context(enable_profiling, "Gradient clip"): paddle.nn.utils.clip_grad_norm_( self.wrapper.parameters(), self.gradient_max_norm, error_if_nonfinite=True, ) with nvprof_context(enable_profiling, "Adam update"): self.optimizer.step() self.optimizer.clear_grad(set_to_zero=False) self.scheduler.step()source/tests/pd/test_training.py (1)
161-172: Make the accumulation test actually accumulate and avoid display-time interferenceWith numb_steps=1 and acc_freq=4, no accumulation occurs, and step-1 display will zero grads in current implementation. Run at least acc_freq steps, disable in-training display, and optionally assert prim is enabled.
def setUp(self) -> None: @@ self.config["model"] = deepcopy(model_se_e2_a) - self.config["training"]["numb_steps"] = 1 + # Use acc_freq steps so we actually accumulate before an optimizer step + self.config["training"]["numb_steps"] = 4 self.config["training"]["save_freq"] = 1 self.config["training"]["acc_freq"] = 4 + # Avoid display-time eval that may touch grads during accumulation + self.config["training"]["disp_training"] = False enable_prim(True) + # Optional sanity check + # assert paddle.framework.core._is_eager_prim_enabled(), \ + # "Eager prim should be enabled for gradient accumulation tests"
🧹 Nitpick comments (2)
deepmd/pd/train/training.py (1)
811-816: Optional: only run heavy display/validation at accumulation boundariesTo avoid extra compute and any future risk of interfering with in-flight accumulation, consider logging only when you step the optimizer or on the very first batch.
Example gate:
- if self.display_in_training and ( - display_step_id % self.disp_freq == 0 or display_step_id == 1 - ): + if self.display_in_training and ( + display_step_id % self.disp_freq == 0 or display_step_id == 1 + ) and ( + ((_step_id + 1 - self.start_step) % self.acc_freq == 0) or ((_step_id + 1) == self.num_steps) + ):source/tests/pd/test_training.py (1)
161-172: Optional: add a remainder-flush test caseTo cover the final-step flush logic, consider setting numb_steps to acc_freq+1 (e.g., 5 with acc_freq=4) and asserting one optimizer update for the remainder. I can provide a concrete test if desired.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
deepmd/pd/train/training.py(2 hunks)source/tests/pd/test_training.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
source/tests/pd/test_training.py (1)
deepmd/pd/utils/env.py (1)
enable_prim(124-204)
deepmd/pd/train/training.py (3)
deepmd/pd/utils/utils.py (1)
nvprof_context(357-366)deepmd/pt/train/training.py (1)
step(715-1083)source/tests/pd/model/test_model.py (1)
step(414-416)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)
- GitHub Check: Build wheels for cp310-manylinux_aarch64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Build wheels for cp311-win_amd64
- GitHub Check: Build wheels for cp311-macosx_arm64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Build wheels for cp311-macosx_x86_64
- GitHub Check: Test C++ (true)
- GitHub Check: Test C++ (false)
- GitHub Check: Test Python (6, 3.12)
- GitHub Check: Test Python (6, 3.9)
- GitHub Check: Test Python (1, 3.12)
- GitHub Check: Test Python (4, 3.12)
- GitHub Check: Test Python (1, 3.9)
- GitHub Check: Test Python (5, 3.12)
- GitHub Check: Test Python (4, 3.9)
- GitHub Check: Test Python (2, 3.9)
- GitHub Check: Test Python (3, 3.9)
- GitHub Check: Test Python (5, 3.9)
- GitHub Check: Test Python (3, 3.12)
- GitHub Check: Test Python (2, 3.12)
- GitHub Check: Build C++ (cuda120, cuda)
- GitHub Check: Build C++ (cuda, cuda)
- GitHub Check: Build C++ (rocm, rocm)
- GitHub Check: Build C library (2.18, libdeepmd_c.tar.gz)
- GitHub Check: Build C library (2.14, >=2.5.0,<2.15, libdeepmd_c_cu11.tar.gz)
- GitHub Check: Analyze (c-cpp)
🔇 Additional comments (1)
source/tests/pd/test_training.py (1)
155-157: LGTM: tearDown added for TestEnergyModelSeAGood cleanup, consistent with the rest of the suite.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## devel #4920 +/- ##
==========================================
- Coverage 84.29% 84.29% -0.01%
==========================================
Files 703 703
Lines 68728 68731 +3
Branches 3572 3573 +1
==========================================
+ Hits 57936 57938 +2
+ Misses 9653 9652 -1
- Partials 1139 1141 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
njzjz
reviewed
Aug 26, 2025
Member
njzjz
left a comment
njzjz
left a comment
There was a problem hiding this comment.
I am wondering that if you don't add this argument to deepmd/utils/argcheck.py, an error will be thrown during the argument check
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
deepmd/utils/argcheck.py (1)
3171-3171: Clarify acc_freq semantics and constraints in docsExplicitly state meaning, constraint (>=1), and effective batch size to avoid user confusion.
- doc_acc_freq = "The accumulation steps for the gradients." + doc_acc_freq = ( + "Number of mini-batches to accumulate gradients before applying optimizer " + "and scheduler updates. Must be >= 1. Effective batch size = batch_size * acc_freq." + )
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
deepmd/utils/argcheck.py(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
- GitHub Check: Test Python (6, 3.12)
- GitHub Check: Test Python (5, 3.9)
- GitHub Check: Test Python (5, 3.12)
- GitHub Check: Test Python (3, 3.12)
- GitHub Check: Test Python (2, 3.12)
- GitHub Check: Test Python (6, 3.9)
- GitHub Check: Test Python (4, 3.9)
- GitHub Check: Test Python (4, 3.12)
- GitHub Check: Test Python (1, 3.9)
- GitHub Check: Test Python (3, 3.9)
- GitHub Check: Test Python (2, 3.9)
- GitHub Check: Test Python (1, 3.12)
- GitHub Check: Build C library (2.18, libdeepmd_c.tar.gz)
- GitHub Check: Build C library (2.14, >=2.5.0,<2.15, libdeepmd_c_cu11.tar.gz)
- GitHub Check: Test C++ (true)
- GitHub Check: Test C++ (false)
- GitHub Check: Build wheels for cp311-macosx_arm64
- GitHub Check: Build wheels for cp310-manylinux_aarch64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Build wheels for cp311-win_amd64
- GitHub Check: Build wheels for cp311-macosx_x86_64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Build C++ (clang, clang)
- GitHub Check: Build C++ (cuda, cuda)
- GitHub Check: Build C++ (cpu, cpu)
- GitHub Check: Build C++ (rocm, rocm)
- GitHub Check: Build C++ (cuda120, cuda)
- GitHub Check: Analyze (c-cpp)
🔇 Additional comments (1)
deepmd/utils/argcheck.py (1)
43-43: Paddle-only doc flag addition looks goodConsistent with existing TF/PT flags; no issues.
njzjz
reviewed
Aug 27, 2025
njzjz
approved these changes
Aug 27, 2025
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds gradient accumulation support to the Paddle backend for DeePMD-kit. Gradient accumulation allows batching optimizer updates across multiple steps, which can improve memory efficiency and training stability.
Key changes:
- Adds
acc_freqconfiguration parameter to control gradient accumulation frequency - Modifies training loop to accumulate gradients and perform updates at specified intervals
- Adds test coverage for gradient accumulation functionality
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
deepmd/utils/argcheck.py |
Adds acc_freq argument definition with Paddle backend documentation |
deepmd/pd/train/training.py |
Implements gradient accumulation logic in training loop |
source/tests/pd/test_training.py |
Adds test case for gradient accumulation functionality |
ChiahsinChu
pushed a commit
to ChiahsinChu/deepmd-kit
that referenced
this pull request
Dec 17, 2025
support gradient accumulation for paddle backend. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Configurable gradient accumulation (acc_freq) that batches optimizer updates, optional gradient clipping, and multi‑GPU gradient sync to occur at the configured interval; acc_freq=1 preserves prior behavior. - **Documentation** - Added argument docs and a Paddle backend notice describing acc_freq. - **Tests** - Added tests exercising gradient accumulation and updated test cleanup. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
iProzd
added a commit
to iProzd/deepmd-kit
that referenced
this pull request
Mar 27, 2026
* feat(pt): support zbl finetune (#4849)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an option to control whether output statistics are computed or
loaded across atomic models.
* **Bug Fixes**
* More robust parameter transfer during fine‑tuning to handle renamed
branches and missing pretrained keys.
* **Refactor**
* Revised output-statistics workflow and refined per‑type output bias
application in composite models.
* **Tests**
* Simplified linear-model bias checks and added a ZBL finetuning test
path.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: anyangml <anyangpeng.ca@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(pt/pd): fix eta computation (#4886)
fix eta computation code
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved ETA accuracy in training/validation progress logs by adapting
calculations to recent step intervals, reducing misleading estimates
early in runs.
* Consistent behavior across both backends, providing more reliable
remaining-time estimates without changing any public interfaces.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* fix: get correct intensive property prediction when using virtual atoms (#4869)
When using virtual atoms, the property output of virtual atom is `0`.
- If predicting energy or other extensive properties, it works well,
that's because the virtual atom property `0` do not contribute to the
total energy or other extensive properties.
- However, if predicting intensive properties, there is some error. For
example, a frame has two real atoms and two virtual atoms, the atomic
property contribution is [2, 2, 0, 0](the atomic property of virtual
atoms are always 0), the final property should be `(2+2)/real_atoms =
2`, not be `(2+2)/total_atoms =1`.
This PR is used to solve this bug mentioned above.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Models now provide accessors to retrieve property names and their
fitting network; property fitting nets expose output definitions.
* **Bug Fixes**
* Intensive property reduction respects atom masks so padded/dummy atoms
are ignored, keeping results invariant to padding.
* **Tests**
* Added PyTorch, JAX, and core tests validating consistent behavior with
padded atoms.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(tf): fix compatibility with TF 2.20 (#4890)
Fix version finding in pip and CMake; pin TF to <2.20 on Windows; fix
TENSORFLOW_ROOT in the CI.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added compatibility with TensorFlow 2.20+ via runtime version
detection and generated version macros.
- Bug Fixes
- Clearer errors when a specified TensorFlow root is invalid.
- Improved version-parsing fallback for newer TensorFlow releases.
- Tightened Windows CPU wheel constraint to avoid incompatible versions.
- Chores
- Updated devcontainer scripts and CI workflows to more reliably locate
TensorFlow without importing it directly.
- Linked TensorFlow during version checks to ensure accurate detection.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>
Signed-off-by: Jinzhe Zeng <njzjz@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: relax `atol` and `rtol` value of padding atoms UT (#4892)
The UT of padding atoms(pytorch backend) sometimes fails like:
```
Mismatched elements: 1 / 2 (50%)
Max absolute difference among violations: 1.97471693e-08
Max relative difference among violations: 6.45619919e-07
ACTUAL: array([[-0.236542],
[ 0.030586]])
DESIRED: array([[-0.236542],
[ 0.030586]])
= 1 failed, 15442 passed, 4135 skipped, 97877 deselected, 224 warnings in 2825.25s (0:47:05) =
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Adjusted numerical comparison assertions to use both absolute and
relative tolerances in padding-related tests.
- Aligns checks between computed results and references, improving
resilience to minor floating-point variation.
- Reduces intermittent test failures across environments and dependency
versions.
- No impact on features, performance, or user workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* doc(pd): update paddle installation scripts and paddle related content in dpa3 document (#4887)
update paddle installation scripts and custom border op error message
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated installation guides to reference PaddlePaddle 3.1.1 for CUDA
12.6, CUDA 11.8, and CPU; added nightly pre-release install examples.
* Refined training docs wording and CINN note; added Paddle backend
guidance and explicit OP-install instructions in DPA3 docs.
* **Chores**
* Improved error messages when custom Paddle operators are unavailable,
adding clearer install instructions and links to documentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: HydrogenSulfate <490868991@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix(pt): fix CMake compatibility with PyTorch 2.8 (#4891)
Fix #4877.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- Bug Fixes
- Improved build compatibility with PyTorch 2.8+ on UNIX-like systems
(excluding macOS) by aligning the default ABI selection with PyTorch’s
behavior. This reduces potential linker/runtime issues when building
against newer PyTorch versions. Behavior on other platforms and with
older PyTorch remains unchanged. No runtime functionality changes for
end users.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* feat: add yaml input file support (#4894)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Training entrypoints now accept YAML configuration files in addition
to JSON, offering more flexibility when launching training.
* Unified configuration loading across frameworks for consistent
behavior (PyTorch, Paddle, TensorFlow).
* Backward compatible: existing JSON-based workflows continue to work
unchanged.
* **Tests**
* Added coverage to verify YAML input produces the expected training
output.
* Improved test cleanup to remove generated artifacts after execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* build(deps): bump actions/checkout from 4 to 5 (#4897)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to
5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/releases">actions/checkout's
releases</a>.</em></p>
<blockquote>
<h2>v5.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Update actions checkout to use node 24 by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li>
<li>Prepare v5.0.0 release by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2238">actions/checkout#2238</a></li>
</ul>
<h2>⚠️ Minimum Compatible Runner Version</h2>
<p><strong>v2.327.1</strong><br />
<a
href="https://github.com/actions/runner/releases/tag/v2.327.1">Release
Notes</a></p>
<p>Make sure your runner is updated to this version or newer to use this
release.</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v4...v5.0.0">https://github.com/actions/checkout/compare/v4...v5.0.0</a></p>
<h2>v4.3.0</h2>
<h2>What's Changed</h2>
<ul>
<li>docs: update README.md by <a
href="https://github.com/motss"><code>@motss</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li>
<li>Add internal repos for checking out multiple repositories by <a
href="https://github.com/mouismail"><code>@mouismail</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li>
<li>Documentation update - add recommended permissions to Readme by <a
href="https://github.com/benwells"><code>@benwells</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li>
<li>Adjust positioning of user email note and permissions heading by <a
href="https://github.com/joshmgross"><code>@joshmgross</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li>
<li>Update CODEOWNERS for actions by <a
href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li>
<li>Update package dependencies by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li>
<li>Prepare release v4.3.0 by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2237">actions/checkout#2237</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/motss"><code>@motss</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li>
<li><a href="https://github.com/mouismail"><code>@mouismail</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li>
<li><a href="https://github.com/benwells"><code>@benwells</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li>
<li><a href="https://github.com/nebuk89"><code>@nebuk89</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li>
<li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v4...v4.3.0">https://github.com/actions/checkout/compare/v4...v4.3.0</a></p>
<h2>v4.2.2</h2>
<h2>What's Changed</h2>
<ul>
<li><code>url-helper.ts</code> now leverages well-known environment
variables by <a href="https://github.com/jww3"><code>@jww3</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li>
<li>Expand unit test coverage for <code>isGhes</code> by <a
href="https://github.com/jww3"><code>@jww3</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v4.2.1...v4.2.2">https://github.com/actions/checkout/compare/v4.2.1...v4.2.2</a></p>
<h2>v4.2.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Check out other refs/* by commit if provided, fall back to ref by <a
href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Jcambass"><code>@Jcambass</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/1919">actions/checkout#1919</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v4.2.0...v4.2.1">https://github.com/actions/checkout/compare/v4.2.0...v4.2.1</a></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>V5.0.0</h2>
<ul>
<li>Update actions checkout to use node 24 by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li>
</ul>
<h2>V4.3.0</h2>
<ul>
<li>docs: update README.md by <a
href="https://github.com/motss"><code>@motss</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li>
<li>Add internal repos for checking out multiple repositories by <a
href="https://github.com/mouismail"><code>@mouismail</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li>
<li>Documentation update - add recommended permissions to Readme by <a
href="https://github.com/benwells"><code>@benwells</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li>
<li>Adjust positioning of user email note and permissions heading by <a
href="https://github.com/joshmgross"><code>@joshmgross</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li>
<li>Update CODEOWNERS for actions by <a
href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li>
<li>Update package dependencies by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li>
</ul>
<h2>v4.2.2</h2>
<ul>
<li><code>url-helper.ts</code> now leverages well-known environment
variables by <a href="https://github.com/jww3"><code>@jww3</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li>
<li>Expand unit test coverage for <code>isGhes</code> by <a
href="https://github.com/jww3"><code>@jww3</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li>
</ul>
<h2>v4.2.1</h2>
<ul>
<li>Check out other refs/* by commit if provided, fall back to ref by <a
href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li>
</ul>
<h2>v4.2.0</h2>
<ul>
<li>Add Ref and Commit outputs by <a
href="https://github.com/lucacome"><code>@lucacome</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1180">actions/checkout#1180</a></li>
<li>Dependency updates by <a
href="https://github.com/dependabot"><code>@dependabot</code></a>- <a
href="https://redirect.github.com/actions/checkout/pull/1777">actions/checkout#1777</a>,
<a
href="https://redirect.github.com/actions/checkout/pull/1872">actions/checkout#1872</a></li>
</ul>
<h2>v4.1.7</h2>
<ul>
<li>Bump the minor-npm-dependencies group across 1 directory with 4
updates by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1739">actions/checkout#1739</a></li>
<li>Bump actions/checkout from 3 to 4 by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1697">actions/checkout#1697</a></li>
<li>Check out other refs/* by commit by <a
href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1774">actions/checkout#1774</a></li>
<li>Pin actions/checkout's own workflows to a known, good, stable
version. by <a href="https://github.com/jww3"><code>@jww3</code></a> in
<a
href="https://redirect.github.com/actions/checkout/pull/1776">actions/checkout#1776</a></li>
</ul>
<h2>v4.1.6</h2>
<ul>
<li>Check platform to set archive extension appropriately by <a
href="https://github.com/cory-miller"><code>@cory-miller</code></a> in
<a
href="https://redirect.github.com/actions/checkout/pull/1732">actions/checkout#1732</a></li>
</ul>
<h2>v4.1.5</h2>
<ul>
<li>Update NPM dependencies by <a
href="https://github.com/cory-miller"><code>@cory-miller</code></a> in
<a
href="https://redirect.github.com/actions/checkout/pull/1703">actions/checkout#1703</a></li>
<li>Bump github/codeql-action from 2 to 3 by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1694">actions/checkout#1694</a></li>
<li>Bump actions/setup-node from 1 to 4 by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1696">actions/checkout#1696</a></li>
<li>Bump actions/upload-artifact from 2 to 4 by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1695">actions/checkout#1695</a></li>
<li>README: Suggest <code>user.email</code> to be
<code>41898282+github-actions[bot]@users.noreply.github.com</code> by <a
href="https://github.com/cory-miller"><code>@cory-miller</code></a> in
<a
href="https://redirect.github.com/actions/checkout/pull/1707">actions/checkout#1707</a></li>
</ul>
<h2>v4.1.4</h2>
<ul>
<li>Disable <code>extensions.worktreeConfig</code> when disabling
<code>sparse-checkout</code> by <a
href="https://github.com/jww3"><code>@jww3</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1692">actions/checkout#1692</a></li>
<li>Add dependabot config by <a
href="https://github.com/cory-miller"><code>@cory-miller</code></a> in
<a
href="https://redirect.github.com/actions/checkout/pull/1688">actions/checkout#1688</a></li>
<li>Bump the minor-actions-dependencies group with 2 updates by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1693">actions/checkout#1693</a></li>
<li>Bump word-wrap from 1.2.3 to 1.2.5 by <a
href="https://github.com/dependabot"><code>@dependabot</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1643">actions/checkout#1643</a></li>
</ul>
<h2>v4.1.3</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/checkout/commit/08c6903cd8c0fde910a37f88322edcfb5dd907a8"><code>08c6903</code></a>
Prepare v5.0.0 release (<a
href="https://redirect.github.com/actions/checkout/issues/2238">#2238</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/9f265659d3bb64ab1440b03b12f4d47a24320917"><code>9f26565</code></a>
Update actions checkout to use node 24 (<a
href="https://redirect.github.com/actions/checkout/issues/2226">#2226</a>)</li>
<li>See full diff in <a
href="https://github.com/actions/checkout/compare/v4...v5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate (#4898)
<!--pre-commit.ci start-->
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.12.8 →
v0.12.9](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.8...v0.12.9)
<!--pre-commit.ci end-->
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix(pt): add comm_dict for zbl, linear, dipole, dos, polar model to fix bugs mentioned in issue #4906 (#4908)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added optional support to pass a communication dictionary through
lower-level model computations across energy, dipole, DOS, polarization,
and related models. This enables advanced workflows while remaining
fully backward compatible.
- Refactor
- Standardized internal propagation of the communication dictionary
across sub-models to ensure consistent behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* docs: add comprehensive GitHub Copilot instructions and environment setup (#4911)
This PR adds comprehensive development support for GitHub Copilot agents
working in the DeePMD-kit codebase.
## What's included
**Comprehensive Copilot Instructions
(`.github/copilot-instructions.md`)**
- Complete build workflow with exact timing expectations (67s Python
build, 164s C++ build)
- Virtual environment setup and dependency installation for all backends
(TensorFlow, PyTorch, JAX, Paddle)
- **Optimized testing guidance**: Emphasizes single test execution
(~8-13 seconds) over full test suite (60+ minutes) for faster
development feedback
- Linting and formatting with ruff (1 second execution)
- Multiple validation scenarios for CLI, Python interface, and training
workflows
- Directory structure reference and key file locations
- Critical warnings with specific timeout recommendations to prevent
premature cancellation
- **Conventional commit specification**: Guidelines for commit messages
and PR titles following `type(scope): description` format
**Automated Environment Setup
(`.github/workflows/copilot-setup-steps.yml`)**
- Pre-configures Python environment using uv for fast dependency
management
- Installs TensorFlow CPU and PyTorch automatically
- Builds the DeePMD-kit package with all dependencies
- Sets up pre-commit hooks for code quality
- Validates installation to ensure environment readiness
**Development Efficiency Features**
- All commands tested and validated with accurate timing measurements
- Imperative tone throughout for clear action items
- Copy-paste ready validation scenarios
- Gitignore rules to prevent temporary test files from being committed
## Key improvements for Copilot agents
- **Faster iteration**: Single test recommendations instead of 60+
minute full test suites
- **Automated setup**: No manual environment configuration needed
- **Precise expectations**: Exact timing guidance prevents timeout
issues during builds
- **Multi-backend support**: Complete coverage of TensorFlow, PyTorch,
JAX, and Paddle workflows
- **Consistent commit standards**: Enforces conventional commit
specification for all changes
The instructions enable any GitHub Copilot agent to work effectively in
this codebase from a fresh clone with precise expectations for build
times, test execution, and validation workflows.
Fixes #4910.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(pt,pd): remove redundant tensor handling to eliminate tensor construction warnings (#4907)
This PR fixes deprecation warnings that occur when `torch.tensor()` or
`paddle.to_tensor()` is called on existing tensor objects:
**PyTorch warning:**
```
UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
```
**PaddlePaddle warning:**
```
UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach(), rather than paddle.to_tensor(sourceTensor).
```
## Root Cause
The warnings were being triggered in multiple locations:
1. **PyTorch**: Test cases were passing tensor objects directly to ASE
calculators, which internally convert them using `torch.tensor()`
2. **PaddlePaddle**: Similar issues in `eval_model` function and
`to_paddle_tensor` utility, plus a TypeError where `tensor.to()` method
was incorrectly using `place=` instead of `device=`
## Solution
**For PyTorch:**
- Modified test cases to convert tensor inputs to numpy arrays before
passing to ASE calculators
- Removed redundant tensor handling in `to_torch_tensor` utility
function since the non-numpy check already handles tensors by returning
them as-is
**For PaddlePaddle:**
- Added proper type checking in `eval_model` function to handle existing
tensors with `clone().detach()`
- Removed redundant tensor handling in `to_paddle_tensor` utility
function, applying the same optimization as PyTorch
- Fixed TypeError by changing `place=` to `device=` in all `tensor.to()`
method calls (PaddlePaddle's tensor `.to()` method expects `device=`
parameter, while `paddle.to_tensor()` correctly uses `place=`)
## Changes Made
1. **`source/tests/pt/test_calculator.py`**: Fixed `TestCalculator` and
`TestCalculatorWithFparamAparam` to convert PyTorch tensors to numpy
arrays before passing to ASE calculator
2. **`deepmd/pt/utils/utils.py`**: Removed redundant tensor-specific
handling in `to_torch_tensor` function
3. **`source/tests/pd/common.py`**: Updated `eval_model` function with
type checking for PaddlePaddle tensors and fixed `tensor.to()` method
calls to use `device=` instead of `place=`
4. **`deepmd/pd/utils/utils.py`**: Removed redundant tensor-specific
handling in `to_paddle_tensor` function for consistency with PyTorch
Both utility functions now use a simplified approach where the `if not
isinstance(xx, np.ndarray): return xx` check handles all non-numpy
inputs (including tensors) by returning them unchanged, eliminating the
need for separate tensor-specific code paths.
This change is backward compatible and maintains the same functionality
while eliminating both deprecation warnings and TypeErrors, improving
code consistency between PyTorch and PaddlePaddle backends.
Fixes #3790.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: Add eval-desc CLI command for descriptor evaluation with 3D output format (#4903)
This PR implements a new command-line interface for evaluating
descriptors using trained DeePMD models, addressing the feature request
for making the `eval_descriptor` function available from the command
line.
## Overview
The new `dp eval-desc` command allows users to generate descriptor
matrices from their models using a simple CLI interface, similar to the
existing `dp test` command.
## Usage
```bash
# Basic usage
dp eval-desc -m model.pb -s /path/to/system
# With custom output directory
dp eval-desc -m model.pth -s /path/to/system -o my_descriptors
# Using datafile with multiple systems
dp eval-desc -m model.pb -f systems_list.txt -o desc_output
# For multi-task models
dp eval-desc -m model.pth -s system_dir --head task_branch
```
## Output Format
Descriptors are saved as NumPy `.npy` files in 3D format (nframes,
natoms, ndesc) preserving the natural structure of the data with
separate dimensions for frames, atoms, and descriptor components. This
format maintains the original data organization and is suitable for
various analysis workflows.
## Implementation Details
The implementation follows the same architectural pattern as the
existing `dp test` command:
- **CLI Parser**: Added argument parser in `deepmd/main.py` with options
for model (`-m`), system (`-s`), datafile (`-f`), output (`-o`), and
model branch (`--head`)
- **Command Routing**: Integrated into the entrypoints system in
`deepmd/entrypoints/main.py`
- **Core Functionality**: New `eval_desc.py` module that uses
`DeepEval.eval_descriptor()` to generate descriptors and saves them as
`.npy` files in their natural 3D format
- **Documentation**: Updated user guide and API documentation with
output format details
- **Testing**: Comprehensive tests following the pattern of existing `dp
test` functionality
Fixes #4503.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/deepmodeling/deepmd-kit/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* build(deps): bump actions/upload-pages-artifact from 3 to 4 (#4918)
Bumps
[actions/upload-pages-artifact](https://github.com/actions/upload-pages-artifact)
from 3 to 4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/upload-pages-artifact/releases">actions/upload-pages-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Potentially breaking change: hidden files (specifically dotfiles)
will not be included in the artifact by <a
href="https://github.com/tsusdere"><code>@tsusdere</code></a> in <a
href="https://redirect.github.com/actions/upload-pages-artifact/pull/102">actions/upload-pages-artifact#102</a>
If you need to include dotfiles in your artifact: instead of using this
action, create your own artifact according to these requirements <a
href="https://github.com/actions/upload-pages-artifact?tab=readme-ov-file#artifact-validation">https://github.com/actions/upload-pages-artifact?tab=readme-ov-file#artifact-validation</a></li>
<li>Pin <code>actions/upload-artifact</code> to SHA by <a
href="https://github.com/heavymachinery"><code>@heavymachinery</code></a>
in <a
href="https://redirect.github.com/actions/upload-pages-artifact/pull/127">actions/upload-pages-artifact#127</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-pages-artifact/compare/v3.0.1...v4.0.0">https://github.com/actions/upload-pages-artifact/compare/v3.0.1...v4.0.0</a></p>
<h2>v3.0.1</h2>
<h1>Changelog</h1>
<ul>
<li>Group tar's output to prevent it from messing up action logs <a
href="https://github.com/SilverRainZ"><code>@SilverRainZ</code></a> (<a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/94">#94</a>)</li>
<li>Update README.md <a
href="https://github.com/uiolee"><code>@uiolee</code></a> (<a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/88">#88</a>)</li>
<li>Bump the non-breaking-changes group with 1 update <a
href="https://github.com/dependabot"><code>@dependabot</code></a> (<a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/92">#92</a>)</li>
<li>Update Dependabot config to group non-breaking changes <a
href="https://github.com/JamesMGreene"><code>@JamesMGreene</code></a>
(<a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/91">#91</a>)</li>
<li>Bump actions/checkout from 3 to 4 <a
href="https://github.com/dependabot"><code>@dependabot</code></a> (<a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/76">#76</a>)</li>
</ul>
<p>See details of <a
href="https://github.com/actions/upload-pages-artifact/compare/v3.0.0...v3.0.1">all
code changes</a> since previous release.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/7b1f4a764d45c48632c6b24a0339c27f5614fb0b"><code>7b1f4a7</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/127">#127</a>
from heavymachinery/pin-sha</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/4cc19c7d3f3e6c87c68366501382a03c8b1ba6db"><code>4cc19c7</code></a>
Pin <code>actions/upload-artifact</code> to SHA</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/2d163be3ddce01512f3eea7ac5b7023b5d643ce1"><code>2d163be</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/107">#107</a>
from KittyChiu/main</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/c70484322b1c476728dcd37fac23c4dea2a0c51a"><code>c704843</code></a>
fix: linted README</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/9605915f1d2fc79418cdce4d5fbe80511c457655"><code>9605915</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/106">#106</a>
from KittyChiu/kittychiu/update-readme-1</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/e59cdfe6d6b061aab8f0619e759cded914f3ab03"><code>e59cdfe</code></a>
Update README.md</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/a2d67043267d885050434d297d3dd3a3a14fd899"><code>a2d6704</code></a>
doc: updated usage section in readme</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/984864e7b70fb5cb764344dc9c4b5c087662ef50"><code>984864e</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/105">#105</a>
from actions/Jcambass-patch-1</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/45dc78884ca148c05eddcd8ac0a804d3365e9014"><code>45dc788</code></a>
Add workflow file for publishing releases to immutable action
package</li>
<li><a
href="https://github.com/actions/upload-pages-artifact/commit/efaad07812d4b9ad2e8667cd46426fdfb7c22e22"><code>efaad07</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-pages-artifact/issues/102">#102</a>
from actions/hidden-files</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/upload-pages-artifact/compare/v3...v4">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: Avoid setting pin_memory in tests (#4919)
Avoid specifying pin_memory for test DataLoaders to eliminate warnings
when no accelerator is available.
#4874
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Updated test configurations to rely on default memory pinning behavior
in data loading, improving compatibility across environments.
* Simplified test setup parameters to reduce potential flakiness and
align with framework defaults.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* [pre-commit.ci] pre-commit autoupdate (#4917)
<!--pre-commit.ci start-->
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.12.9 →
v0.12.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.9...v0.12.10)
<!--pre-commit.ci end-->
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* chore(CI): bump PyTorch from 2.7 to 2.8 (#4884)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded PyTorch to 2.8 across CPU and CUDA 12.x environments for
improved compatibility and stability.
* Updated development container to download the matching LibTorch 2.8
CPU bundle.
* Refreshed CI pipelines (build, test, analysis) to install and validate
against PyTorch 2.8.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Jinzhe Zeng <njzjz@qq.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* fix(pd): change numel function return type from int to size_t to prevent overflow (#4924)
The `numel` function in the Paddle backend was using `int` for computing
tensor element counts, which can overflow for large tensors. This fix
changes the return type and intermediate calculations to `size_t` to
handle larger tensor sizes safely.
## Problem
The original implementation multiplied tensor dimensions as `int`
values:
```cpp
int numel(const paddle_infer::Tensor& x) const {
// TODO: There might be a overflow problem here for multiply int numbers.
int ret = 1;
std::vector<int> x_shape = x.shape();
for (std::size_t i = 0, n = x_shape.size(); i < n; ++i) {
ret *= x_shape[i]; // Can overflow for large tensors
}
return ret;
}
```
For large tensors (e.g., shape `[50000, 50000, 10]` = 25 billion
elements), this causes integer overflow and returns negative values.
## Solution
- Changed return type from `int` to `size_t`
- Changed intermediate calculations to use `size_t` with explicit
casting
- Updated all calling sites to use `size_t` variables
- Removed the TODO comment since the overflow issue is now resolved
```cpp
size_t numel(const paddle_infer::Tensor& x) const {
size_t ret = 1;
std::vector<int> x_shape = x.shape();
for (std::size_t i = 0, n = x_shape.size(); i < n; ++i) {
ret *= static_cast<size_t>(x_shape[i]); // Safe from overflow
}
return ret;
}
```
The `size_t` type can handle up to 2^64 elements on 64-bit systems (vs
2^31 for `int`), making it appropriate for tensor element counts. This
change is backward compatible since `std::vector::resize()` and other
consumers already accept `size_t`.
Fixes #4551.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/deepmodeling/deepmd-kit/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
* feat(pd): support gradient accumulation (#4920)
support gradient accumulation for paddle backend.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Configurable gradient accumulation (acc_freq) that batches optimizer
updates, optional gradient clipping, and multi‑GPU gradient sync to
occur at the configured interval; acc_freq=1 preserves prior behavior.
- **Documentation**
- Added argument docs and a Paddle backend notice describing acc_freq.
- **Tests**
- Added tests exercising gradient accumulation and updated test cleanup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* feat(pt): add model branch alias (#4883)
Introduces model branch alias and info fields to model configuration,
adds utility functions for handling model branch dictionaries, and
updates related modules to use alias-based lookup and provide detailed
branch information. Enhances multi-task model usability and improves
logging of available model branches.
example:
```
dp --pt show 0415_compat_new.pt model-branch
[2025-08-14 10:05:54,246] DEEPMD WARNING To get the best performance, it is recommended to adjust the number of threads by setting the environment variables OMP_NUM_THREADS, DP_INTRA_OP_PARALLELISM_THREADS, and DP_INTER_OP_PARALLELISM_THREADS. See https://deepmd.rtfd.io/parallelism/ for more information.
[2025-08-14 10:05:59,122] DEEPMD INFO This is a multitask model
[2025-08-14 10:05:59,122] DEEPMD INFO Available model branches are ['Dai2023Alloy', 'Zhang2023Cathode', 'Gong2023Cluster', 'Yang2023ab', 'UniPero', 'Huang2021Deep-PBE', 'Liu2024Machine', 'Zhang2021Phase', 'Jinag2021Accurate', 'Chen2023Modeling', 'Wen2021Specialising', 'Wang2022Classical', 'Wang2022Tungsten', 'Wu2021Deep', 'Huang2021Deep-PBEsol', 'Transition1x', 'Wang2021Generalizable', 'Wu2021Accurate', 'MPTraj', 'Li2025APEX', 'Shi2024SSE', 'Tuo2023Hybrid', 'Unke2019PhysNet', 'Shi2024Electrolyte', 'ODAC23', 'Alex2D', 'OMAT24', 'SPICE2', 'OC20M', 'OC22', 'Li2025General', 'RANDOM'], where 'RANDOM' means using a randomly initialized fitting net.
[2025-08-14 10:05:59,125] DEEPMD INFO Detailed information:
+-----------------------+------------------------------+--------------------------------+--------------------------------+
| Model Branch | Alias | description | observed_type |
+-----------------------+------------------------------+--------------------------------+--------------------------------+
| Dai2023Alloy | Alloys, Domains_Alloy | The dataset contains | ['La', 'Fe', 'Ho', 'Cu', 'Sn', |
| | | structure-energy-force-virial | 'Cd', 'Y', 'Be', 'V', 'Sm', |
| | | data for 53 typical metallic | 'In', 'Pr', 'Mo', 'Mn', 'Gd', |
| | | elements in alloy systems, | 'Ru', 'Nd', 'Li', 'Tm', 'K', |
| | | including ~9000 intermetallic | 'Pt', 'Ir', 'Na', 'Hf', 'Dy', |
| | | compounds and FCC, BCC, HCP | 'Ca', 'Nb', 'Au', 'Sr', 'Si', |
| | | structures. It consists of two | 'Ge', 'Co', 'W', 'Cr', 'Zn', |
| | | parts: DFT-generated relaxed | 'Ag', 'Ti', 'Ni', 'Zr', 'Pd', |
| | | and deformed structures, and | 'Os', 'Ta', 'Rh', 'Sc', 'Tb', |
| | | randomly distorted structures | 'Al', 'Ga', 'Re', 'Lu', 'Er', |
| | | produced covering pure metals, | 'Mg', 'Ce', 'Pb'] |
| | | solid solutions, and | |
| | | intermetallics with vacancies. | |
+-----------------------+------------------------------+--------------------------------+--------------------------------+
| OMAT24 | Default, Materials, Omat24 | OMat24 is a large-scale open | ['La', 'Fe', 'Cu', 'Cd', 'Be', |
| | | dataset containing over 110 | 'Ar', 'V', 'Sm', 'In', 'Pm', |
| | | million DFT calculations | 'Pr', 'Mn', 'Ru', 'He', 'Nd', |
| | | spanning diverse structures | 'Th', 'Pa', 'K', 'Pt', 'Yb', |
| | | and compositions. It is | 'Dy', 'Sr', 'Co', 'Np', 'Cr', |
| | | designed to support AI-driven | 'Tl', 'Br', 'Se', 'Ni', 'Zr', |
| | | materials discovery by | 'Pu', 'O', 'Xe', 'Tb', 'Ga', |
| | | providing broad and deep | 'Lu', 'H', 'Ne', 'Er', 'Ce', |
| | | coverage of chemical space. | 'I', 'Kr', 'Ho', 'Cs', 'Sn', |
| | | | 'Rb', 'Y', 'N', 'F', 'Mo', |
| | | | 'Gd', 'B', 'Li', 'Tm', 'Sb', |
| | | | 'Ir', 'Hf', 'Na', 'Ca', 'Nb', |
| | | | 'Au', 'As', 'Si', 'Ge', 'W', |
| | | | 'Zn', 'Hg', 'Ag', 'Bi', 'Ti', |
| | | | 'Os', 'Cl', 'Pd', 'P', 'U', |
| | | | 'Tc', 'Ta', 'Ba', 'Rh', 'Sc', |
| | | | 'C', 'S', 'Te', 'Al', 'Re', |
| | | | 'Eu', 'Mg', 'Pb', 'Ac'] |
+-----------------------+------------------------------+--------------------------------+--------------------------------+
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Alias-based multi-task branch selection for evaluation and
fine-tuning; new API to query model alias/branch info; show now prints a
detailed model-branch table.
* **Documentation**
* Model config gains optional fields to declare branch aliases and
per-branch info (PyTorch-only).
* **Examples**
* Added a two-task PyTorch example demonstrating aliases, shared
components, and per-branch info.
* **Tests**
* Tests include the new example and now filter out table-like show
output.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Duo <50307526+iProzd@users.noreply.github.com>
Co-authored-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
* feat(ci): skip workflows on bot branches to avoid redundant CI runs (#4916)
This PR implements a feature request to skip all GitHub workflows on
push events for bot-created branches to avoid redundant CI runs and save
resources.
## Problem
Bot-created branches (`copilot/*`, `dependabot/*`, and
`pre-commit-ci-update-config`) currently trigger workflows on both push
events and when PRs are created. This creates duplicate CI runs since
the same tests will run again when the PR is opened, wasting CI time and
resources.
## Solution
Added `branches-ignore` patterns to workflow files that have push
triggers to skip the following branch patterns:
- `copilot/**` - GitHub Copilot branches
- `dependabot/**` - Dependabot dependency update branches
- `pre-commit-ci-update-config` - Pre-commit CI configuration update
branches
## Changes Made
Updated 8 workflow files with bot branch ignore patterns:
- `build_cc.yml`, `build_wheel.yml`, `codeql.yml`, `package_c.yml`,
`test_cc.yml`, `test_python.yml` - Added bot branch patterns to existing
`branches-ignore` lists
- `copilot-setup-steps.yml` - Added `branches-ignore` alongside existing
`paths` filter
- `mirror_gitee.yml` - Converted from array syntax to explicit push
configuration with `branches-ignore`
The `todo.yml` workflow was left unchanged since it only runs on the
`devel` branch, making bot branch exclusions unnecessary.
Example of the change:
```yaml
on:
push:
branches-ignore:
- "gh-readonly-queue/**" # existing
- "copilot/**" # new
- "dependabot/**" # new
- "pre-commit-ci-update-config" # new
```
## Impact
- ✅ Bot branches will skip workflows on push events but still trigger
them when PRs are created
- ✅ Normal development branches continue to trigger workflows as
expected
- ✅ Reduces unnecessary CI runs and resource usage
- ✅ Maintains full test coverage through PR-triggered workflows
- ✅ All workflow files maintain valid YAML syntax
Fixes #4915.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
* feat: handle masked forces in test (#4893)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added per-atom weighting for force evaluation: computes and reports
weighted MAE/RMSE alongside unweighted metrics, includes weighted
metrics in system-average summaries, logs weighted force metrics, and
safely handles zero-weight cases. Also propagates the per-atom weight
field into reporting.
- Tests
- Added end-to-end tests validating weighted vs unweighted force
MAE/RMSE and verifying evaluator outputs when using per-atom weight
masks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: add comprehensive type hints to core modules excluding backends and tests (#4936)
- [x] Add comprehensive type hints to core modules excluding backends
and tests
- [x] **Fixed type annotation issues from code review:**
- Fixed `head` parameter type from `Any` to `str` in calculator.py
- Fixed `neighbor_list` parameter type to use proper ASE NeighborList
type annotation
- Fixed `**kwargs` type from `object` to `Any` in deep_polar.py
- Fixed `write_model_devi_out` return type from `None` to `np.ndarray`
to match actual return value
- Fixed `get_natoms_vec` return type from `list[int]` to `np.ndarray` to
match actual return type
- Fixed `_get_natoms_2` return type from `list[int]` to `tuple[int,
np.ndarray]` to match actual return values
- Fixed `make_index` return type from `dict[str, int]` to `str` to match
actual return value
- Added missing imports for type annotations (ASE NeighborList, Any)
**Current status:** All type annotation suggestions from code review
have been addressed. All ruff checks pass with zero violations.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/deepmodeling/deepmd-kit/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
* feat: support using train/valid data from input.json for dp test (#4859)
This pull request extends the testing functionality in DeepMD by
allowing users to specify training and validation data directly via
input JSON files, in addition to existing system and datafile options.
It updates the command-line interface, the main test logic, and adds
comprehensive tests to cover these new features, including support for
recursive glob patterns when selecting systems from JSON files.
### Feature enhancements to testing data sources
* The `test` function in `deepmd/entrypoints/test.py` now accepts
`train_json` and `valid_json` arguments, allowing users to specify
training or validation systems for testing via input JSON files. It
processes these files to extract system paths, including support for
recursive glob patterns. The function also raises an error if no valid
data source is specified.
[[1]](diffhunk://#diff-299c01ed4ee7d0b3f636fe4cb4f0d660a5012b7e95ca0740098b3ace617ab16eL61-R71)
[[2]](diffhunk://#diff-299c01ed4ee7d0b3f636fe4cb4f0d660a5012b7e95ca0740098b3ace617ab16eL104-R151)
* **The command-line interface in `deepmd/main.py` is updated to add
`--train-data` and `--valid-data` arguments for the test subparser,
enabling direct specification of input JSON files for training and
validation data.**
### Test coverage improvements
* New and updated tests in `source/tests/pt/test_dp_test.py` verify the
ability to run tests using input JSON files for both training and
validation data, including cases with recursive glob patterns. This
ensures robust handling of various data source configurations.
[[1]](diffhunk://#diff-ce70e95ffdb1996c7887ea3f63b54d1ae0fef98059572ad03875ca36cfef3c34L33-R35)
[[2]](diffhunk://#diff-ce70e95ffdb1996c7887ea3f63b54d1ae0fef98059572ad03875ca36cfef3c34R49-R59)
[[3]](diffhunk://#diff-ce70e95ffdb1996c7887ea3f63b54d1ae0fef98059572ad03875ca36cfef3c34R103-R116)
[[4]](diffhunk://#diff-ce70e95ffdb1996c7887ea3f63b54d1ae0fef98059572ad03875ca36cfef3c34R164-R273)
* Additional argument parser tests in
`source/tests/common/test_argument_parser.py` confirm correct parsing of
the new `--train-data` and `--valid-data` options.
### Internal code improvements
* Refactored imports and type annotations in
`deepmd/entrypoints/test.py` to support the new functionality and
improve code clarity.
[[1]](diffhunk://#diff-299c01ed4ee7d0b3f636fe4cb4f0d660a5012b7e95ca0740098b3ace617ab16eR17)
[[2]](diffhunk://#diff-299c01ed4ee7d0b3f636fe4cb4f0d660a5012b7e95ca0740098b3ace617ab16eR42-R50)
[[3]](diffhunk://#diff-299c01ed4ee7d0b3f636fe4cb4f0d660a5012b7e95ca0740098b3ace617ab16eL77-R95)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added support for supplying test systems via JSON files, including
selecting training or validation data.
- Introduced CLI options --train-data and --valid-data for the test
command.
- Supports resolving relative paths from JSON and optional recursive
glob patterns.
- Changes
- Test command now requires at least one data source (JSON, data file,
or system); clearer errors when none or no systems found.
- Tests
- Expanded test coverage for JSON-driven inputs and recursive glob
patterns; refactored helpers for improved readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Chun Cai <amoycaic@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(tf): implement change-bias command (#4927)
Implements TensorFlow support for the `dp change-bias` command with
proper checkpoint handling and variable restoration. This brings the
TensorFlow backend to feature parity with the PyTorch implementation.
## Key Features
- **Checkpoint file support**: Handles individual checkpoint files
(`.ckpt`, `.meta`, `.data`, `.index`) and frozen models (`.pb`)
- **Proper variable restoration**: Variables are correctly restored from
checkpoints using session initialization before bias modification
- **User-defined bias support**: Supports `-b/--bias-value` option with
proper validation against model type_map
- **Data-based bias calculation**: Leverages existing
`change_energy_bias_lower` functionality for automatic bias computation
- **Checkpoint preservation**: Saves modified variables to separate
checkpoint directory for continued training
- **Cross-backend consistency**: Identical CLI interface and
functionality as PyTorch backend
## Before vs After
**Variable restoration**:
- Before: `Change energy bias of ['O', 'H'] from [0. 0.] to [calculated
values]` (variables never restored)
- After: `Change energy bias of ['O', 'H'] from [-93.57 -187.15] to
[-93.60 -187.19]` (proper restoration)
**Output**: Creates both updated checkpoint files AND frozen model for
continued training
**Documentation**: Comprehensive documentation covering both TensorFlow
and PyTorch backends with examples and backend-specific details
The implementation includes comprehensive test coverage with real model
training to validate functionality without mocks.
Fixes #4018.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/deepmodeling/deepmd-kit/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Signed-off-by: Jinzhe Zeng <njzjz@qq.com>
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
Co-authored-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* style: complete type annotation enforcement for deepmd.pt (#4943)
This PR implements comprehensive type annotation coverage for the
deepmd.pt PyTorch backend and resolves critical TorchScript compilation
errors that prevented model deployment.
## Type Annotation Enforcement
Added complete type annotations to all deepmd.pt module functions,
eliminating 7,030+ ANN violations across 107 Python files. This
provides:
- Better IDE support and code maintainability
- Consistent typing standards throughout the PyTorch backend
- Enhanced developer experience with clear function signatures
## TorchScript Compilation Fixes
Resolved multiple TorchScript compilation errors that prevented model
deployment:
```python
# Before: TorchScript compilation failed
sw.to(dtype=env.GLOBAL_PT_FLOAT_PRECISION) # Error on Optional[Tensor]
# After: Proper None handling
sw.to(dtype=env.GLOBAL_PT_FLOAT_PRECISION) if sw is not None else None
```
Key fixes include:
- Added proper None checks before `.to()` calls on
`Optional[torch.Tensor]` values
- Resolved issues across all descriptor types (SE-A, SE-T, SE-T-TEBD,
DPA1, DPA2, DPA3)
- Fixed abstract method patterns that conflicted with TorchScript
compilation
- Corrected return type annotations in SpinModel to accurately reflect
Optional types
## Pre-commit Compliance
- Fixed deprecated type annotation imports (Dict→dict, Tuple→tuple)
- Resolved import ordering and undefined name issues
- Removed unnecessary imports and improved code consistency
- All pre-commit checks now pass with zero violations
The PyTorch backend now has complete type coverage and full TorchScript
deployment compatibility, enabling production model serving scenarios.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/deepmodeling/deepmd-kit/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: njzjz <9496702+njzjz@users.noreply.github.com>
Co-authored-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>
* fix(tf): fix serialization of dipole fitting with sel_type (#4934)
Fix #3672.
Fixes backend conversion issues for dipole models when using the
`sel_type` parameter. The `dp convert-backend` command was failing due
to missing serialization support for `None` networks and incomplete
dipole fitting serialization.
- [x] Fix NetworkCollection serialization to handle `None` networks
- [x] Add missing `@variables` dictionary for DipoleFittingSeA PyTorch
compatibility
- [x] Include `sel_type` in serialized data for proper backend
conversion
- [x] Fix TF fitting deserialization to skip `None` networks
- [x] Add comprehensive tests for `sel_type` parameter
- [x] Remove duplicate test classes and merge parameterized tests
- [x] Clean up accidentally committed test output files
- [x] Refactor addi…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
support gradient accumulation for paddle backend.
Summary by CodeRabbit
New Features
Documentation
Tests