Skip to content

pt: refactor data stat#3285

Merged
wanghan-iapcm merged 42 commits intodeepmodeling:develfrom
njzjz:env-mat-stat
Feb 18, 2024
Merged

pt: refactor data stat#3285
wanghan-iapcm merged 42 commits intodeepmodeling:develfrom
njzjz:env-mat-stat

Conversation

@njzjz
Copy link
Copy Markdown
Member

@njzjz njzjz commented Feb 17, 2024

No description provided.

njzjz added 30 commits February 16, 2024 15:34
@njzjz
checkpoint
f352a67 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
Merge branch 'devel' into env-mat-stat
b2b48b2
@njzjz
record sum
1c3e2bb 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
compute_std
d6bf4ab 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
protection
4f029b0 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
save stat
afd6d6a 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
get std
2f67aac 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
compute avg
89d413c 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
sea looks good
8b6e24e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
rm init_desc_stat
4736ae5 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
rm get_stat_name
6cb1275 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
rewrite compute_input_stats
b99d330 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
hybrid
da1e72d 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix hash
959583a 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
se atten
524366d 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
to make it work
87f0d85 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
compute_or_load_stat
2e815ea 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
init
ed34d59 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix shape
d04d16c 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix concat
2580f8e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
make it work
2771b1e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
rm save_stats and load_stats
3eb5577 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
assert_allclose
3ad5484 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix shape
2b9bbd8 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
add env mat type
7d40b9f 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
remove print
a55e21f 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
merge methods
c34622d 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
merge
af0711c 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
clean
37e9b28 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
clean
2c63335 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
njzjz added 2 commits February 16, 2024 21:23
@njzjz
fix load stats
d343c32 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
rm process_stat_path
6d8955a 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 17, 2024 •
edited
Loading

Codecov Report

Attention: 70 lines in your changes are missing coverage. Please review.

Comparison is base (9053caf) 75.06% compared to head (2ad6990) 74.84%.

Files Patch % Lines
deepmd/utils/path.py 63.15% 21 Missing ⚠️
deepmd/utils/env_mat_stat.py 75.43% 14 Missing ⚠️
deepmd/pt/model/descriptor/descriptor.py 42.85% 8 Missing ⚠️
deepmd/pt/entrypoints/main.py 45.45% 6 Missing ⚠️
deepmd/pt/model/descriptor/repformers.py 73.33% 4 Missing ⚠️
deepmd/pt/model/descriptor/se_a.py 80.00% 4 Missing ⚠️
deepmd/pt/model/descriptor/se_atten.py 73.33% 4 Missing ⚠️
deepmd/pt/model/task/ener.py 66.66% 3 Missing ⚠️
deepmd/pt/model/atomic_model/dp_atomic_model.py 77.77% 2 Missing ⚠️
deepmd/pt/utils/env_mat_stat.py 96.96% 2 Missing ⚠️
... and 2 more
Additional details and impacted files 
@@            Coverage Diff             @@
##            devel    #3285      +/-   ##
==========================================
- Coverage   75.06%   74.84%   -0.22%     
==========================================
  Files         387      389       +2     
  Lines       33640    33587      -53     
  Branches     1604     1604              
==========================================
- Hits        25251    25139     -112     
- Misses       7528     7587      +59     
  Partials      861      861

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

njzjz added 2 commits February 16, 2024 21:38
@njzjz
fix py38 compatibility
582451e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix typo
1c1c1a5 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
wanghan-iapcm
wanghan-iapcm reviewed Feb 17, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@wanghan-iapcm wanghan-iapcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the case of mixed_type, or not distinguish_type is not covered by UTs.

Comment thread deepmd/pt/utils/env_mat_stat.py
Comment thread deepmd/pt/utils/env_mat_stat.py
Comment thread source/tests/pt/test_stat.py Outdated
njzjz added 4 commits February 17, 2024 17:39
@njzjz
rm unused compute_std
b47014b 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
bugfix
ef5a92e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
Merge branch 'devel' into env-mat-stat
367f472 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
make test work
a8aef18 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
Copy link
Copy Markdown
Member Author

njzjz commented Feb 18, 2024

It seems that the case of mixed_type, or not distinguish_type is not covered by UTs.

Resolved.

@njzjz njzjz requested a review from wanghan-iapcm February 18, 2024 01:18
iProzd
iProzd reviewed Feb 18, 2024
View reviewed changes
Comment thread deepmd/pt/model/descriptor/descriptor.py
Comment thread deepmd/pt/utils/env_mat_stat.py
njzjz added 2 commits February 18, 2024 02:51
@njzjz
add type_map to stat_file_path
6a83465 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
update share_params
ba688a9 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
iProzd
iProzd reviewed Feb 18, 2024
View reviewed changes
Comment thread deepmd/pt/model/descriptor/descriptor.py
njzjz added 2 commits February 18, 2024 04:30
@njzjz
base_env starts from base_class.stats
7c9a66e 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz
fix py38 compatibility
2ad6990 
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@wanghan-iapcm wanghan-iapcm added this pull request to the merge queue Feb 18, 2024
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Feb 18, 2024
@wanghan-iapcm wanghan-iapcm merged commit c55ceac into deepmodeling:devel Feb 18, 2024
njzjz added a commit to njzjz/deepmd-kit that referenced this pull request Feb 18, 2024
@njzjz
pt: process frames in parallel for env mat stat
117acd9 
Resolves deepmodeling#3285 (comment)

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz njzjz mentioned this pull request Feb 18, 2024
Merged
github-merge-queue Bot pushed a commit that referenced this pull request Feb 19, 2024
@njzjz
pt: process frames in parallel for env mat stat (#3293)
ab35468 
Resolves
#3285 (comment).

We don't even need to consider whether the data uses mixed type in this
piece of code. The data can be reshaped to the atom level, and type
masks can be used to get an environmental matrix with certain types. No
frame-level things are involved here.

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
@njzjz njzjz mentioned this pull request Apr 2, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iProzd iProzd iProzd approved these changes

@wanghan-iapcm wanghan-iapcm Awaiting requested review from wanghan-iapcm

Assignees

No one assigned

Labels

Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@njzjz @iProzd @wanghan-iapcm