Skip to content

Add Simple Check to Prevent npasses>1 When Caliper Enabled#648

Merged
MrBurmark merged 1 commit intodevelopfrom
cali-npasses
Feb 26, 2026
Merged

Add Simple Check to Prevent npasses>1 When Caliper Enabled#648
MrBurmark merged 1 commit intodevelopfrom
cali-npasses

Conversation

@michaelmckinsey1
Copy link
Contributor

@michaelmckinsey1 michaelmckinsey1 commented Feb 20, 2026

Summary

This PR addresses a known issue (#563) when using --npasses>1, where seemingly "incorrect" values will be propagated into the caliper files.

@MrBurmark and I discussed something like multiplying reps by npasses. Another option is we could rename reps to be reps/pass. However, @MrBurmark during this discussion, I forgot the issue of the other attributes being incorrectly set on the subsequent passes after the first pass. So the solution cannot be this simple.

To solve this:

  1. We could add a guard so attributes are not set again. This would also require one of the solutions discussed above as well, as metrics would need to be divided by npasses and reps in post-processing.
  2. flush Caliper files after each pass. So files would be named variant-tuning-passnum.cali like Base_Seq-1.cali. I think this would be a solution more befitting of Caliper's model than the first option.

This is just a temporary mechanism to prevent users from generating bad data by accident, until I implement a better solution, which I'm not sure will land before the 2025.12.0 release, so I am making this PR. This issue can also be really difficult for a user to catch, so it does need to be addressed in some form.

I tested some cases, which show what a user will see:

# If not building with caliper
### fine
raja-perf.exe --npasses 1 --dryrun
### fine
raja-perf.exe --npasses 2 --dryrun

# If building with Caliper
### fine
raja-perf.exe --npasses 1 --dryrun
### bad input
$ raja-perf.exe --npasses 2 --dryrun


Running with 1 MPI ranks...


Reading command line input...

Bad input: --npasses cannot be >1 when profiling with Caliper
Compiler: icpx-2025.2.0
...

Copy link

@tdrwenski tdrwenski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me as a temporary fix to alert users!

I am okay with either solution as long as you can still get back the info about how many reps and passes were run. I don't think 10 reps and 2 passes is exactly equivalent to 20 reps and 1 pass so would be good to have that info.

@MrBurmark
Copy link
Member

This seems like a reasonably simple stop-gap for the moment.
If you think that a file per pass is the solution that makes the most sense for caliper users then we can go to that.

@michaelmckinsey1
Copy link
Contributor Author

Seems good to me as a temporary fix to alert users!

I am okay with either solution as long as you can still get back the info about how many reps and passes were run. I don't think 10 reps and 2 passes is exactly equivalent to 20 reps and 1 pass so would be good to have that info.

You are correct, they are not equivalent. There is different caching behavior for those two options.

@MrBurmark
Copy link
Member

@rhornung67 Shall we merge?

@rhornung67
Copy link
Member

@MrBurmark yes, go for it.

@MrBurmark MrBurmark merged commit 1a7bb50 into develop Feb 26, 2026
18 checks passed
@MrBurmark MrBurmark deleted the cali-npasses branch February 26, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect attributes when npasses > 1 and using caliper

4 participants