Conversation
Closed
Since https://gitlab.com/petsc/petsc/-/commit/4237731afe36b4db0a3147aff0aed51d4657afa2 and the introduction of `petsc_raises` (c9d3922) we have been seeing random failures on CI that I have been unable to reproduce locally. I was able to reproduce the stochastic failures with a Docker image so my best guess is that these are caused by differences in the Python version (since the PETSc error handling touches a *lot* of the Python internals for handling exceptions). The best approach to me seems to be to allow the more permissive exception check until we reach a Python version that we think should be fine. We can revisit then if the issue continues.
connorjward
force-pushed
the
connorjward/petsc_raises
branch
from
999c421 to
5972883
Compare
dham
approved these changes
Jun 25, 2025
connorjward
added a commit
that referenced
this pull request
Jun 26, 2025
* Specialise petsc_raises on Python version (<3.12.11) Since https://gitlab.com/petsc/petsc/-/commit/4237731afe36b4db0a3147aff0aed51d4657afa2 and the introduction of `petsc_raises` (c9d3922) we have been seeing random failures on CI that I have been unable to reproduce locally. I was able to reproduce the stochastic failures with a Docker image so my best guess is that these are caused by differences in the Python version (since the PETSc error handling touches a *lot* of the Python internals for handling exceptions). The best approach to me seems to be to allow the more permissive exception check until we reach a Python version that we think should be fine. We can revisit then if the issue continues. * update comment
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since https://gitlab.com/petsc/petsc/-/commit/4237731afe36b4db0a3147aff0aed51d4657afa2 and the introduction of
petsc_raises(c9d3922) we have been seeing random failures on CI that I have been unable to reproduce locally.I was able to reproduce the stochastic failures with a Docker image (Python 3.12.3) which appear to go away when I install Python 3.12.11 (matches my local machine). My best guess is therefore that Python fixed things between these releases.
The best approach to me seems to be to allow the more permissive exception check until we reach a Python version that we think should be fine. We can revisit then if the issue continues.
The only other clue I have is we get the following reported error when things fail:
The
Exception ignored in tp_clear of: <class 'dict'>makes it seem like a GC issue.