Skip to content

Fix VERBOSE level not registered in multiprocessing workers#546

Open
samc24 wants to merge 1 commit intodevfrom
sameerc/fix-verbose-subprocess
Open

Fix VERBOSE level not registered in multiprocessing workers#546
samc24 wants to merge 1 commit intodevfrom
sameerc/fix-verbose-subprocess

Conversation

@samc24
Copy link
Copy Markdown
Collaborator

@samc24 samc24 commented Apr 7, 2026

Summary

  • Move VERBOSE level registration from setup_logging() to module-level in log_settings.py, so it's available in any process that imports the module
  • Fixes all physics methods silently returning NaN on macOS/Windows (spawn) and Linux with Python 3.14+ (forkserver)

Problem

The custom VERBOSE level and logger.verbose() method are registered inside setup_logging(), which only runs in the main process. Worker processes created by multiprocessing.Pool don't inherit this on:

  • macOS/Windows (default start method: spawn)
  • Linux + Python 3.14 (default changed from fork to forkserver)

This causes params.logger.verbose() in the physics method decorator (decorator.py:61) to throw AttributeError. The broad except in populate_method catches it and replaces the method's real result with NaN -- for every method, every shot.

Verification

Tested with MAST shot 30420, num_processes=1, macOS + Python 3.11 (spawn):

  • dev: 0/33 parameters have data (100% NaN, every method errors with AttributeError)
  • This branch: 32/33 parameters have data (1 expected NaN: power_nbi missing for this shot)

Greg also confirmed: dies on Linux + Python 3.14 (forkserver), works from this branch.

Test plan

  • Reproduced on macOS with spawn — all NaN on dev, real data on fix branch
  • Greg confirmed reproduction on Linux + Python 3.14 (forkserver)
  • CI (Linux + fork, unaffected but should pass as before)

References [edit by GLT]

https://docs.python.org/3/whatsnew/3.14.html#concurrent-futures

concurrent.futures

Signed-off-by: Sameer Chaturvedi <sameerc@mit.edu>
@samc24 samc24 requested a review from gtrevisan April 7, 2026 14:17
@gtrevisan
Copy link
Copy Markdown
Member

yes, I was able to replicate it on a linux laptop:

export REPO="git+https://github.com/mit-psfc/disruption-py"
  • 🟢 uvx --python 3.13 --from "$REPO@dev" disruption-py -m get_kappa_area
  • 🔴 uvx --python 3.14 --from "$REPO@dev" disruption-py -m get_kappa_area

this hinges on using uvx because a normal git clone + uv sync would rightfully detect that python 3.14 is unsupported -- uvx bug tracked at https://github.com/astral-sh/uv/issues /8206 ...
(just FYI: I pinned python below 3.14 due to some packages back in the day, so we might unpin it soon.)

with your fix, both work:

  • 🟢 uvx --python 3.13 --from "$REPO@0fe51d0" disruption-py -m get_kappa_area
  • 🟢 uvx --python 3.14 --from "$REPO@0fe51d0" disruption-py -m get_kappa_area

BUT it does not look like I can switch log level anymore with python 3.14 (!):

  • 🟢 uvx --python 3.13 --from "$REPO@0fe51d0" disruption-py -m get_kappa_area -l critical
  • 🔴 uvx --python 3.14 --from "$REPO@0fe51d0" disruption-py -m get_kappa_area -l critical

I read again the release notes for 3.14, but I could not see anything that might help.
I did check and in both cases we run the same loguru version:

  • uvx --python 3.13 --from "$REPO@0fe51d0" python -c "import loguru; print(loguru.__version__)" = 0.7.3
  • uvx --python 3.14 --from "$REPO@0fe51d0" python -c "import loguru; print(loguru.__version__)" = 0.7.3

Copy link
Copy Markdown
Member

@gtrevisan gtrevisan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samc24 please LMK whether you can replicate this loglevel behavior from your macbook!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants