Skip to content

Add Python 3.14 support#425

Closed
Copilot wants to merge 2 commits intoadd-python3.14from
copilot/sub-pr-402-another-one
Closed

Add Python 3.14 support#425
Copilot wants to merge 2 commits intoadd-python3.14from
copilot/sub-pr-402-another-one

Conversation

Copy link
Contributor

Copilot AI commented Feb 25, 2026

Adds Python 3.14 support with minimal implementation changes. The core issue was that Python 3.14's multiprocessing changes caused deadlocks and ordering failures in the parallel extraction path. The fix restores correct parallel behavior without sacrificing the original heap-based load balancing.

DevOps

  • Bump Python version constraint to <3.15 in setup.py, conda/dev.yml
  • Add Python 3.14 to CI matrix with fail-fast: false

Parallel extraction fixes (zstash/extract.py, zstash/parallel.py)

Removed serializing wait_turn() call from extraction start — it was blocking workers from downloading/extracting until previous workers finished printing, collapsing parallelism to serial. Output ordering is still enforced inside print_all_contents(), which calls wait_turn() internally before flushing output.

HPSS auth fallbackextractFiles() now falls back to config.hpss when args.hpss is None. Previously it raised unconditionally, breaking --workers without --hpss when the HPSS path came from the database.

Manager lifecyclemanager.shutdown() is called after all worker processes exit, preventing a leaked manager server process.

Reliable failure queue drain — replaced queue.empty() polling (unreliable across processes per Python docs) with get_nowait() / queue.Empty loops.

Precomputed tar→index map in PrintMonitor_tar_to_index: Dict[str, int] replaces linear list.index() lookups; unknown tar names now raise RuntimeError instead of silently returning.

tar.extract() filter for Python 3.12+ — added filter="tar" to suppress the deprecation warning introduced in 3.12.

Select one: This pull request is...

  • a bug fix: increment the patch version
  • a small improvement: increment the minor version
  • a new feature: increment the minor version
  • an incompatible (non-backwards compatible) API change: increment the major version

Big Change

  • To merge, I will use "Create a merge commit". That is, this change is large enough to require multiple units of work (i.e., it should be multiple commits).

1. Does this do what we want it to do?

Required:

  • Product Management: I have confirmed with the stakeholders that the objectives above are correct and complete.
  • Testing: I have added at least one automated test. Every objective above is represented in at least one test.
  • Testing: I have considered likely and/or severe edge cases and have included them in testing.

If applicable:

  • Testing: this pull request adds at least one new possible command line option. I have tested using this option with and without any other option that may interact with it.

2. Are the implementation details accurate & efficient?

Required:

  • Logic: I have visually inspected the entire pull request myself.
  • Logic: I have left GitHub comments highlighting important pieces of code logic. I have had these code blocks reviewed by at least one other team member.

If applicable:

  • Dependencies: This pull request introduces a new dependency. I have discussed this requirement with at least one other team member. The dependency is noted in zstash/conda, not just an import statement.

3. Is this well documented?

Required:

  • Documentation: by looking at the docs, a new user could easily understand the functionality introduced by this pull request.

4. Is this code clean?

Required:

  • Readability: The code is as simple as possible and well-commented, such that a new team member could understand what's happening.
  • Pre-commit checks: All the pre-commits checks have passed.

If applicable:

  • Software architecture: I have discussed relevant trade-offs in design decisions with at least one other team member. It is unlikely that this pull request will increase tech debt.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

… manager cleanup, queue drain, precomputed tar index map

Co-authored-by: forsyth2 <30700190+forsyth2@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for Python 3.14 Add Python 3.14 support Feb 25, 2026
Comment on lines -552 to -554
multiprocess_worker.print_monitor.wait_turn(
multiprocess_worker, files_row.tar
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolves #402 (comment)

hpss = config.hpss
else:
raise TypeError("Invalid args.hpss={}".format(args.hpss))
raise TypeError("Invalid config.hpss={}".format(config.hpss))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolves #402 (comment)

except queue.Empty:
pass

manager.shutdown()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolves #402 (comment)

# Drain any remaining failures after all processes have exited.
try:
while True:
failures.append(failure_queue.get_nowait())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolves #402 (comment)

@forsyth2
Copy link
Collaborator

Cherry-picked commit to #402

@forsyth2 forsyth2 closed this Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants