Unsafe threading termination causes hanging

[wk9874]

Describe the bug
Inside FileMonitor, you generate an exception callback:

Multiparser/multiparser/monitor.py

Line 131 in 36a865f

self._exception_callback = self._generate_exception_callback(exception_callback)

If abort_on_fail is True, this calls abort_func, which defaults to self.terminate():

Multiparser/multiparser/monitor.py

Line 178 in 36a865f

abort_func()

self.terminate() calls self._close_processes(), which calls join() on the file and log monitor threads:

Multiparser/multiparser/monitor.py

Line 621 in 36a865f

self._file_monitor_thread.join()

That is fine if only the FileMonitor can do this, as that runs from the main thread. However when creating the FullFileThreadMonitor or LogFileThreadMonitor threads in _create_monitor_threads(), this exception callback is passed to the threads themselves:

Multiparser/multiparser/monitor.py

Line 208 in 36a865f

exception_callback=self._exception_callback,

Which means now, if an exception is thrown during the main loop of a thread, it will execute this callback, effectively trying to call join() on itself, resulting in a deadlock.

This also happens in _handle_monitor_thread_exception:

Multiparser/multiparser/thread.py

Line 71 in 36a865f

self._exception_callback(e)

And in _raise_exceptions:

Multiparser/multiparser/thread.py

Line 452 in 36a865f

def _raise_exceptions(self) -> None:

To Reproduce
?????

Expected behavior
The threads should not have any way to call terminate or join. They should simply set a trigger, and allow the main thread (ie, FileMonitor), clear them up if it sees that the trigger has been set, reraising any exceptions as needed.

Evidence

Environment (please complete the following information):

• OS: Ubuntu 22.04
• Python version: 3.11

Additional context