Skip to content

crash on exit (race condition) #7

@speed47

Description

@speed47

This happens when we're in the done function, which ends up calling libc's exit() to finish the program. Once in a while (roughly once every ~100000 executions on our production environment), we can get a crash here.

Code dumps indicate that if we get a SIGCHLD or SIGHUP signal while we're in the libc's exit code, the execution flow gets diverted as it should be, and calls our handlers. The handlers of these two signals end up calling the done function too, which ends up calling exit() again.
Then, we get either a SIGABRT or a SIGSEGV.

Some of the stacktraces follow:

Program terminated with signal SIGABRT, Aborted.
#0  0x00007fb790a0b7bb in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) #0  0x00007fb790a0b7bb in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fb7909f6535 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fb790a4d508 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fb790a53c1a in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x00007fb790a556fd in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x000055c9cc1ec79a in done (status=<optimized out>) at ttyrec.c:1590
#6  0x000055c9cc1ec944 in finish (signal=<optimized out>) at ttyrec.c:977
#7  <signal handler called>
#8  0x00007fb790bd5600 in ?? () from /lib64/ld-linux-x86-64.so.2
#9  0x00007fb790a0dd8c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#10 0x00007fb790a0deba in exit () from /lib/x86_64-linux-gnu/libc.so.6
#11 0x000055c9cc1ec7ad in done (status=<optimized out>) at ttyrec.c:1592
#12 0x000055c9cc1ed39f in sighup_handler (signal=<optimized out>) at ttyrec.c:1565
#13 <signal handler called>
#14 0x00007fb790ba7092 in waitpid () from /lib/x86_64-linux-gnu/libpthread.so.0
#15 0x000055c9cc1ed4f9 in doinput () at ttyrec.c:945
#16 0x000055c9cc1ea324 in main (argc=<optimized out>, argv=<optimized out>) at ttyrec.c:819
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000559c68395115 in ZSTD_resetCCtx_internal ()
(gdb) #0  0x0000559c68395115 in ZSTD_resetCCtx_internal ()
#1  0x0000559c6839edeb in ZSTD_compressStream2.part.32 ()
#2  0x0000559c683a3227 in ZSTD_endStream ()
#3  0x0000559c683942ec in fclose_wrapper_zstd (fp=0x559c695d94c0) at compress_zstd.c:118
#4  0x0000559c68392783 in done (status=<optimized out>) at ttyrec.c:1576
#5  0x0000559c6839339f in sighup_handler (signal=<optimized out>) at ttyrec.c:1565
#6  <signal handler called>
#7  0x00007feedae411d7 in munmap () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x0000559c683a28e0 in ZSTD_freeCStream ()
#9  0x0000559c68394316 in fclose_wrapper_zstd (fp=0x559c695d94c0) at compress_zstd.c:125
#10 0x0000559c68392783 in done (status=<optimized out>) at ttyrec.c:1576
#11 0x0000559c683930b4 in dooutput () at ttyrec.c:1476
#12 0x0000559c683903c2 in main (argc=<optimized out>, argv=<optimized out>) at ttyrec.c:793

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions