Alcotest silently fails when aborted from C stub

[SturdyPose]

This example doesn't show if test failed at all.

CAMLprim value caml_crashing_function()
{
  CAMLparam0();
  int* p = NULL;
  *p = 0;
  CAMLreturn(Val_unit);
}

(* link with cstub*)
external crashing_function: unit -> unit = "caml_crashing_function"

let () = 
  run "Tests" ["Crash test", [test_case "call crashing function" `Quick (function _ -> crashing_function ())]]

There's no output and if the test has crashed somewhere in the middle of test, remaining suite isn't executed and it won't show any error.

Environment: win11 x86-64 using msys2-mingw64

[MisterDA]

I think we're going to have a hard time recovering from an access violation / segfault on any system. That's probably a bit out-of-scope for Alcotest.
AFAIK Alcotest doesn't run tests in sub-processes, so cannot protect from process-level crashes. It can catch OCaml exceptions, of course.

[SturdyPose]

Then would that be a feature request to run sub processes? 🤔 Instead of only Quick and Slow flag if there could be `Subprocess

Maybe I'll experiment if I can run alcotests from C and setup signal trap.

[SturdyPose]

Managed to catch any nullptrs and print addresses where the exception was called. I'm not sure how to get correct function names though, there might be something going on with flexlink.

#if defined(_WIN32) || defined(_WIN64)
#define PLATFORM_WINDOWS
#include <windows.h>
#include <dbghelp.h>

void print_stack_trace(CONTEXT* context) {
    HANDLE process = GetCurrentProcess();
    HANDLE thread  = GetCurrentThread();

    if(!SymInitialize(process, NULL, TRUE))
    {
      printf("SymInitialize failed\n");
    }

    STACKFRAME64 frame = {0};
    DWORD machine_type;

    // Initialize the frame based on architecture
#ifdef _M_IX86
    machine_type = IMAGE_FILE_MACHINE_I386;
    frame.AddrPC.Offset    = context->Eip;
    frame.AddrPC.Mode      = AddrModeFlat;
    frame.AddrFrame.Offset = context->Ebp;
    frame.AddrFrame.Mode   = AddrModeFlat;
    frame.AddrStack.Offset = context->Esp;
    frame.AddrStack.Mode   = AddrModeFlat;
#else
    machine_type = IMAGE_FILE_MACHINE_AMD64;
    frame.AddrPC.Offset    = context->Rip;
    frame.AddrPC.Mode      = AddrModeFlat;
    frame.AddrFrame.Offset = context->Rbp;
    frame.AddrFrame.Mode   = AddrModeFlat;
    frame.AddrStack.Offset = context->Rsp;
    frame.AddrStack.Mode   = AddrModeFlat;
#endif

    printf("\n--- Call Stack ---\n");
    DWORD64 base = (DWORD64)GetModuleHandle(NULL);
    printf("  [0x%llx] base address\n", base);
    while (StackWalk64(machine_type, process, thread, &frame, context, 
                       NULL, SymFunctionTableAccess64, SymGetModuleBase64, NULL)) {
        
        DWORD64 displacement = 0;
        char buffer[sizeof(SYMBOL_INFO) + MAX_SYM_NAME * sizeof(TCHAR)];
        PSYMBOL_INFO symbol = (PSYMBOL_INFO)buffer;
        symbol->SizeOfStruct = sizeof(SYMBOL_INFO);
        symbol->MaxNameLen = MAX_SYM_NAME;
        if (SymFromAddr(process, frame.AddrPC.Offset, &displacement, symbol)) {
            printf("  [0x%llx] %s + 0x%llx\n", symbol->Address, symbol->Name, displacement);
        } else {
            DWORD lastErr = GetLastError();
            printf("  [0x%llx] (Unknown symbol) with error %i\n", frame.AddrPC.Offset, lastErr);
        }
    }
    SymCleanup(process);
}


LONG WINAPI windows_exception_handler(PEXCEPTION_POINTERS pExceptionInfo) {
    if (pExceptionInfo->ExceptionRecord->ExceptionCode == EXCEPTION_ACCESS_VIOLATION) {
        void* faulting_address = (void*)pExceptionInfo->ExceptionRecord->ExceptionInformation[1];
        print_stack_trace(pExceptionInfo->ContextRecord);
        caml_failwith("SEGFAULT"); 
    }
    return EXCEPTION_CONTINUE_SEARCH; 
}
#else
#define PLATFORM_UNIX
#include <signal.h>
  void unix_signal_handler(int sig, siginfo_t *si, void *unused) {
    caml_failwith("Exception occured");
    _exit(1);
  }
#endif


CAMLprim value caml_setup_exception_handler() 
{
  CAMLparam0();
#ifdef PLATFORM_WINDOWS
    AddVectoredExceptionHandler(1, windows_exception_handler);
#elif defined(PLATFORM_UNIX)
    struct sigaction sa;
    sa.sa_flags = SA_SIGINFO;
    sigemptyset(&sa.sa_mask);
    sa.sa_sigaction = unix_signal_handler;
    sigaction(SIGSEGV, &sa, NULL);
#endif
  CAMLreturn(Val_unit);
}

I call setup_exception_handler before running alcotests. Seems like even having segfault in the middle of tests doesn't crash.
On windows I just need to link against -ldbghelp

[MisterDA]

That's super cool! Would you like to open a PR?

[SturdyPose]

Sure thing! I also managed to get better stack trace where issue happened by following this awesome article: https://smhk.net/note/2025/03/c-stack-trace-in-windows/

On windows while compiling with gcc I only get DWARF symbols and they need to be converted to PDB with external cv2pdb tool. I'm thinking a way on how to automate this - if there's drag and drop dll or something.

Screenshot of segfault:

Edit: this needs a bit more finetuning. For example msvc can export PDB files by default whereas gcc/clang builds don't export pdb...

[SturdyPose]

@MisterDA If it's compiled with msvc it should give PDB files needed for debugging but gcc/clang and others need to be converted. I had an idea to give command line option which would scan PATH for cv2pdb to get correct stack trace. Do you think that the implementation of spawning child process which would convert the exe and then perform the tests would be right approach? That operation would run only when passed "-windemangle" flag I want to align before implementing it.

[MisterDA]

This sounds really complicated. I think it's ok if you get something working with your favorite toolchain first.