Allocator inconsistency between bin and dylib

[tsurai]

I recently had a problem with a dylib causing a segfault when triggering drop in an object passed to it from a rust binary.

jethrogb on the #rust irc channel found out that rustc build the dylib with alloc_system and the binary with alloc_jemalloc leading to all kinds of memory problems. I am not sure if this could be considered a bug or just a lack of documentation/warnings.

Another problem is that there seems to be no good way to fix this problem because #![feature(alloc_jemalloc)] requires nightly and building the dylib with -C prefer-dynamic makes the lib depend on rust std lib to be installed on the host machine.

Please correct me if I'm missing something

[eefriedman]

You're building a dylib and an exe, and statically linking libstd into both? rustc should definitely print an error message if you try to build an executable in that configuration. It's not just a matter of the allocator; libstd has other internal state that would be duplicated.

Why are you trying to do this? If you're distributing dynamic libraries anyway, why not just include libstd.dylib in your distribution?

[tsurai]

I uploaded a minimal test case to illustrate my problem. The program will seg fault when trying to drop the string in the dylib function.

[alexcrichton]

As @eefriedman mentioned, this is unfortunately just the classical "don't statically link a library twice" problem going on here. Statically linking the standard library to two artifacts and then loading them at runtime tends to always cause weird problems like this. In general this is where having resource ownership cross DLL boundaries is a "definite no-no". The compiler can't help you much here because of the dynamic/runtime aspect of loading (you'd get warnings and such if you linked the two together).

Otherwise, though, it's expected that binaries use jemalloc (e.g. where Rust controls the world) and dylibs use the system allocator (where Rust is just a guest in someone else's world). As a result I'm going to close this as working as intended, but I think I may also try to write up some docs on custom allocators!

[eefriedman]

@alexcrichton Does it sound feasible to print an error message in this scenario? That would be a lot more friendly than crashing at runtime.

[alexcrichton]

Unfortunately not in an uninvasive manner that I know of. We could install a dlopen hook that looks for some other registered piece of Rust code and make sure the allocator is the same, but it may be overkill to do so.

[eefriedman]

@alexcrichton Oh, right, I wasn't looking carefully enough; we can't really do anything if someone is using unsafe code to load a library at runtime.

[tsurai]

It is a fairly common use case though and would be nice for this behaviour to be properly documented somewhere. Took me a lot time to find the out what caused my memory problems and that only with the help of the rust irc channel.

[alexcrichton]

Certainly! I started writing some documentation over at #28869, and more eyes are always welcome!