Support WASIX

[syrusakbary]

Zig currently doesn't support Wasm/WASIX applications.

WASIX is a new standard emerging (as a non-breaking addition) over WASI Preview 1. WASIX adds full support for things like: pthreads, sockets, fork, process spawning and many more things.
See more detailed blogpost for the announcement here: https://wasmer.io/posts/announcing-wasix

And WASIX website: https://wasix.org/

Expected Behavior

Zig supporting WASIX would be huge for the Wasm and Zig communities.

Why?

• It would let using Zig in many platforms (including the browser)
• It would allow having a generic - C compiler that compiles C files (using LLVM under the hood) to any architecture, and runnable from any architecture that supports running Wasm
• It would allow creating http servers (or TCP/UDP ones) to run Zig or C files in the Edge and even the browser

[rohlem]

I'll be taking a look at this.
Anyone interested in collaborating feel free to let me know.

[git-bruh]

Is it correct to assume that the first step should be to integrate https://github.com/wasix-org/wasix-libc into wasi_libc.zig (Translating the wasix-libc Makefile steps), then get C programs (via zig cc) working, and then work on porting the wasm-related zig stdlib code to wasix-libc?

Working on it here https://github.com/git-bruh/zig

[syrusakbary]

Yup, that seems like a reasonable approach @git-bruh .

[rohlem]

@git-bruh That sounds like a reasonable approach.
I've been trying the opposite, to start with manually specifying wasix-libc using the --libc CLI argument with a finished build, however I'm a bit stuck on runtime errors in the startup code of the resulting wasm module.
It's possible that some flags or known workarounds in the wasi_libc.zig logic resolve those issues (or that my wasix build got messed up for other reasons).

[Dispersia]

Question for your last part, wasix can run UDP in the browser? Or is that just worded in a way I am confusing. I thought even with wasm it wasn't possible to use UDP from the browser

[git-bruh]

Got a super hacky PoC working -- had to stub out a bunch of files that don't compile (related to pthreads and processes)

[rohlem]

Alright, I've got something working, but it might not be worthwhile progress yet.

I'm still hoping that wasi_libc.zig solves some of these issues, but it's probably not the only missing piece.

---

zig cc <C source file> --sysroot=...: Fails with undeclared function errors, I might be doing it wrong.

This should be the easiest, because the wasix readme basically says clang is supposed to work / the only supported compiler.
The --sysroot argument is recognized (in that it doesn't lead to the error that unrecognized arguments do), but I keep getting undefined function errors regardless, so I assume Zig keeps choosing the packaged libc instead.

---

zig build-exe <C source file> --libc=....: Works with tweaks / if code doesn't use TLS.

• The current Makefile allows choosing THREAD_MODEL as posix or single,
  however it also contains wildcards and find invocations which always end up including the pthread-dependent files anyway.
  FWICT this Makefile logic needs to be fixed to re-enable single-threaded builds of wasix-libc; seems like that hasn't been tested in a while.
• When compiling with THREAD_MODEL=posix, the resulting executables trigger Wasmer to emit error: RuntimeError: out of bounds memory access:
  • any time they access __wasilibc_pthread_self, which is in ____wasi_init_tp during startup and throughout the library, f.e. in getrandom.
    I assume this might be a linker bug related to TLS, but not sure.
  • also once in __wasilibc_init_ssp. There's no TLS usage here, so I'm not sure what the cause might be.
    Maybe global initialization / CRT constructor calls happen too soon for some reason?

Without much prior knowledge of linkers and WASM I might poke around with binaryen's wasm-dis and wasm-as to see what the issue is.

---

zig build-exe <Zig source file> --libc=...: Fails due to strange lookup failure of symbol main.
libc.a somehow depends on main but doesn't find it. This might have to do with exporting behavior, not sure.
Should probably look for differences between wasix-libc and the bundled wasi-libc, since the latter works.

---

@Dispersia How I understand it is that WASIX is a standardization effort of features beyond WASI, like UDP sockets.
For now (only?) Wasmer supports the standard, but the goal would be for other environments (like Browsers) to also support it in the future.

[git-bruh]

@rohlem Some of the errors at runtime when using threads/processes are probably an upstream thing, they happen for me even when compiling everything using the official wasi-sdk toolchain

[rohlem]

@git-bruh Great progress!
Interesting, from looking at the Makefile it seemed like at some point a single-threaded build without TLS should have worked by just excluding those files.

[git-bruh]

Yeah many parts of the Makefile are broken, like the globs unconditionally include threading-related code even if it's disabled

[ifreund]

Before people put large amounts of effort into implementing this, let's make clear that this is a feature request/proposal which has not (yet?) been accepted by the Zig leadership.

As for my personal take, I see WASIX as an effort to port legacy POSIX APIs (including many with questionable design such as fork()) to WASM. This of course makes it technically easier to port existing programs relying on these legacy POSIX APIs to WASM but I worry about how runtimes could possibly implement this API surface securely while maintaining reasonable performance. It frankly seems quite like "containers, but with extra steps"

Furthermore, It appears to be driven by a single company which is also responsible for the only implementation. I don't any information on the standardization process and can only assume that it is not open. Without any 3rd party implementations including support in browsers, I think adding support to the Zig standard library would be quite premature.

Overall, I do not see how supporting WASIX in upstream Zig aligns with the goals and values of the Zig project.

[rohlem]

@ifreund The way I see it, if WASIX is a functional libc, and Zig allows users to provide a custom libc, then there shouldn't be harm in ironing out bugs (in both projects) to enable that support.

Zig is of course free to decide not to ship with it, and not to mention it as supported target, nor in the official std module.
But a third-party package should ultimately be able to offer all of the same functionality via the Bring-Your-Own-OS mechanism, right?
(Only difference being users have to add the corresponding entry to their projects' build.zon.)