move autodocs logic from the compiler into the browser and avoid unnecessary javascript files #16490
Description
With Disable Cache checked, Firefox downloads 13 MiB over the network for the std lib docs. Meanwhile, a tarball of the standard library source files (GZIP=-9 tar cfz std.tar.gz std/) is only 2.1 MiB. This comparison isn't even counting all the .zig files that get converted into .zig.html. The preprocessing steps that the compiler does bloats the data for dubious benefits, when autodoc could be operating on a more compact data set instead.
Furthermore, #3403 is not implemented yet, and it is a hard requirement that I am imposing. Having all the source code available instead of prerendered in separate html files is the simplest and least wasteful way to implement that feature.
I also would like to see these files deleted from zig:
Especially the lexer. There is absolutely no reason to have this in javascript instead of sharing code with the actual lexer that we have written in zig. I don't want to hear any excuses about passing strings over the wasm/js boundary. There are various strategies to do it cleanly, you just need to use your imagination.
In other software projects, "use the right tool for the job" is a fair mantra. In this particular repository, however, we need to demonstrate that Zig is truly a general-purpose programming language. Please don't invest in any more javascript logic that should be zig logic instead. I'm going to lose my shit if somebody suggests adding a package.json to the docs/ folder.