Update Omake for Linux running#1060
Conversation
|
Ok, so I've done some testing, I've figured out how to get the readout from #define _GNU_SOURCE
#include <spawn.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <poll.h>
extern char **environ;
int main()
{
posix_spawn_file_actions_t spawn_actions;
posix_spawnattr_t attrs;
posix_spawn_file_actions_init(&spawn_actions);
posix_spawnattr_init(&attrs);
pid_t spawned_pid;
int pipes[2];
int ret = pipe2(pipes, O_CLOEXEC);
if (ret != 0)
{
printf("Pipe failed\n");
}
// Copy stdout to the writable pipe, pipes[0] is the readable pipe. Mirrors stdin vs stdout
posix_spawn_file_actions_adddup2(&spawn_actions, pipes[1], 1);
// Ensure pipes close
posix_spawn_file_actions_addclose(&spawn_actions, pipes[1]);
posix_spawn_file_actions_addclose(&spawn_actions, pipes[0]);
ret = posix_spawn(&spawned_pid, "/bin/sh", &spawn_actions, &attrs, (char *const[]){"sh", "-c", "--", "ls /var/", NULL}, environ);
if (ret)
{
printf("Failed to spawn! errno: %d\n", ret);
}
posix_spawn_file_actions_destroy(&spawn_actions);
posix_spawnattr_destroy(&attrs);
int wait_status = 0;
do
{
wait_status = 0;
char val[80] = {0};
struct pollfd polls = {.events = POLLIN, .fd = pipes[0]};
// Poll for < 100ms, if we timeout on having any data whatsoever, we check if the process is done, if the process is done, exit the read loop.
int ret = poll(&polls, 1, 100);
if (ret > 0)
{
if (polls.revents & POLLIN)
{
int success = read(pipes[0], val, sizeof(val) - 1);
printf("%s", val);
if (success < 1)
{
break;
}
}
}
if (ret == 0)
{
waitpid(spawned_pid, &wait_status, WUNTRACED);
if (!(!WIFEXITED(wait_status) && !WIFSIGNALED(wait_status) && !WIFSTOPPED(wait_status)))
{
break;
}
}
} while (1);
close(pipes[0]);
close(pipes[1]);
}I have verified in WSL that this prints the exact output of |
|
😄 |
|
Minor update: |
|
Well... I got back to this. On unix systems, this should default to forward slashes, I'll have to search for that logic somewhere.... |
I've been staring at this result through WSL and I have absolutely no idea what in the universe is causing omake to only run the first time, not the 2nd. |
|
so i don't know anything about linux, but, is there some possibility that the first run doesn't clean something up? Maybe Leaves a handle to the file being spawned open or something? |
|
Hmmm, re-looking with more info this might be an artifact of posix_spawn_file_actions_addclose only performing a |
|
I figured it out, I forgot to actually fill in the pipe by calling |
…'t point out where EBADF would appear from that, whoops.
|
Current status with the new build: |
|
You know, if you want to put the updated omake to some serious test, there's #168 for that ;-) |
|
I'll put it on my list for later this week, to recheck the operation under windows.... then maybe we can roll it into master... |
|
I'm actually still working on a few issues with omake running on the OrangeC repo, notably, if I try to run it, it will try calling |
|
What about just setting it to a "draft" until you considered this goal to be reached? |
|
Well, since I'm not the repository owner, and this was changed from draft by the repo owner, I can't exactly do that anymore, thanks github. |
|
yeah sorry about that. I pressed a button that did nothing like what I was expecting it to do... sigh.... |
|
Ah, I've figured out why it's running the way it does, on Linux systems oftentimes |
|
Ok, so there's STILL an issue with make on linux: This jobserver implementation was tested a long time ago and almost certainly requires re-testing, but I will not touch the command line parsing since that's... a bit fragile for my tastes. |
|
Once I finish cleanup of #810 I'm going to return to work here, I think I can pretty decisively make some progress overall since I think 99% of the error right now is just "Get the switch-parser to parse stuff with commas" and that should unlock the "You have forgotten errors with Linux pipes" issue... At least that I tested (once).... |
Full linux build still doesn't work yet.
|
I've gotten a bit further, I'm doing |
|
i don't know why that wouldn't work. I'll try it after a while. Right now I found that the changes you made OCCPARSE, without anychanges for anything else, cause BUILD.BAT to fail. I'm trying to diagnose it......... |
|
so the problem with the crash in omake is the change you made to templateinst.cpp. Problem is I'm looking at it and I'm agreeing that you deleting the code you deleted there is probably correct, however, when it is removed it caused other problems.... I'll look into it some more tomorrow.... |
|
@chuggafan... i updated the master branch with your compiler fixes and an additional fix that fixed a problem when compiling the omake on the master branch.... |
|
I merged with the |
|
@chuggafan, hehehe.... usually these days i don't much have to deal with syntax errors out of the linker.... the 'hashing' branch is several days old, I've made improvements since then. I've compiled the entire suite with it several times now (but not your latest omake) I just pushed it if you want to see if that resolves anything... I'm planning on merging it soon but I think I'm gonna ask if I can get more time on appveyor first... I was noticing, you put made some fprintfs in expr.cpp for what appear to be diagnostics... usually for diagnostics I use the DIAG() macro. That collects them as they happen and then gives you a summary if there are any that happened, you can use the /C+d compiler switch to dump them after everything else is done... |
|
Ah, for the expr.cpp are you talking about the "Candidate '%s' found at....." messages? Those are actually deliberately NOT DIAGs, those are meant to be additional information for the error. It's about getting more information to know exactly what the location is, it's very related to the style of update that I am thinking of for #1104 |
|
Not sure exactly which change did it, but NORMALcompiles.bat now functions properly and runs until success. |
Also change to printerr for some things.
|
We now build on linux and run! I can retarget this to my (actual) base at this point, which is the hashing branch, or we can merge with master. Woohoo! |
|
as far as the fprintfs.... for things that are outside the 'normal' scope I've been putting extra %s in the error message then wrting a custom function for that one error message (or class of error messages). The 'is is not convertible to' error message is done that way for example, it has two types in the message.... I decided we had enough testing on the hashing branch now, and merged it with 'master'..... tomorrow I can accept your changes if you are up for that! Merry Christmas! |
|
Yhea, I had forgotten about "Notes" entirely, and am converting all of the So I'm going to do that next, hopefully speeding up that since that will put us in modern memcpy territory on all CPUs made after 2012 can dumb us below the build limit threshold. |
|
I've pushed my changes ;) This file now produces the following errors & notes: #include <string>
std::string operator"" _s(const char* s, size_t sz) { return std::string(s, sz); }
std::string operator"" _s3(unsigned long long int val) { return std::to_string(val); }
int main()
{
std::string s = 12_s;
std::string s2 = 12_s2;
std::string s3 = 1.2_s3;
return 0;
} |
|
so i don't know about the compiler generating file errors as I haven't seen it, however, I pulled the latest from this PR, built omake with MSVC, then copied the built omake into \orangec\src, then i typed 'omake clean' and got this. omake on the master branch doesn't do this. I also tried to run BUILD.BAT, mostly it worked to build the compiler but similar problems did cause it to not build the runtime library every time it tried to build it. probably NUL is getting lost somehow on the lines where it doesn't work... |
|
Hmmmm.... that's weird. I think I figured it out though: I needed to add |
|
thanks for the quick response. I'll check it out later today and maybe we can get this accepted :) yeah i dunno what to do about things like memcpy since it isn't practical to go for super-intensive cpu-specific changes. Thanks for looking at it! Thank you for looking at those printfs as well.... |
|
Yhea, my plan for memcpy is to just use 9 cycles per operation could mean that @4gHz processor core speeds, we get ~444MB/s speeds for just using I would highly recommend taking a look at the "Intel 64 and IA-32 Architectures Optimization Reference Manual Volume 1" figure 3-3, table 3-5, and table 3-6. |
|
so it seems fine now, other than that the compile of the RTL is much more choppy than it was (it may take a little longer). I'm going to accept it though and we can deal with the choppiness at some other time.... |
3 participants
This is a prototype with some basic changes for linux, this DOES NOT CURRENTLY WORK, and I know this because I haven't tested anything to do with actually capturing process output, this is just some fill-in from before to get some eyeballs on this.
The main thing stopping me from getting anywhere is my complete inability to understand how the dup2 stuff works, and all of the pipe issues, which gets me lost immediately upon trying to understand it, I'm at the point of trying to understand how glibc implements popen and it has also gotten me absolutely nowhere...
@GitMensch I assume you have the best eyes to know what in the universe POSIX is supposed to be doing and if I'm on the right track here, because I sure know I don't.