-
Notifications
You must be signed in to change notification settings - Fork 7
fix: make app failed if a container is stopped #116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
9be4ebb to
1bfe213
Compare
dc07a7f to
09691b6
Compare
|
we should think of some test for this |
| containerState: []container.ContainerState{container.StateRunning, container.StateDead, container.StateRemoving, container.StateRestarting, container.StateExited}, | ||
| want: StatusFailed, | ||
| }, | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should add a test for the new check
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created as status failed!
i've also added statusMessage string example inspired by some real cases
3db2a77 to
0243557
Compare
Co-authored-by: Luca Rinaldi <[email protected]>
0243557 to
2549af2
Compare
2549af2 to
a09d652
Compare
internal/orchestrator/helpers.go
Outdated
| continue | ||
| } | ||
| if slices.ContainsFunc(s, func(v containerStateInfo) bool { | ||
| return v.State == StatusStopped && strings.Contains(v.StatusMessage, "Exited (0)") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should account for all exit codes that aren't signals, so everything that is less than 128
If you write an app with Python-like
exit(1)You say that this isn't a fail, which in fact it is, because the app exits right after be started.
internal/orchestrator/helpers.go
Outdated
| } | ||
| appsStatusMap[appPath] = append(appsStatusMap[appPath], StatusFromDockerState(c.State)) | ||
| appsStatusMap[appPath] = append(appsStatusMap[appPath], containerStateInfo{ | ||
| State: StatusFromDockerState(c.State), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't you update this mapping function and don't account for the status message there? If a container exited with anything that isn't 128 + 9 we could consider that a failing state, because it means that the process exited without Docker killing it.
9b7128a to
fbcc770
Compare
fbcc770 to
fe0cced
Compare
internal/orchestrator/helpers.go
Outdated
| continue | ||
| } | ||
| if slices.ContainsFunc(s, func(v Status) bool { return v == StatusStopping }) { | ||
| if slices.ContainsFunc(s, func(v containerState) bool { return v.Status == StatusStopped && checkExitCode(v) }) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I said in a previous review
Why don't you update this mapping function and don't account for the status message there? If a container exited with anything that isn't 128 + 9 we could consider that a failing state, because it means that the process exited without Docker killing it.
I would like to move the parsing function checkExitCode to the StatusFromDockerState mapping function. I think it is clearer to have a single function for mapping docker container status to our app status, instead of adding a special check here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've finally change it to a check bool function, that parse and check that the code is grater than 128, so we now if it was killed or it failed
internal/orchestrator/status.go
Outdated
| return []Status{StatusStarting, StatusRunning, StatusStopping, StatusStopped, StatusFailed} | ||
| } | ||
|
|
||
| func checkExitCode(statusMessage string) bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure which is the meaning of checkExitCode. I would rename this function to
| func checkExitCode(statusMessage string) bool { | |
| func parseExitCode(statusMessage string) (int, error) { |
And use it above in the following way
if code, err := parseExitCode(statusMessage); err != nil {
slog.Warn("Failed to parse exit code from status message", slog.String("statusMessage", statusMessage), slog.String("error", err.Error())
} else if code >= 0 && code < 128 {
return StatusFailed
}
return StatusStopped
internal/orchestrator/status.go
Outdated
| slog.Error("Failed to parse exit code from status message", slog.String("statusMessage", statusMessage), slog.String("error", err.Error())) | ||
| return false | ||
| } | ||
| if exitCode >= 0 && exitCode < 128 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that 128 should be included, i.e., https://tldp.org/LDP/abs/html/exitcodes.html. If we want to consider a failing status, everything that isn't a signal that should include 128, signals start from 128+n, and the signal number cannot be 0.
339d18c to
09a7e8b
Compare
09a7e8b to
acba544
Compare
Motivation
If at least a container of an application is stopped due to any reason, the application status should be failed.
In this way even if other containers are up and running the state of the application state is set as failed, and the user can debug the failed container.