fix(graceful): prevent WaitGroup panic on Ctrl+C shutdown#35578
fix(graceful): prevent WaitGroup panic on Ctrl+C shutdown#35578ita004 wants to merge 2 commits intogo-gitea:mainfrom
Conversation
…repeated shutdowns Fixes go-gitea#35559 Signed-off-by: ita004 <shfahmd001@gmail.com>
|
I don't think I did twice Ctrl+C in #35559 |
yes, I also reproduced the panic with a single Ctrl+C.. the shutdown flow triggers multiple paths that wait on the same WaitGroup, which causes the reuse panic.... this fix ensures wg.Wait() is only called once, so shutdown completes cleanly without the panic. |
|
This PR's change is just one more patch to the existing messy code (from Restore Graceful Restarting & Socket Activation (#7274)). The root problem is that the "WaitGroup" is abused. Although @zeripath ever complained that I rewrote his code, but for this case, I think we still need to rewrite the code with a correct design to fix the root problem, just like other complete fixes: logger (#24726), queue (#24505), diff (#19958), etc. It shouldn't add more hacky and fragile patches like #11219. |
thanks for the detailed feedback, that makes sense... |
|
FYI: there are some related fixes like "Refactor graceful manager, fix misused WaitGroup #29738". |
… reject new work during shutdown
| // Shutdown the waitgroup to prevent new connections | ||
| // and wait for existing connections to finish | ||
| srv.wg.Shutdown() |
There was a problem hiding this comment.
I don't think the current design would work or it can be improved.
doHammer means that it will stop the server immediately without waiting for any running tasks (for example: existing user request connection).
The graceful package's design overall looks problematic and it seems unfixable to me.
There was a problem hiding this comment.
thanks for clarifying, i understand what you mean now... that makes sense: doHammer should remain immediate and not wait for any running connections, just forcefully stop the server....
i'll keep this note in mind, and if there’s a future plan to refactor or redesign the graceful manager to address the deeper issues, i'd be happy to participate or help with that whenever it’s appropriate...
|
The rewritten: |
This PR fixes a panic issue in the WaitGroup that occurs when Gitea is shut down using Ctrl+C. It ensures that all active connection pointers in the server are properly tracked and forcibly closed when the hammer shutdown is invoked. The process remains graceful — the normal shutdown sequence runs before the hammer is triggered, and existing connections are given a timeout period to complete gracefully. This PR also fixes `no logger writer` problem. Now the log close will only be invoked when the command exit. - Fixes #35468 - Fixes #35551 - Fixes #35559 - Replace #35578 --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
What
Guard the
wg.Wait()call usingsync.Onceto avoid"sync: WaitGroup is reused before previous Wait has returned"panic during shutdown.Why
Multiple shutdown paths could cause
Wait()to be invoked more than once,which led to a negative WaitGroup counter and panic when pressing Ctrl+C.
How tested
make backend)./gitea weband triggered Ctrl+C — server shuts down cleanly (no panic)Fixes #35559