Refactor git command stdio pipe by wxiaoguang · Pull Request #36422 · go-gitea/gitea

wxiaoguang · 2026-01-21T07:41:00Z

Almost the final round, the interfaces should be stable now.

Most potential deadlock problems should have been fixed, and new code is unlikely to cause new problems with the new design.

Also raise the minimum Git version required to 2.6.0 (its release date is 2015)

Copilot

Pull request overview

This PR refactors how git command stdio is wired and piped throughout the codebase to reduce deadlock risk, consolidates around the new gitcmd pipe APIs, and raises the minimum supported Git version to 2.6.0. It also adjusts various LFS- and diff-related pipelines to use the new abstractions and removes now-unneeded third-party buffering libraries.

Changes:

Introduces new gitcmd pipe abstractions (MakeStdinPipe, MakeStdoutPipe, buffer-based stdout, legacy stdin/stdout) and updates many call sites to use them instead of manual io.Pipe / os pipes.
Refactors LFS scanning and related pipelines (both gogit and non-gogit) to use errgroup and the new pipe API, simplifying error propagation and channel handling.
Updates higher-level services (pull requests, diffs, archives, wiki, temporary repos, hooks, etc.) to the new stdio model and bumps the required Git version to 2.6.0.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/integration/pull_merge_test.go	Adapts tests to new `RunStdString` + stdin-bytes API for hash/commit creation.
services/wiki/wiki.go	Switches wiki page hashing to new `HashObjectBytes` helper.
services/repository/gitgraph/graph.go	Moves commit graph generation to `MakeStdoutPipe`-based pipeline.
services/repository/files/temp_repo.go	Updates temp-repo file operations to use buffer/stdin-bytes and new diff-index pipeline.
services/repository/files/patch.go	Uses `WithStdinBytes` for applying patches via `git apply`.
services/repository/contributors_graph.go	Refactors contributor stats to use a stdout pipe instead of `io.ReadCloser` indirection.
services/pull/update_rebase.go	Uses buffer-based stdout in PR update-by-rebase logic.
services/pull/temp_repo.go	Changes PR temp-repo context to use `bytes.Buffer` and `WithStdoutBuffer`.
services/pull/review.go	Documents a remaining `io.Pipe` usage with a FIXME about `os.Pipe`.
services/pull/pull.go	Uses `MakeStdoutPipe` + pipeline for `git diff --name-only` in PR content-change checks.
services/pull/patch_unmerged.go	Refactors unmerged `ls-files` reader to a `MakeStdoutPipe`-based pipeline.
services/pull/patch.go	Switches conflict-checking `git apply --check` to the new pipe API (with a regression noted in stderr handling).
services/pull/merge_prepare.go	Adopts `MakeStdoutPipe` for `diff-tree` and buffer-based stdout in merge prep.
services/pull/lfs.go	Reworks LFSPush pipeline to use `errgroup` and gitcmd pipes; rewrites cat-file LFS meta-object creation (with EOF handling issues).
services/mirror/mirror_push.go	Adapts mirror LFS push to new `SearchPointerBlobs` signature and error channel contract.
services/migrations/gitea_uploader.go	Adds FIXME for `io.Pipe` in reviews similar to PR review code.
services/gitdiff/gitdiff.go	Switches diff generation to `MakeStdoutPipe` instead of `io.Pipe`.
routers/web/repo/setting/lfs.go	Updates LFS pointer-file listing to new `SearchPointerBlobs` API and error handling.
routers/web/repo/githttp.go	Uses legacy stdin/stdout wrappers for HTTP git service RPC.
routers/private/hook_verification.go	Uses stdout pipes + pipeline funcs for commit verification.
modules/repository/repo.go	Updates repo-level LFS object storage to new `SearchPointerBlobs` contract.
modules/lfs/pointer_scanner_nogogit.go	Completely refactors pointer scanning to cat-file pipelines + errgroup and new pipe helpers.
modules/lfs/pointer_scanner_gogit.go	Simplifies gogit pointer scanning to return error directly and close channels in-function.
modules/gitrepo/diff.go	Uses legacy stdout writer wrapper in reverse diff generation.
modules/gitrepo/commit_file.go	Refactors commit-file status reading to use `MakeStdoutPipe` + goroutine parser.
modules/gitrepo/blame.go	Simplifies blame reader to use gitcmd stdout pipes instead of `os.Pipe`.
modules/gitrepo/archive.go	Uses legacy stdout writer wrapper for `git archive`.
modules/git/submodule.go	Refactors template submodule commit listing to stdout pipe + scanner.
modules/git/repo_tree.go	Uses stdin-bytes for commit-tree commit messages.
modules/git/repo_tag.go	Reworks tag listing to use `MakeStdoutPipe` and a pipeline func; paginates and counts tags within the pipeline.
modules/git/repo_stats.go	Refactors code-activity stats to use stdout pipe scanning.
modules/git/repo_ref_nogogit.go	Simplifies ref listing to a `for-each-ref` stdout pipe + inline scanner.
modules/git/repo_object.go	Replaces reader-based hashing with `HashObjectBytes` and uses it for object-format detection.
modules/git/repo_index.go	Uses stdin-bytes for index updates.
modules/git/repo_compare.go	Switches diff output to legacy stdout writer wrapper.
modules/git/repo_commit.go	Refactors `CommitsByFileAndRange` to use `MakeStdoutPipe` + pipeline-based reading.
modules/git/repo_branch_nogogit.go	Reimplements `WalkShowRef` using `for-each-ref` + stdout pipe and pipeline func.
modules/git/repo_archive.go	Removes an unused repo-level archive helper in favor of `modules/gitrepo` equivalents.
modules/git/pipeline/revlist.go	Simplifies rev-list and blob selection to use gitcmd `Command` plus pipe-based in/out.
modules/git/pipeline/namerev.go	Introduces `fillResultNameRev` using git `name-rev` and gitcmd stdin/stdout pipes.
modules/git/pipeline/lfs_nogogit.go	Refactors non-gogit LFS file discovery to use stdout pipes, cat-file batches, and `fillResultNameRev`.
modules/git/pipeline/lfs_gogit.go	Aligns gogit LFS result naming with `fillResultNameRev` and clarifies error messages.
modules/git/pipeline/lfs_common.go	Removes the now-unneeded `lfsError` helper.
modules/git/pipeline/catfile.go	Refactors cat-file batch helpers to use command-based pipes instead of explicit `io.Pipe`.
modules/git/log_name_status.go	Replaces third-party nio buffered pipes with gitcmd stdout pipes and context-based teardown.
modules/git/grep.go	Reworks grep search to use stdout pipe and pipeline func for streaming results.
modules/git/gitcmd/pipe.go	Introduces internal pipe and buffer interfaces/wrappers used by gitcmd.Command.
modules/git/gitcmd/command.go	Overhauls gitcmd.Command stdio handling, adds pipe helpers, and inlines LC_ALL default.
modules/git/git.go	Raises the minimum required git version to 2.6.0.
modules/git/diff.go	Uses legacy stdout writer wrapper in raw diff helpers.
modules/git/catfile_batch_reader.go	Refactors cat-file batch communicator to use gitcmd pipes and managed cancellation.
modules/git/attribute/checker.go	Switches attribute checking to RunStdBytes with stdin-bytes.
modules/git/attribute/batch.go	Adapts attribute batch checker to gitcmd stdin pipes and legacy stdout writer.
go.mod / go.sum / assets/go-licenses.json	Removes unused buffer/nio dependencies and their license entries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

services/pull/patch.go

services/pull/lfs.go

modules/git/gitcmd/command.go

Copilot

Pull request overview

Copilot reviewed 61 out of 62 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

modules/git/attribute/batch.go

# This is the 1st commit message: feat: Add max-parallel implementation inside the Gitea server # This is the commit message go-gitea#2: fix: Remove MatrixID and Capacity functionality # This is the commit message go-gitea#3: Fix incorrect text content detection (go-gitea#36364) Fix go-gitea#36325 # This is the commit message go-gitea#4: clean watches when make a repository private and check permission when send release emails (go-gitea#36319) # This is the commit message go-gitea#5: Fix bug when compare in the pull request (go-gitea#36363) The pull request comparison should not use `direct compare`. # This is the commit message go-gitea#6: Fix permission check on org project operations (go-gitea#36318) # This is the commit message go-gitea#7: Release attachments must belong to the intended repo (go-gitea#36347) # This is the commit message go-gitea#8: Indicate when only optional checks failed (go-gitea#36367) Currently it's not clear that you can merge a PR when only optional checks failed: <img width="922" height="447" alt="Screenshot 2026-01-14 at 4 08 17 pm" src="https://github.com/user-attachments/assets/e11670c7-5ab9-42d7-af09-2d8a8fd532d3" /> This PR changes the text to say "Some optional checks failed" when only optional checks failed: <img width="922" height="443" alt="Screenshot 2026-01-14 at 3 59 08 pm" src="https://github.com/user-attachments/assets/9ea69b13-38d6-4cfc-b4f7-952eff58e546" /> When a required check fails it'll still say "Some checks failed": <img width="928" height="343" alt="Screenshot 2026-01-14 at 3 59 20 pm" src="https://github.com/user-attachments/assets/d3764a95-9737-4482-851e-d3406b1e4d76" /> --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> # This is the commit message go-gitea#9: Rename CSS variables and improve colorblind themes (go-gitea#36353) Followup go-gitea#36215, rename the variables for consistency with existing vars and change green to value of `--color-blue` in the relevant color blind themes: <img width="1305" height="303" alt="image" src="https://github.com/user-attachments/assets/3d131ab7-99ab-4b03-93ab-715ce0030b08" /> The blue coloring also matched GitHub: <img width="1313" height="393" alt="image" src="https://github.com/user-attachments/assets/f97e35b2-4ff4-49b0-841f-ffd49a02e03d" /> --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> # This is the commit message go-gitea#10: Migrate to `import.meta.env` and clean up types and eslint (go-gitea#36362) `import.meta.env` is supported in both vitest and webpack [as of recent](webpack/webpack#19996), so replace all previous use of `process.env` with it. Current usage is limited to test files, I've also verified it works in actual frontend code. `webpack/module` is added to typescript types which includes the definition for `import.meta.env`. I've also made the eslint globals more precise. Finally, `__webpack_public_path__` is removed from our type definitions because `webpack/module` also provides it. # This is the commit message go-gitea#11: Add chunked transfer encoding support for LFS uploads (go-gitea#36380) Enable chunked transfer encoding for Git LFS uploads by adding Transfer-Encoding: chunked header to upload action responses. This prevents large file uploads (100+ MB) from being blocked by reverse proxies like Cloudflare that buffer non-chunked requests. Fix go-gitea#22233 --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> # This is the commit message go-gitea#12: Fix bug on notification read (go-gitea#36339) When a user has been revoked permission to access a repository, the related notification could still be visited. But the repository's information should not be leaked any more. # This is the commit message go-gitea#13: Add ability to download subpath archive (go-gitea#36371) closes: go-gitea#4478 --------- Signed-off-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> # This is the commit message go-gitea#14: Update JS and PY deps (go-gitea#36383) - Update JS and PY dependencies - Workaround stylelint/stylelint#8893 by moving the stylint config file to JS - Regenerate SVGs - Bump to python 3.14 in devcontainer and actions - Verified `@github/text-expander-element` - Removed obsolete type stub # This is the commit message go-gitea#15: Fix CODEOWNERS review request attribution using comment metadata (go-gitea#36348) Fixes go-gitea#36333 ## Problem When CODEOWNERS automatically assigns reviewers to a pull request, the timeline incorrectly shows the PR author as the one who requested the review (e.g., "PR_AUTHOR requested review from CODE_OWNER"). This is misleading since the action was triggered automatically by CODEOWNERS rules, not by the PR author. ## Solution Store CODEOWNERS attribution in comment metadata instead of changing the doer user: - Add `SpecialDoerName` field to `CommentMetaData` struct (value: `"CODEOWNERS"` for CODEOWNERS-triggered requests) - Pass `isCodeOwners=true` to `AddReviewRequest` and `AddTeamReviewRequest` functions - Template can check this metadata to show appropriate attribution message --------- Signed-off-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> # This is the commit message go-gitea#16: Hide delete directory button for mirror or archive repository and disable the menu item if user have no permission (go-gitea#36384) # This is the commit message go-gitea#17: add support for archive-upload rpc (go-gitea#36391) Add support for fetching archives with `git archive --remote <repo-url>` closes: go-gitea#23425 --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> # This is the commit message go-gitea#18: fix: typos in comments (go-gitea#36394) # This is the commit message go-gitea#19: Refactor git command stdio pipe (go-gitea#36393) And remove the incorrect `ensureValidGitRepository` # This is the commit message go-gitea#20: Some refactors about GetMergeBase (go-gitea#36186) Maybe fix go-gitea#32018 - Use `gitrepo.GetMergeBase` method instead of other two implementations. - Add `FetchRemoteCommit` so that we don't need to add many `remote` to the git repository to avoid possible git lock conflicts. A lock will start when invoke the function, it will be invoked when cross-repository comparing. The head repository will fetch the base repository's base commit id. In most situations, it should lock the fork repositories so that it should not become a bottleneck. - Improve `GetCompareInfo` to remove unnecessarily adding remote. - Remove unnecessary parameters of `SignMerge`. # This is the commit message go-gitea#21: Refactor git command stderr handling (go-gitea#36402) And clean up legacy fragile & incorrect logic # This is the commit message go-gitea#22: [skip ci] Updated translations via Crowdin # This is the commit message go-gitea#23: Fix missing repository id when migrating release attachments (go-gitea#36389) This PR fixes missed repo_id on the migration of attachments to Gitea. It also provides a doctor check to fix the dirty data on the database. Refactor git command context & pipeline (go-gitea#36406) Less and simpler code, fewer bugs Allow foreachref parse max tokens from 4*64KB to 4MB (go-gitea#36414) Fix go-gitea#36408 --------- Signed-off-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Update material-icon-theme to v5.31.0 (go-gitea#36427) Update chroma to v2.23.0 (go-gitea#36423) Fix markdown newline handling during IME composition (go-gitea#36421) ### Summary Fix incorrect newline handling in markdown editor when using IME input. ### Details While composing text with an IME, pressing Enter should not trigger markdown indentation logic. This change skips indentation handling during composition by checking `e.isComposing`. This prevents unexpected line breaks and formatting issues for CJK users. [skip ci] Updated translations via Crowdin Fix typos: unknow -> unknown, pktLineTypeUnknow -> pktLineTypeUnknown (go-gitea#36419) Fix issue filter menu layout (go-gitea#36426) Fix go-gitea#36420 Fix spelling (go-gitea#36399) Signed-off-by: Thomas Beutlich <115483027+thbeu@users.noreply.github.com> Refactor git command stdio pipe (go-gitea#36422) Most potential deadlock problems should have been fixed, and new code is unlikely to cause new problems with the new design. Also raise the minimum Git version required to 2.6.0 (released in 2015) Remove `node-check` and `go-check`, support node prerelease versions (go-gitea#36382) 1. Remove those checks for the sake of build performance and because go and node will fail anyways if their versions are incorrect. 3. Support pre-release Node version for determining NODE_VARS. 2. Update to the chinese READMEs to mention `pnpm` which is already present in english README. --------- Co-authored-by: techknowlogick <techknowlogick@gitea.com> Co-authored-by: Giteabot <teabot@gitea.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> fix: Adjust the unittests fix: Lint issues fix: Adjust the swagger config

* giteaofficial/main: Remove `node-check` and `go-check`, support node prerelease versions (go-gitea#36382) Refactor git command stdio pipe (go-gitea#36422) Fix spelling (go-gitea#36399) Fix issue filter menu layout (go-gitea#36426) Fix typos: unknow -> unknown, pktLineTypeUnknow -> pktLineTypeUnknown (go-gitea#36419) [skip ci] Updated translations via Crowdin Fix markdown newline handling during IME composition (go-gitea#36421) Update chroma to v2.23.0 (go-gitea#36423) Update material-icon-theme to v5.31.0 (go-gitea#36427)

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jan 21, 2026

github-actions bot added the modifies/go Pull requests that update Go code label Jan 21, 2026

wxiaoguang force-pushed the refactor-git-pipe branch 2 times, most recently from bb940d8 to 2c74550 Compare January 21, 2026 08:13

github-actions bot added the modifies/dependencies label Jan 21, 2026

fix

f493d0e

wxiaoguang force-pushed the refactor-git-pipe branch from 2c74550 to f493d0e Compare January 21, 2026 09:39

wxiaoguang requested a review from Copilot January 21, 2026 09:46

Copilot started reviewing on behalf of wxiaoguang January 21, 2026 09:47 View session

Copilot AI reviewed Jan 21, 2026

View reviewed changes

services/pull/patch.go Outdated Show resolved Hide resolved

services/pull/lfs.go Show resolved Hide resolved

modules/git/gitcmd/command.go Show resolved Hide resolved

fix

17f8351

wxiaoguang requested a review from Copilot January 21, 2026 10:19

Copilot started reviewing on behalf of wxiaoguang January 21, 2026 10:19 View session

This comment was marked as resolved.

Sign in to view

fix

0758c8f

wxiaoguang requested a review from Copilot January 21, 2026 10:58

Copilot started reviewing on behalf of wxiaoguang January 21, 2026 10:58 View session

This comment was marked as resolved.

Sign in to view

wxiaoguang requested a review from Copilot January 21, 2026 11:22

Copilot started reviewing on behalf of wxiaoguang January 21, 2026 11:22 View session

fine tune

50ad336

wxiaoguang force-pushed the refactor-git-pipe branch from e948399 to 50ad336 Compare January 21, 2026 11:23

wxiaoguang added the type/refactoring Existing code has been cleaned up. There should be no new functionality. label Jan 21, 2026

wxiaoguang added this to the 1.26.0 milestone Jan 21, 2026

Copilot AI reviewed Jan 21, 2026

View reviewed changes

wxiaoguang added 2 commits January 21, 2026 20:00

fine tune

04f77fd

fix test

11f2f3f

wxiaoguang force-pushed the refactor-git-pipe branch from a13cc37 to 11f2f3f Compare January 21, 2026 17:36

wxiaoguang mentioned this pull request Jan 21, 2026

Refactor git command context & pipeline #36406

Merged

lunny reviewed Jan 21, 2026

View reviewed changes

modules/git/attribute/batch.go Show resolved Hide resolved

lunny approved these changes Jan 22, 2026

View reviewed changes

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Jan 22, 2026

wxiaoguang mentioned this pull request Jan 22, 2026

Remove node-check and go-check, support node prerelease versions #36382

Merged

silverwind approved these changes Jan 22, 2026

View reviewed changes

GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Jan 22, 2026

Merge branch 'main' into refactor-git-pipe

79809ff

wxiaoguang enabled auto-merge (squash) January 22, 2026 05:34

wxiaoguang merged commit 3a09d7a into go-gitea:main Jan 22, 2026
24 checks passed

wxiaoguang deleted the refactor-git-pipe branch January 22, 2026 06:04

tototomate123 mentioned this pull request Jan 22, 2026

Allow configuring default PR base branch (fixes #36412) #36425

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comments

Refactor git command stdio pipe#36422

Refactor git command stdio pipe#36422
wxiaoguang merged 7 commits intogo-gitea:mainfrom
wxiaoguang:refactor-git-pipe

wxiaoguang commented Jan 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Comments

Conversation

wxiaoguang commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wxiaoguang commented Jan 21, 2026 •

edited

Loading