FAILED: Rebase shears/main (#23521115901)#65
Draft
gitforwindowshelper[bot] wants to merge 182 commits intobase/shears/main-23521115901from
Draft
FAILED: Rebase shears/main (#23521115901)#65gitforwindowshelper[bot] wants to merge 182 commits intobase/shears/main-23521115901from
gitforwindowshelper[bot] wants to merge 182 commits intobase/shears/main-23521115901from
Conversation
The output of `git clone` is a vital component for understanding what has happened when things go wrong. However, these logs are partially under the control of the remote server (via the "sideband", which typically contains what the remote `git pack-objects` process sends to `stderr`), and is currently not sanitized by Git. This makes Git susceptible to ANSI escape sequence injection (see CWE-150, https://cwe.mitre.org/data/definitions/150.html), which allows attackers to corrupt terminal state, to hide information, and even to insert characters into the input buffer (i.e. as if the user had typed those characters). To plug this vulnerability, disallow any control character in the sideband, replacing them instead with the common `^<letter/symbol>` (e.g. `^[` for `\x1b`, `^A` for `\x01`). There is likely a need for more fine-grained controls instead of using a "heavy hammer" like this, which will be introduced subsequently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The preceding commit fixed the vulnerability whereas sideband messages (that are under the control of the remote server) could contain ANSI escape sequences that would be sent to the terminal verbatim. However, this fix may not be desirable under all circumstances, e.g. when remote servers deliberately add coloring to their messages to increase their urgency. To help with those use cases, give users a way to opt-out of the protections: `sideband.allowControlCharacters`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This happens only when the corresponding commits are not exported in the current fast-export run. This can happen either when the relevant commit is already marked, or when the commit is explicitly marked as UNINTERESTING with a negative ref by another argument. This breaks fast-export basec remote helpers. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
The preceding two commits introduced special handling of the sideband channel to neutralize ANSI escape sequences before sending the payload to the terminal, and `sideband.allowControlCharacters` to override that behavior. However, some `pre-receive` hooks that are actively used in practice want to color their messages and therefore rely on the fact that Git passes them through to the terminal. In contrast to other ANSI escape sequences, it is highly unlikely that coloring sequences can be essential tools in attack vectors that mislead Git users e.g. by hiding crucial information. Therefore we can have both: Continue to allow ANSI coloring sequences to be passed to the terminal, and neutralize all other ANSI escape sequences. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When a Unix socket is initialized, the current directory's path is stored so that the cleanup code can `chdir()` back to where it was before exit. If the path that needs to be stored exceeds the default size of the `sun_path` attribute of `struct sockaddr_un` (which is defined as a 108-sized byte array on Linux), a larger buffer needs to be allocated so that it can hold the path, and it is the responsibility of the `unix_sockaddr_cleanup()` function to release that allocated memory. In Git's CI, this stack allocation is not necessary because the code is checked out to `/home/runner/work/git/git`. Concatenate the path `t/trash directory.t0301-credential-cache/.cache/git/credential/socket` and a terminating NUL, and you end up with 96 bytes, 12 shy of the default `sun_path` size. However, I use worktrees with slightly longer paths: `/home/me/projects/git/yes/i/nest/worktrees/to/organize/them/` is more in line with what I have. When I recently tried to locally reproduce a failure of the `linux-leaks` CI job, this t0301 test failed (where it had not failed in CI). The reason: When `credential-cache` tries to reach its daemon initially by calling `unix_sockaddr_init()`, it is expected that the daemon cannot be reached (the idea is to spin up the daemon in that case and try again). However, when this first call to `unix_sockaddr_init()` fails, the code returns early from the `unix_stream_connect()` function _without_ giving the cleanup code a chance to run, skipping the deallocation of above-mentioned path. The fix is easy: do not return early but instead go directly to the cleanup code. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[PT: ensure we add an additional element to the argv array] Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In some implementations, `regexec_buf()` assumes that it is fed lines; Without `REG_NOTEOL` it thinks the end of the buffer is the end of a line. Which makes sense, but trips up this case because we are not feeding lines, but rather a whole buffer. So the final newline is not the start of an empty line, but the true end of the buffer. This causes an interesting bug: $ echo content >file.txt $ git grep --no-index -n '^$' file.txt file.txt:2: This bug is fixed by making the end of the buffer consistently the end of the final line. The patch was applied from https://lore.kernel.org/git/20250113062601.GD767856@coredump.intra.peff.net/ Reported-by: Olly Betts <olly@survex.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This addresses: - CVE-2024-52005: Insufficient neutralization of ANSI escape sequences in sideband payload can be used to mislead Git users into believing that certain remote-generated messages actually originate from Git. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
These fixes have been sent to the Git mailing list but have not been picked up by the Git project yet. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When asking the mingw-w64 variant of GCC to compile C11 code, it seems to link implicitly to libwinpthread, which does implement a pthread emulation (that is more complete than Git's). In preparation for vendoring in mimalloc (which requires C11 support), let's keep preferring Git's own pthread emulation. To avoid linker errors where it thinks that the `pthread_self` and the `pthread_create` symbols are defined twice, let's give our version a `win32_` prefix, just like we already do for `pthread_join()`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On LLP64 systems, such as Windows, the size of `long`, `int`, etc. is only 32 bits (for backward compatibility). Git's use of `unsigned long` for file memory sizes in many places, rather than size_t, limits the handling of large files on LLP64 systems (commonly given as `>4GB`). Provide a minimum test for handling a >4GB file. The `hash-object` command, with the `--literally` and without `-w` option avoids writing the object, either loose or packed. This avoids the code paths hitting the `bigFileThreshold` config test code, the zlib code, and the pack code. Subsequent patches will walk the test's call chain, converting types to `size_t` (which is larger in LLP64 data models) where appropriate. Signed-off-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We are about to vendor in `mimalloc`'s source code which we will want to include `compat/posix.h` after defining that constant. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Co-authored-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Ian Bearman <ianb@microsoft.com> Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Continue walking the code path for the >4GB `hash-object --literally` test. The `hash_object_file_literally()` function internally uses both `hash_object_file()` and `write_object_file_prepare()`. Both function signatures use `unsigned long` rather than `size_t` for the mem buffer sizes. Use `size_t` instead, for LLP64 compatibility. While at it, convert those function's object's header buffer length to `size_t` for consistency. The value is already upcast to `uintmax_t` for print format compatibility. Note: The hash-object test still does not pass. A subsequent commit continues to walk the call tree's lower level hash functions to identify further fixes. Signed-off-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Update to newer mimalloc versions like this:
update_mimalloc ()
{
test $# = 1 || {
echo "Need a mimalloc version" 1>&2;
return 1
};
for oneline in 'mimalloc: adjust for building inside Git' 'Import the source code of mimalloc';
do
git revert -n HEAD^{/^"$oneline"} && git checkout HEAD -- Makefile && git commit -sm "Temporarily revert \"$oneline\"" -m 'In preparation for upgrading to a newer mimalloc version.' || return 1;
done;
for file in $(git show --format='%n' --name-only --diff-filter=A HEAD^{/^"Import the source code of mimalloc "}) compat/mimalloc/arena-abandon.c compat/mimalloc/free.c compat/mimalloc/libc.c compat/mimalloc/prim/prim.c compat/mimalloc/mimalloc-stats.h;
do
file2=${file#compat/mimalloc/};
case "$file2" in
segment-cache.c)
: no longer needed;
continue
;;
bitmap.h | *.c)
file2=src/$file2
;;
*.h)
file2=include/$file2
;;
esac;
mkdir -p "${file%/*}" && git -C /usr/src/mimalloc/ show "$1":$file2 > "$file" && git add "$file" || {
echo "Failed: $file2 -> $file" 1>&2;
return 1
};
done;
conv_sed='sed -n "/^ *eval/d;/ /p"' && git commit -sm "Import the source code of mimalloc $1" -m "Update to newer mimalloc versions like this:" -m "$(set | sed -n '/^update_mimalloc *() *$/,/^}/{s/^./ &/;p}')" -m ' update_mimalloc $MIMALLOC_VERSION' -m 'For convenience, you can set `MIMALLOC_VERSION` and then run:' -m ' eval "$(git show -s <this-commit> | '"$conv_sed"')"' || return 1;
git cherry-pick HEAD^{/^'mimalloc: adjust for building inside Git'} || return 1
}
update_mimalloc $MIMALLOC_VERSION
For convenience, you can set `MIMALLOC_VERSION` and then run:
eval "$(git show -s <this-commit> | sed -n "/^ *eval/d;/ /p")"
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In this context, a "feature" is a dependency combined with its own dependencies. Signed-off-by: Ian Bearman <ianb@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Correct some wording and inform users regarding the Visual Studio changes (from V16.6) to the default generator. Subsequent commits ensure that Git for Windows can be directly opened in modern Visual Studio without needing special configuration of the CMakeLists settings. It appeares that internally Visual Studio creates it's own version of the .sln file (etc.) for extension tools that expect them. The large number of references below document the shifting of Visual Studio default and CMake setting options. refs: https://docs.microsoft.com/en-us/search/?scope=C%2B%2B&view=msvc-150&terms=Ninja 1. https://docs.microsoft.com/en-us/cpp/linux/cmake-linux-configure?view=msvc-160 (note the linux bit) "In Visual Studio 2019 version 16.6 or later ***, Ninja is the default generator for configurations targeting a remote system or WSL. For more information, see this post on the C++ Team Blog [https://devblogs.microsoft.com/cppblog/linux-development-with-visual-studio-first-class-support-for-gdbserver-improved-build-times-with-ninja-and-updates-to-the-connection-manager/]. For more information about these settings, see CMakeSettings.json reference [https://docs.microsoft.com/en-us/cpp/build/cmakesettings-reference?view=msvc-160]." 2. https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-160 "CMake supports two files that allow users to specify common configure, build, and test options and share them with others: CMakePresets.json and CMakeUserPresets.json." " Both files are supported in Visual Studio 2019 version 16.10 or later. ***" 3. https://devblogs.microsoft.com/cppblog/linux-development-with-visual-studio-first-class-support-for-gdbserver-improved-build-times-with-ninja-and-updates-to-the-connection-manager/ " Ninja has been the default generator (underlying build system) for CMake configurations targeting Windows for some time***, but in Visual Studio 2019 version 16.6 Preview 3*** we added support for Ninja on Linux." 4. https://docs.microsoft.com/en-us/cpp/build/cmakesettings-reference?view=msvc-160 " `generator`: specifies CMake generator to use for this configuration. May be one of: Visual Studio 2019 only: Visual Studio 16 2019 Visual Studio 16 2019 Win64 Visual Studio 16 2019 ARM Visual Studio 2017 and later: Visual Studio 15 2017 Visual Studio 15 2017 Win64 Visual Studio 15 2017 ARM Visual Studio 14 2015 Visual Studio 14 2015 Win64 Visual Studio 14 2015 ARM Unix Makefiles Ninja Because Ninja is designed for fast build speeds instead of flexibility and function, it is set as the default. However, some CMake projects may be unable to correctly build using Ninja. If this occurs, you can instruct CMake to generate Visual Studio projects instead. To specify a Visual Studio generator in Visual Studio 2017, open the settings editor from the main menu by choosing CMake | Change CMake Settings. Delete "Ninja" and type "V". This activates IntelliSense, which enables you to choose the generator you want." "To specify a Visual Studio generator in Visual Studio 2019, right-click on the CMakeLists.txt file in Solution Explorer and choose CMake Settings for project > Show Advanced Settings > CMake Generator. When the active configuration specifies a Visual Studio generator, by default MSBuild.exe is invoked with` -m -v:minimal` arguments." 5. https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-160#enable-cmakepresetsjson-integration-in-visual-studio-2019 "Enable CMakePresets.json integration in Visual Studio 2019 CMakePresets.json integration isn't enabled by default in Visual Studio 2019. You can enable it for all CMake projects in Tools > Options > CMake > General: (tick a box)" ... see more. 6. https://docs.microsoft.com/en-us/cpp/build/cmakesettings-reference?view=msvc-140 (whichever v140 is..) "CMake projects are supported in Visual Studio 2017 and later." 7. https://docs.microsoft.com/en-us/cpp/overview/what-s-new-for-cpp-2017?view=msvc-150 "Support added for the CMake Ninja generator." 8. https://docs.microsoft.com/en-us/cpp/overview/what-s-new-for-cpp-2017?view=msvc-150#cmake-support-via-open-folder "CMake support via Open Folder Visual Studio 2017 introduces support for using CMake projects without converting to MSBuild project files (.vcxproj). For more information, see CMake projects in Visual Studio[https://docs.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-150]. Opening CMake projects with Open Folder automatically configures the environment for C++ editing, building, and debugging." ... +more! 9. https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-160#supported-cmake-and-cmakepresetsjson-versions "Visual Studio reads and evaluates CMakePresets.json and CMakeUserPresets.json itself and doesn't invoke CMake directly with the --preset option. So, CMake version 3.20 or later isn't strictly required when you're building with CMakePresets.json inside Visual Studio. We recommend using CMake version 3.14 or later." 10. https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-160#enable-cmakepresetsjson-integration-in-visual-studio-2019 "If you don't want to enable CMakePresets.json integration for all CMake projects, you can enable CMakePresets.json integration for a single CMake project by adding a CMakePresets.json file to the root of the open folder. You must close and reopen the folder in Visual Studio to activate the integration. 11. https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-160#default-configure-presets ***(doesn't actually say which version..) "Default Configure Presets If no CMakePresets.json or CMakeUserPresets.json file exists, or if CMakePresets.json or CMakeUserPresets.json is invalid, Visual Studio will fall back*** on the following default Configure Presets: Windows example JSON { "name": "windows-default", "displayName": "Windows x64 Debug", "description": "Sets Ninja generator, compilers, x64 architecture, build and install directory, debug build type", "generator": "Ninja", "binaryDir": "${sourceDir}/out/build/${presetName}", "architecture": { "value": "x64", "strategy": "external" }, "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug", "CMAKE_INSTALL_PREFIX": "${sourceDir}/out/install/${presetName}" }, "vendor": { "microsoft.com/VisualStudioSettings/CMake/1.0": { "hostOS": [ "Windows" ] } } }, " Signed-off-by: Philip Oakley <philipoakley@iee.email>
Continue walking the code path for the >4GB `hash-object --literally`
test to the hash algorithm step for LLP64 systems.
This patch lets the SHA1DC code use `size_t`, making it compatible with
LLP64 data models (as used e.g. by Windows).
The interested reader of this patch will note that we adjust the
signature of the `git_SHA1DCUpdate()` function without updating _any_
call site. This certainly puzzled at least one reviewer already, so here
is an explanation:
This function is never called directly, but always via the macro
`platform_SHA1_Update`, which is usually called via the macro
`git_SHA1_Update`. However, we never call `git_SHA1_Update()` directly
in `struct git_hash_algo`. Instead, we call `git_hash_sha1_update()`,
which is defined thusly:
static void git_hash_sha1_update(git_hash_ctx *ctx,
const void *data, size_t len)
{
git_SHA1_Update(&ctx->sha1, data, len);
}
i.e. it contains an implicit downcast from `size_t` to `unsigned long`
(before this here patch). With this patch, there is no downcast anymore.
With this patch, finally, the t1007-hash-object.sh "files over 4GB hash
literally" test case is fixed.
Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We want to compile mimalloc's source code as part of Git, rather than requiring the code to be built as an external library: mimalloc uses a CMake-based build, which is not necessarily easy to integrate into the flavors of Git for Windows (which will be the main benefitting port). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matthias Aßhauer <mha1993@live.de>
NTFS junctions are somewhat similar in spirit to Unix bind mounts: they point to a different directory and are resolved by the filesystem driver. As such, they appear to `lstat()` as if they are directories, not as if they are symbolic links. _Any_ user can create junctions, while symbolic links can only be created by non-administrators in Developer Mode on Windows 10. Hence NTFS junctions are much more common "in the wild" than NTFS symbolic links. It was reported in git-for-windows#2481 that adding files via an absolute path that traverses an NTFS junction: since 1e64d18 (mingw: do resolve symlinks in `getcwd()`), we resolve not only symbolic links but also NTFS junctions when determining the absolute path of the current directory. The same is not true for `git add <file>`, where symbolic links are resolved in `<file>`, but not NTFS junctions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The CMakeSettings.json file is tool generated. Developers may track it should they provide additional settings. Signed-off-by: Philip Oakley <philipoakley@iee.email>
Just like the `hash-object --literally` code path, the `--stdin` code path also needs to use `size_t` instead of `unsigned long` to represent memory sizes, otherwise it would cause problems on platforms using the LLP64 data model (such as Windows). To limit the scope of the test case, the object is explicitly not written to the object store, nor are any filters applied. The `big` file from the previous test case is reused to save setup time; To avoid relying on that side effect, it is generated if it does not exist (e.g. when running via `sh t1007-*.sh --long --run=1,41`). Signed-off-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It seems to be not exactly rare on Windows to install NTFS junction points (the equivalent of "bind mounts" on Linux/Unix) in worktrees, e.g. to map some development tools into a subdirectory. In such a scenario, it is pretty horrible if `git clean -dfx` traverses into the mapped directory and starts to "clean up". Let's just not do that. Let's make sure before we traverse into a directory that it is not a mount point (or junction). This addresses git-for-windows#607 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
By defining `USE_MIMALLOC`, Git can now be compiled with that nicely-fast and small allocator. Note that we have to disable a couple `DEVELOPER` options to build mimalloc's source code, as it makes heavy use of declarations after statements, among other things that disagree with Git's conventions. We even have to silence some GCC warnings in non-DEVELOPER mode. For example, the `-Wno-array-bounds` flag is needed because in `-O2` builds, trying to call `NtCurrentTeb()` (which `_mi_thread_id()` does on Windows) causes the bogus warning about a system header, likely related to https://sourceforge.net/p/mingw-w64/mailman/message/37674519/ and to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99578: C:/git-sdk-64-minimal/mingw64/include/psdk_inc/intrin-impl.h:838:1: error: array subscript 0 is outside array bounds of 'long long unsigned int[0]' [-Werror=array-bounds] 838 | __buildreadseg(__readgsqword, unsigned __int64, "gs", "q") | ^~~~~~~~~~~~~~ Also: The `mimalloc` library uses C11-style atomics, therefore we must require that standard when compiling with GCC if we want to use `mimalloc` (instead of requiring "only" C99). This is what we do in the CMake definition already, therefore this commit does not need to touch `contrib/buildsystems/`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Some platforms (e.g. Windows) provide API functions to resolve paths much quicker. Let's offer a way to short-cut `strbuf_realpath()` on those platforms. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we commit the template directory as part of `make vcxproj`, the `branches/` directory is not actually commited, as it is empty. Two tests were not prepared for that situation. This developer tried to get rid of the support for `.git/branches/` a long time ago, but that effort did not bear fruit, so the best we can do is work around in these here tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The vcpkg_install batch file depends on the availability of a working Git on the CMD path. This may not be present if the user has selected the 'bash only' option during Git-for-Windows install. Detect and tell the user about their lack of a working Git in the CMD window. Fixes git-for-windows#2348. A separate PR git-for-windows/build-extra#258 now highlights the recommended path setting during install. Signed-off-by: Philip Oakley <philipoakley@iee.email>
There are no Windows/ARM64 agents in GitHub Actions yet, therefore we just skip adjusting the `vs-test` job for now. Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…for-windows#5971) Every once in a while, there are bug reports in Git for Windows' bug tracker that describe an issue running [inside MSYS2 proper](https://gitforwindows.org/install-inside-msys2-proper), totally ignoring the big, honking warning on top of [the page](https://gitforwindows.org/install-inside-msys2-proper) that spells out clearly that this is an unsupported use case. At the same time, we cannot easily deflect and say "just use MSYS2 directly" (and leave the "and stop pestering us" out). We cannot do that because there is only an _MSYS_ `git` package in MSYS2 (i.e. a Git that uses the quite slow POSIX emulation layer provided by the MSYS2 runtime), but no `mingw-w64-git` package (which would be equivalent in speed to Git for Windows). In msys2/MINGW-packages#26470, I am preparing to change that. As part of that PR, I noticed and fixed a couple of issues _in `git-for-windows/git` that prevented full support for `mingw-w64-git` in MSYS2, such as problems with CLANG64 and UCRT64. While at it, I simplified the entire setup to trust MSYS2's `MINGW_PREFIX` & related environment variables instead of hard-coding values like the installation prefix and what `MSYSTEM` to fall back on if it is unset.
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
…e-and-headless Include Windows-specific maintenance and headless-git
Teach our ARM64 based builds to embed the manifest file correctly. This fixes git-for-windows#4707
…it-for-windows#4410) As per git-for-windows#4350 (comment), the major block for upgrading Git for Windows' OpenSSL from v1.1 to v3 is the tricky part where such an upgrade would break `git fetch`/`git clone` and `git push` because the libcurl depends on the OpenSSL DLL, and the major version bump will _change_ the file name of said DLL. To overcome that, the plan is to build libcurl flavors for each supported SSL/TLS backend, aligning with the way MSYS2 builds libcurl, then switch Git for Windows' SDK to the Secure Channel-flavored libcurl, and teach Git to look for the specific flavor of libcurl corresponding to the `http.sslBackend` setting (if that was configured). Here is the PR to teach Git that trick.
This patch adds a GitHub workflow (to be triggered manually) to allow for conveniently verifying that Git and Scalar still work as intended in Windows Nano Server (a relatively small container base image that is frequently used where a "small Windows" is needed, e.g. in automation ;-)) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…ing (git-for-windows#4528) Another (hopefully clean) PR for showing the error warning about atomic append on windows after failure on APFS, which returns EBADF not EINVAL. Signed-off-by: David Lomas <dl3@pale-eds.co.uk> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
…ows#4700) Windows 10 version 1511 (also known as Anniversary Update), according to https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences introduced native support for ANSI sequence processing. This allows using colors from the entire 24-bit color range. All we need to do is test whether the console's "virtual processing support" can be enabled. If it can, we do not even need to start the `console_thread` to handle ANSI sequences. Incidentally, this addresses git-for-windows#3712.
This topic branch contains a patch that made it into Git for Windows v2.45.1 but not into Git v2.45.1 (because the latter does not come with symlink support on Windows). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This fixes a long-time compile warning turned error by GCC v14. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch adds support for a more Windows-native user-wide config file than `XDG_CONFIG_HOME` (or `~/.config/`) will ever be. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since Git LFS v3.5.x implicitly dropped Windows 7 support, we now want users to be advised _what_ is going wrong on that Windows version. This topic branch goes out of its way to provide users with such guidance. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is an updated version of gitgitgadget#1785, intended for early consumption into Git for Windows. The idea here is to add a new `--full-name-hash` option to `git pack-objects` and `git repack`. This adjusts the name-hash value used for finding delta bases in such a way that uses the full path name with a lower likelihood of collisions than the default name-hash algorithm. In many repositories with name-hash collisions and many versions of those paths, this can significantly reduce the size of a full repack. It can also help in certain cases of `git push`, but only if the pack is already artificially inflated by name-hash collisions; cases that find "sibling" deltas as better choices become worse with `--full-name-hash`. Thus, this option is currently recommended for full repacks of large repos, and on client machines without reachability bitmaps. Some care is taken to ignore this option when using bitmaps, either writing bitmaps or using a bitmap walk during reads. The bitmap file format contains name-hash values, but no way to indicate which function is used, so compatibility is a concern for bitmaps. Future work could explore this idea. After this PR is merged, then the more-involved `--path-walk` option may be considered.
…5171) This is a follow up to git-for-windows#5157 as well as motivated by the RFC in gitgitgadget#1786. We have ways of walking all objects, but it is focused on visiting a single commit and then expanding the new trees and blobs reachable from that commit that have not been visited yet. This means that objects arrive without any locality based on their path. Add a new "path walk API" that focuses on walking objects in batches according to their type and path. This will walk all annotated tags, all commits, all root trees, and then start a depth-first search among all paths in the repo to collect trees and blobs in batches. The most important application for this is being fast-tracked to Git for Windows: `git pack-objects --path-walk`. This application of the path walk API discovers the objects to pack via this batched walk, and automatically groups objects that appear at a common path so they can be checked for delta comparisons. This use completely avoids any name-hash collisions (even the collisions that sometimes occur with the new `--full-name-hash` option) and can be much faster to compute since the first pass of delta calculations does not waste time on objects that are unlikely to be diffable. Some statistics are available in the commit messages.
This introduces `git survey` to Git for Windows ahead of upstream for the express purpose of getting the path-based analysis in the hands of more folks. The inspiration of this builtin is [`git-sizer`](https://github.com/github/git-sizer), but since that command relies on `git cat-file --batch` to get the contents of objects, it has limits to how much information it can provide. This is mostly a rewrite of the `git survey` builtin that was introduced into the `microsoft/git` fork in microsoft#667. That version had a lot more bells and whistles, including an analysis much closer to what `git-sizer` provides. The biggest difference in this version is that this one is focused on using the path-walk API in order to visit batches of objects based on a common path. This allows identifying, for instance, the path that is contributing the most to the on-disk size across all versions at that path. For example, here are the top ten paths contributing to my local Git repository (which includes `microsoft/git` and `gitster/git`): ``` TOP FILES BY DISK SIZE ============================================================================ Path | Count | Disk Size | Inflated Size -----------------------------------------+-------+-----------+-------------- whats-cooking.txt | 1373 | 11637459 | 37226854 t/helper/test-gvfs-protocol | 2 | 6847105 | 1723307 git-rebase--helper | 1 | 6027849 | 15269664 compat/mingw.c | 6111 | 5194453 | 463466970 t/helper/test-parse-options | 1 | 3420385 | 8807968 t/helper/test-pkt-line | 1 | 3408661 | 8778960 t/helper/test-dump-untracked-cache | 1 | 3408645 | 8780816 t/helper/test-dump-fsmonitor | 1 | 3406639 | 8776656 po/vi.po | 104 | 1376337 | 51441603 po/de.po | 210 | 1360112 | 71198603 ``` This kind of analysis has been helpful in identifying the reasons for growth in a few internal monorepos. Those findings motivated the changes in git-for-windows#5157 and git-for-windows#5171. With this early version in Git for Windows, we can expand the reach of the experimental tool in advance of it being contributed to the upstream project. Unfortunately, this will mean that in the next `microsoft/git` rebase, Jeff Hostetler's version will need to be pulled out since there are enough conflicts. These conflicts include how tables are stored and generated, as the version in this PR is slightly more general to allow for different kinds of data. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
I should probably add some tests for this.
Currently not _strictly_ necessary, but still good to have. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
To avoid `check-whitespace` failures when rebasing Git for Windows onto new Git versions, let's limit that job's scope to downstream commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…indows#6063) Git v2.53.0-rc0 included f406b89 (Merge branch 'ar/run-command-hook', 2026-01-06), which caused a regression on Windows. While this merge was reverted for independent reasons in a3d1f39 (Revert "Merge branch 'ar/run-command-hook'", 2026-01-15), it seems worthwhile to ensure that writing to standard error from a `pre-push` hook remains unbroken. The symptom, when running this regression test case against v2.53.0-rc0.windows.1 is that the `git push` fails, with this message printed to standard error: .git/hooks/pre-push: line 2: /dev/stderr: No such file or direct[61/1940] error: failed to push some refs to 'repo1' When that hook runs, `/dev/stderr` is a symlink to `/proc/self/fd/2`, as always, but `ls -l /proc/self/fd/` shows this in the failing run total 0 lrwxrwxrwx 1 me 4096 0 Jan 27 14:34 0 -> pipe:[0] lrwxrwxrwx 1 me 4096 0 Jan 27 14:34 1 -> pipe:[0] lrwxrwxrwx 1 me 4096 0 Jan 27 14:34 2 -> pipe:[0] instead of the expected contents (which are shown when running this against v2.53.0-rc1.windows.1): total 0 lrwxrwxrwx 1 me 4096 0 Jan 27 14:53 0 -> 'pipe:[0]' lrwxrwxrwx 1 me 4096 0 Jan 27 14:53 1 -> /dev/cons1 lrwxrwxrwx 1 me 4096 0 Jan 27 14:53 2 -> '/path/to/git/t/trash directory.t5571-pre-push-hook/actual' This suggests that the underlying reason might be that `stdout` has an exclusive handle to that pipe, and opening `stderr` (which points to the same pipe) fails because of that exclusively-opened `stdout` handle. This closes git-for-windows#6053.
There have been too many challenges supporting `git svn`, including lack of participation in developing/maintaining the required stack. See git-for-windows#5405 for full details. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is the branch thicket of patches in Git for Windows that are considered ready for upstream. To keep them in a ready-to-submit shape, they are kept as close to the beginning of the branch thicket as possible.
Historically, the macOS jobs have always been among the longest-running ones, and recently the `git p4` tests became another liability: They started to fail much more often (maybe as of the switch away from the `macos-13` pool?), requiring re-runs of the jobs that already were responsible for long CI build times. Of the 35 test scripts that exercise `git p4`, 32 are actually run on macOS (3 are skipped for reasons like case-sensitivee filesystem), and they take an accumulated runtime of over half an hour. Furthermore, the `git p4` command is not really affected by Git for Windows' patches, at least not as far as macOS is concerned, therefore it is not only causing developer friction to have these long-running, frequently failing tests, it is also quite wasteful: There has not been a single instance so far where any `git p4` test failure in Git for Windows had demonstrated an actionable bug. While upstream Git is confident to have addressed the flakiness of the `git p4` tests via ffff0bb (Use Perforce arm64 binary on macOS CI jobs, 2025-11-16) (which got slipped in at the 11th hour into the v2.52.0 release, fast-tracked without ever hitting `seen` even after -rc2 was released), I am not quite so confident, and besides, the runtime penalty of running those tests in Git for Windows' CI runs is still a worrisome burden. So let's just disable those tests in the CI runs, at least on macOS. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…or-windows#5923) There have been too many challenges supporting `git svn`, including lack of participation in developing/maintaining the required stack. See git-for-windows#5405 for full details.
Historically, the macOS jobs have always been among the longest-running ones, and recently the `git p4` tests became another liability: They started to fail much more often (maybe as of the switch away from the `macos-13` pool?), requiring re-runs of the jobs that already were responsible for long CI build times. Of the 35 test scripts that exercise `git p4`, 32 are actually run on macOS (3 are skipped for reasons like case-sensitivee filesystem), and they take an accumulated runtime of over half an hour. Furthermore, the `git p4` command is not really affected by Git for Windows' patches, at least not as far as macOS is concerned, therefore it is not only causing developer friction to have these long-running, frequently failing tests, it is also quite wasteful: There has not been a single instance so far where any `git p4` test failure in Git for Windows had demonstrated an actionable bug. So let's just disable those tests in the CI runs, at least on macOS.
Author
|
Artifacts: release assets |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workflow run failed to rebase shears/main.
Rebase Summary: main
From: 0433b7404f (
git svn: remove deprecation note (since it is no longer included in Git for Windows, anyway) (git-for-windows#6142), 2026-03-24) (74c93a9612..0433b7404f)